As a developer working on end-user apps that consume platform services through public APIs, it can be very frustrating to develop and test my features due to the environments. The bigger and more complex the platform side becomes, the harder it is to have a stable and working environment that can be depended upon. That is the moment I realized that using mocks is perhaps a better idea, so I can focus solely on my applications rather than the issues of the dependencies within an environment.
After creating multiple mocks I realized that the platform is way more complex than what it seemed to be initially and that it is hard to keep consistency across all of the mocks. Regardless, I managed to get the mocks working and I was proud of my work.
Having said that, the next day the mocks no longer worked, because the dates had changed, so my application did not respond as I would like it to respond. I changed the dates. When I started redoing some of the mocks I had created, I started to get incoherent responses and it got harder and harder to keep them working. Plus, if I wanted a different scenario, I had to create a whole new set of mocks. On top of that, I had more than one application to work on.
So I needed mocks for all of them, this despite the fact that some of the requests were common between mocks. But, as I was trying to keep consistency, it was better to just copy the mocks and keep each project with its own version, so I could change them at any time.
Long story short, the issues just kept popping up. This very quickly became a terrible nightmare that would make anyone want to go back to an unstable environment.
The next logical step was to create yet another application that should accomplish my proposal: one that would allow me to instantly generate mocks that I can use and discard based on a very dear scenario. To achieve this, I created a .Net Console App that I can run through the command line. It starts by converting the user input into a scenario data, which is then converted to platform data, that is later used to mock the endpoints I want by writing the data into mock files. The next step is to read those mock files and use them to send the expectations to the MockServer, which in turn will reply to my applications simulating my platform.
For those who might not know, the MockServer is a simple application that will receive our application http requests and try to reply to them based on the expectations that were previously configured. Said expectations are composed of a request matcher (which can be based on the http verb, url, headers, body, and so on) and an http response (which will be used if a match is done). You can visit their website for more information at https://www.mock-server.com/.
The goal here is to have a simple command line user interface, which is based mostly on true/false or multiple choice questions so it is easy and fast to answer. That input is then saved in a .mgs.json file which I call a scenario. This allows the configuration to be stored, shared and even tweaked in a very convenient way. These are my very special pets.
The main challenge in this step is the trade-off between the level of customization on the one hand and the user input effort on the other hand. I aim for a configuration that covers most of the use cases I have, leaving the edge cases to be done by manually tweaking either the scenario file, the mock files, or the application code itself. This provides a very straightforward and simple configuration by constraining the choices to a predetermined set of data. For example, I allow adding an address to a user by indicating one of the countries from a shortlist I provide. This makes the user interaction very simple (just have to type the number matching the option and enter), but I use a predetermined address for the chosen country locking the possibility for deeper configuration. As I said, it is a trade-off one has to think carefully about.
Once I have a scenario, the application converts it to application data. Following the previous example, it has to convert the country code saved in the JSON file, into a matching address object of the platform side. This step can be very tricky depending on the business rules, the platform data complexity and the way the scenario is mapped to the data. At one point, I had to find a strategy to do it coherently.
To start, I examined the relations between the platform data so I could understand the dependencies. For example, there are currencies and countries. However, the country has a reference to a currency. So I mapped the currencies first, then the countries; that way, I already had the currencies to be included.
However, countries and currencies are still static data, in the sense that, regardless of the scenario I am building, my list of countries and currencies will not alter. The true challenge comes when mapping dynamic data, like orders, which are defined in a scenario. So an order has items, an address, a product, a buyer, maybe a return, maybe it can be cancelled, and so on and so forth. There is no magic solution here, but rather one must clearly understand the business rules and the data relationships. Then start mapping with the one with the least dependencies.
I developed the application in C# so I could have the strongly typed objects from the platform DTOs. This helps in ensuring coherence among data, as well as the current platform contract. I could discard the strong typing which would make the mapping so much easier. Or even better, I could discard the platform data step altogether and convert from scenario to mock files. However, that would make maintenance and data coherence so much harder to keep. Once again, trade-offs...
The next step was to convert the platform data to mock files. The mock files are JSON files containing the data required to define one expectation on the MockServer. This means there is a JSON file for each desired expectation. These are my not-so-special cattle.
The only goal of these files is to perform manual tweaks and check what data is actually being used (like, what order ID was it used since that can be a random value). Since they contain volatile information (dates, for instance), the files should be discarded after use (since future dates will become past dates in time) and regenerated when necessary (that is why the scenario exists).
Although this step seems pretty simple, it still has some complexity. Firstly, I have to be aware of which endpoints are to be mocked and, secondly, how they actually respond. For instance, I might have an endpoint for requesting country information that uses the object Country as a response and another endpoint to request an address which answers with an object Address that contains a Country object, however, the information it contains may not be fulfilled completely as is in the first endpoint. If the mock contains all the information, it will probably not cause any harm to my application, since the DTOs objects are the same, but I, as a developer, might start to think that it provides some information that it actually does not and lead me to problems down the line.
From MockServer expectations to my application responses
Finally, my applications can start to request data from my MockServer rather than my testing environment. I use Fiddler to see the requests and debug any issues that might arise. Also, the MockServer logs provide some intel. The development and testing of my applications can be done with ease and multiple scenarios tried out.
The Limitations
Of course, everything comes with its own limitations and this is no exception. There are quite a few, to be honest, but I will talk about my major concerns in the following topics.
The first issue I want to talk about is the state. For instance, when I see a list of orders and I cancel the first, I would expect the order to remain cancelled even upon a refresh. However, mocks have no state and will reply with my original list of orders.
The MockServer is capable of dynamically responding to the request by using callbacks. However, that would require (in the approach I have been presenting) the Configurator to keep executing in order to handle the callback. Although that would work in local use, when using automated pipelines, testing environments, and so on, it might be an unfeasible option. Regardless, the callback is still very limited and would not have the context from outside that request.
Another approach would be to specify the order by which the mocks are returned. Like, the first time the orders list is requested, it returns the original list, but on the second time, it returns the list with the first order cancelled. This could work nicely in integration tests, since the interaction flow is strict and that is the way it always happens. However, while developing and testing the applications, I want to be able to freely interact with them, rather than being forced to follow that exact interaction. Besides, that would still hide possible errors, since the cancellation could not have been done or failed and it still would show as cancelled on my second request.
The second issue is the mock spreading. The problem here is that it might become hard to interact with unmocked services and applications from a mocked environment, which forces the mocking of the remaining services.
For instance, my applications require the user be authenticated to see their orders and so on. So I mock the access token, since the user's orders are mocked (the user might even exist but I want to test them with multiple order configurations). However, I want to run my applications within the testing environment, which also contains the remaining website. Let's imagine I want to see a product that has nothing to do with my applications. When I attempt to do it, I will face problems, since it will check my access token, as I am a logged user, only to find out that, despite the user actually existing, that token is not valid (as it was not created by the identity server) and errors start to pop up.
The natural consequence of this fact is to start mocking every other service as well or just accept that I can only use my mocked applications. I am not saying it is impossible to flow between mocked and unmocked contexts naturally, but depending on one's environment that might become very challenging.
Besides those issues, there are also some pitfalls to be mindful about. Failing to understand and deal with them might lead to unexpected issues.
The first thing to be careful about is that mocks might not actually match platform data. What I truly mean here is that mocks will always match what I told them to match, the way I told them to match. If I do not keep track of the changes from the platform side, I might lead people to believe that my application is working nicely since it is matching my expectations, when, in reality, it will crashin an unmocked environment since the platform is not providing the same info.
Also, since this is a mock generator, which is based on business logic implemented on it, it might have bugs where the generated mocks are not what the platform actually does in that specific case. The more of an edge case it is, the more likely it is that the generation logic used might not match.
Lastly, some mock responses might be different on purpose to allow more flexibility. For instance, I perform translation requests by sending a set of keys. I can generate a mock that matches these keys and returns those translations (as it works in an unmocked environment). However, if I add one more key later on, I will have to update the generator to include that other key. Also, different applications might even request some of the same keys, but I need a mock for each of those requests. This can become a burden to maintain. So a strategy to avoid this is to accept any keys and reply with all the existing translations, since the application itself is then matching the reply to the requested translations. This is clearly not working given differences in data in the unmocked environment and might lead to hidden mistakes (imagine you are requesting the wrong keys but expecting the right ones, it will work in the mocked environment and fail on the unmocked one).
Regardless, the use of mocks should never completely replace testing on an environment with more reliable or live-copy data.
The other pitfall to avoid is the idea that, once the generator is created, I do not need to worry about it anymore. This is not the case at all, since any interaction changed between the applications and the platform will most likely force me to update the generator to include that other endpoint or at least review if this new interaction is working as expected with the preexisting business logic implemented. This might become a burden, particularly if a more strict approach to the mock matching was done.
With all that said and done, the mock generator has been a great addition to our toolset as it increased productivity and helped to mitigate several issues. Nevertheless, I always look to the future and how this can be improved.
My proposal for the future is to go even further than the mock generator and move to a platform emulation application. It would be a special service that would be running alongside my applications (replacing the MockServer), but that could emulate the platform itself without the business rules. This would allow the mocked environment to be stateful and ease up proxying mechanisms to navigate through mocked and unmocked contexts and so on.
For the time being, however, let's just enjoy the generator.
Besides that, there is also the challenge of the request matchers. I want my application to give me my access token if my application sends both username and password correctly, but I want an error otherwise. Also, I don't really care if the username precedes the password or the other way around in my request body, but the MockServer matcher does. So perhaps using a regex might be necessary. Also, I might want to have some flexibility, like when I am cancelling an order; I don't really care about which reason is used as my reply will be the same regardless, as long as one is sent. So maybe using a JSON schema might be useful. There is, once more, a trade-off in the expectation matching flexibility and a challenging work that requires the study of the MockServer matchers carefully.
