A simple error re-processor for our Async Spring services
Well hello again my dear readers. Today I wanted to focus on a simple error re-processor that I built for our microservices platform. It is quite a simple solution and we found it worked well with our platform.
So a bit about myself, I work as a backend developer at this amazing start-up called Pryzm Health. We are doing some amazing stuff in the healthcare space and more on that in future posts. We have a microservices based architecture for our platform. I will do a post on our architecture in a different post as well.
As the platform has event based processing, we were in need of an error-reprocessor in case the messages we process off our queues fail somewhere down the line. We wanted to be able to do this in a standard way that can be used across our plethora of microservices and this was the motive to come up with this little library that does just that.
Before we start, the code is available on my GitHub. Go check it out.
Pre-requisites
This library will only work in a Spring aware context as it depends on Spring for some of its functionality. As we use Spring boot within our platform, this lent itself to be a natural fit to write the functionality on.
Also, since we use MongoDB as our primary data store, this was written to cater for it. We could of course easily make it a pluggable data store with a bit more work if needed.
So what does it do exactly?
Well I’m glad you asked that question. A picture speaks a thousand words and saves a thousand + 1 keystrokes. So let me do just that.
The step 1 is all about handling an error scenario. Whenever an exception occurs, we make use of the ReprocessConfig class to write the details of the exceptions to the database. This is then read by one of our Quartz scheduler jobs which picks up all the errors that need to be re-processed and invokes the ReprocessService to process that error and invoke the Spring service from which it failed last time around.
Why did you want this functionality?
On our platform, we rely on AWS SQS for certain event based functionality. If and when some of these fail due to various reasons such as a downstream application being unresponsive, that event is lost in eternity and we are not able to re-process it. We wanted a way to seamlessly handle errors and re-process them in a platform wide manner.
Ok, enough about the what’s and the why’s, tell me how it works
For sure. So let us take an example some code that executes functionality based on some asynchronous event.
In this instance, we are trying to process some data event we receive via our AWS SQS queue. As an exception is thrown out, we make use of the ReprocessConfigService which is exposed as a Spring bean to handle that error and re-submit it back into this same method. All we do is call the reprocess method, passing in the exception and the list of parameters that are used in this method. One thing to note here is the library expects all the method parameters to be Serializable as we write these values to the database(more on this later).
Now that we got that out of the way, let’s look at what the ReprocessConfigService actually does behind the scene.
Ok the code block looks awful on here. Sorry about that!
As mongo does not play nice with dots used as keys, I replace it here with a forward slash when I save the parameter class name for each parameter passed in. The byte stream is then converted to its base 64 encoded format to be saved on mongo.
Now let’s dive into how the reprocessing is done.
The ReprocessService implements the ApplicationContextAware interface in order to get a reference to the Spring context. We use this instance to invoke the appropriate bean to re-process the error.
We load the Spring bean class that we want to invoke using the application context we hooked into. Afterwards it is all reflection where we create the parameters needed to invoke the method on the bean. The following is where the actual magic happens;
Method method = reprocessClass.getMethod(methodName, parameterTypesList.toArray(new Class[]{}));
method.invoke(objectToInvoke, paramValueList.toArray(new Object[]{}));That is about it for this post. I hope you enjoyed reading about it and as always, comments are welcome and you can also leave an issue on my GitHub repo.
Also posted on my Dimashup blog.
