Send Message To RabbitMQ in Transaction

What do I meant by sending message in Transaction ?

Sending message in transaction mode mean if the transaction is succeed then message/event should be published. But in case of transaction failure or rollbacks the message/event shouldn't be published.

In many Enterprise Application transaction is the bread and butter of the architecture. Traditional architecture is highly dependent on transaction. Even in new era of Microservices  there comes the need to send the messages across the components in some what transaction way to maintain the data integrity across the component.

Let's take some simple example of microservice: 

Here there is simple use case once the order is placed it will notify the user about the order confirmation. In microservice architecture, order and notification are two different component having their own JVM and DB. Definitely this flow should be async and also consistent. 

Here there is a need/requirement  that notification should not trigger if the order placement fails. Notification should only be delivered if the order transaction succeed. So there is need to send message to RabbitMQ  ( Any Broker) in transaction mode.

Currently RabbitMQ  doesn't participate/support the transaction there is way for two phase commit, but there are pitfalls which I am not going to discuss here. 

So question comes how we can leverage the RabbitMQ power to use it under transaction answer is Orchestration. 

Based on my experience there are following options each having their pos and cons.

Option -1 Orchestration Using RabbitMQ Itself. 

For this option create some broker framework which can get embedded into the application or it can run standalone.  

1. Here application create the payload and generate unique hash value.
2. Under transaction it send message to temp queue and writes the hash value to some table in DB.
3. Broker Framework listen to the temp queue and upon receive of any message it checks for the hash value stored in step-2.
4. If the hash value is present it forward the message to original queue and if not present it pushes back to the temp queue.
5. After no of retry the message will get discarded / moved to error queue reason being transaction might not succeed and hash value is never written to DB. 

Challenges:

• If the transaction takes longer the broker will receive the message before the transaction ends and due to which retry might get triggered more frequently.
• If volume is high as in many case message queued back after retry will take longer time to reach destination.

+Ve:

• Highly scalable. In case of embedded it autoscale as the node scale.
• Easy to monitor and maintain.
• Event Driven

-Ve:

• Data Cleanup
• More delay in case of longer transaction time

Option -2 Orchestration Using Poller.

This is very traditional approach where poller will poll the transaction data and publish the same to queue.

1. App stores the payload data in transaction. ( DB Isolation Level read committed).
2. Poller polls the data and push it to queue.
3. Once the data is push the data is cleared from DB via update / delete.

Challenges:

• Poller scaling.
• Heavy read and write operation in DB.
• Poller memory issues (How much data to poll ).

+Ve:

• Truly transaction.

-Ve:

• Difficult to scale.
• High Maintenance and tuning requirements. 

Option -3 Orchestration In Memory.

Here the message will be stored in memory and will delivered to MQ after transaction succeed.

1. Application will store the message in memory till the transaction completes.
2. Once the transaction is completes in memory message is pushed to queue.
3. Shutdown hook is implemented which will write the in memory data to disk if shutdown is triggered so that once the application comes up it will able to check the transaction and push to queue.

Challenges:

• Transaction complete triggering.
• Memory management to some extent.
• Shutdown and Restart.

+Ve:

• Very easy to implement ( Remember to take concurrency effect).
• Scalable.

-Ve:

• Rollback cleanups
• High maintenance in multi node scenario.

Option -4 Orchestration via coordinator.

Here the message will be send to rest broker which can act like coordinator which will coordinate for the transaction processing.

1. Application submit the payload to the broker which return the unique token back in response.
2. On post processing of the transaction the application submit the token back to the broker, and broker send the message to queue.
3. Broker work on time based it will hold the token for stipulated time and will discard the message if the token is not received in stipulated time.
4. Optionally if broker needs to scale then broker might store the message temporarily in some distributed cache like redis etc.

Challenges:

• Transaction completing triggering. ( What if the node goes down after the transaction is complete and was not able to submit the token).
• Complex development and monitoring.

+Ve:

• Scalable

-Ve:

• High Monitoring and fallback.

Conclusion:

Which option to choose depends on the requirement.  Example Option-3 is suitable for low volumes and option-1 and option-4 can be used for high volume loads. Option-2 which is pure transactional stuff, but not event driven and difficult to scale needs rethinking before adopting it.

If you believe in KISS (Keep it simple Stupid) principle then go for Option-1. As per me this is easiest to implement, fault tolerance and highly scalable

Before starting, evaluate various options via POC and find out the best which suits you. 

Happy coding.