Monthly Archives: September 2021

Building Microservices with Event-Driven Architecture

In this post, we will discuss how we can build microservices with event-driven architecture. As part of the post, I will also show an example of an event-driven microservice. If you don’t know what a microservice is, you can start with my primer here.

Microservices – Event-Driven Architecture

Traditionally, we would use a REST Based Microservice. In this microservice, a client would request data and then the server would respond with the data. But there were disadvantages in that client has to wait for the server to respond. A server can be down or serving other requests, in-process of delaying the response to the current client requests.

In short, when a system becomes slow because of synchronized connections, we can use event-driven architecture to make the system asynchronous.

Event-Drive microservices use an eventually consistent approach.  Each service publishes event data whenever there is an update or transaction. Other services subscribe to this service publishing events. When these subscribed services receive an event, they update their data.

A simple example of this approach: When a customer redeems a gift card, a single redemption event is created and consumed by different services.

  1. A Reward Service that can write a redemption record in the database
  2. A Customer receiving getting an item bought through a gift card
  3. A Partner Service verifying the gift card and allowing the redemption and accordingly processing of the item that the customer bought.

Event-Driven architecture is either through queues or the pub-sub model. In Pub/Sub model, a service publishes the event, and subscribed services consume that event. It is not much different from what queues and topics do.

Benefits of Event-Driven Architecture

  • Loose Coupling – Services don’t need to depend on other services. Considering the architecture is reactive, services can be independent of each other.
  • Asynchronous – A publishing service will publish the event. A subscribing service can consume the event whenever it is ready to consume. The major advantage of asynchronous architecture is that services don’t block resources.
  • Scaling – Since the services are independent, most services perform a single task. It becomes easier to scale as well to find out bottle-neck.

Drawbacks of Event-Driven Architecture

Every design is a trade-off. We do not have a perfect design in distributed systems. With event-driven architecture, one can easily over-engineer the solution by separating concerns.

Event-Driven architecture needs upfront investment. Since the data is not necessarily available immediately, it can cause some concerns with transactions. Eventual consistency can be hard to investigate if there are issues with data. There can be possibilities of duplicate events, resulting in duplicate data. Event-driven models do not support ACID transactions.

Framework for Architecture

Irrespective of those drawbacks, event-driven architecture is fast and delivers results successfully. So the next question arises what framework to choose to build this architecture. Currently, there are two choices

  • Message Processing
  • Stream Processing

Message Processing

In message processing, a service creates a message and sends it to the destination. A subscribing service picks up the message from that destination. In AWS, we use SNS (Simple Notification Service) and SQS (Simple Queue Service). A service sends a message to a topic and a queue subscribing to that topic picks up that message and processes it further.

SNS and SQS are not the only frameworks out there. Message queues use a store and forward system of brokers where events travel from broker to broker. ActiveMQ and RabbitMQ are the other two examples of message queues

Stream Processing

In stream processing, a service sends an event and subscribed service picks up that event. Nevertheless, events are not for a particular target.

Usually, a producer of events emits events and can store them in storage. A consumer of events can consume those events from the data storage. The most popular framework for stream processing is Kafka. Basically, it follows a pub-sub model.

Above all, stream processors (like Kafka) offer the durability of data. Data is not lost and if the system goes offline, it can reproduce the history of events.

Demo of Event-Driven Architecture Based Microservice

As part of this demo, we will implement a Spring Boot application along with the ActiveMQ message broker service.

ActiveMQ Messaging Service

ActiveMQ is an open-source message broker. Presently, it supports clients written in Java, Python, .Net, C++, and more.

Download the ActiveMQ from here. Once, you extract the downloaded folder on your machine, you can go to bin directory to start the ActiveMQ server with a command activemq.bat start. This will start the ActiveMQ server at http://localhost:8161.

Sender Application with Spring Boot

Now, let’s create a Message Sender application using Spring Boot. We will need the following dependencies


dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-activemq'
	implementation 'org.springframework.boot:spring-boot-starter-web'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

We will add JMS Configuration to create an ActiveMQ Queue.


@Configuration
public class JmsConfig
{
    @Bean
    public Queue queue()
    {
        return new ActiveMQQueue("demo-queue");
    }
}

This creates a bean for our queue demo-queue. To send message to this queue through our sender application, we will create a REST API as follows:


@RestController
@RequestMapping("/v1/betterjavacode/api")
public class MessageController
{
    @Autowired
    private Queue queue;

    @Autowired
    private JmsTemplate jmsTemplate;

    @GetMapping("/message/")
    public ResponseEntity sendMessage(@RequestBody String message)
    {
        jmsTemplate.convertAndSend(queue, message);
        return new ResponseEntity(message, HttpStatus.OK);
    }

}

Subsequently, we have injected queue and jmsTemplate beans in our RestController so we can send the message.

On the other hand, we will also have a receiver application which will be a destination service or consumer service consuming the message from the sender application.

Create a message consumer class in our receiver application


@Component
@EnableJms
public class MessageConsumer
{
    private final Logger logger = LoggerFactory.getLogger(MessageConsumer.class);

    @JmsListener(destination = "demo-queue")
    public void receiveMessage(String message)
    {
        // TO-DO
        logger.info("Received a message = {}", message);
    }
}

The annotation of @JmsListener with destination makes the application to listen to that queue. @EnableJms enables the annotation @JmsListener.

We still need to add ActiveMQ properties so that both applications know where ActiveMQ server is running. So, add the following properties to application.properties


spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.user=admin
spring.activemq.password=admin

Now start both of the Spring Boot applications. Sender Application is running on 8080 and Receiver Application is running on 8081.
Event-Driven Architecture Microservices - Sending Message

Now if we check the logs of receiver application, we will see that it has consumed that message from ActiveMQ queue demo-queue.

Microservices - Event-Driven Architecture - Receiving Message

We can also see the status of queue in ActiveMQ server.

Apache ActiveMQ Event-Driven Architecture

Here, you can see there have been two messages that the queue has received from the sender and delivered to the consumer.  The code for this demo is available on my github repository.

Conclusion

In this post, I discussed Event-Driven architecture for microservices. We also discussed the benefits and drawbacks of this architecture. At last, we showed how we can use ActiveMQ to set up an event-driven architecture-based microservice for asynchronous communication.

On another note, if you still haven’t bought my book for Spring Security, you can buy here OR you can read about it more here.

References

Event-Driven Microservices using ActiveMQ – ActiveMQ

On Being A Senior Software Engineer

In this post, I cover what it means to be a senior software engineer. When I say senior, it means anyone other than Junior, Associate, or Software Engineer. So it can include Senior Software Engineer, Staff Software Engineer, or Principal Software Engineer. If you are a Junior Developer, you can read my previous post on what makes a good junior developer.

Staff and Principal Engineers are usually on the same level as Engineering managers without anyone reporting to them. But this can vary in organizations. So, I am not going to on that but will focus on what all these engineers do and what they can do better.

Two Career Paths

Most Software Organizations have two career paths for all engineers.

  1. Individual Contributors
  2. Management

Individual contributors usually keep the engineering team on the engineering path while managers keep the team aligned for the overall goal of the team. Most senior engineers usually get a choice after a certain level of engineering experience if they want to be individual contributors or become managers. It can also depend on the performance.

Staff and Principal Engineers are individual contributor roles. Usually, those engineers remain on that path for the rest of their careers.

All three types of senior engineers have a certain role to play in the team, but I will not go over that much, but what they do and how they are different from Junior engineers.

Not a 10x Engineer

Most Senior engineers can be considered 10x engineers. If you don’t know what a 10x engineer is, then search for it. It’s a famous meme. Most senior engineers can definitely close a lot of tickets and code better. But that’s not their only role and they are not really 10x engineers.

A great senior engineer makes the whole team great by advocating the best practices. This is where their experience comes in handy. Senior engineers contribute in the following areas – Coding standards, coding review guidelines, system design guidelines, and understanding of the system. They become a mentor for junior engineers. A good senior engineer can distinguish between engineering language and product language. She can decipher product requirements from business to engineering and communicate engineering challenges to products. She can become a bridge between business and engineering.

One key skill a senior engineer possesses is communication. Communication to get the team to do better and focus on the goal. Communication to make sure the business understands the engineering side. Nevertheless, interpersonal skills are important for senior engineers.

Mentoring

Another important role a senior engineer does is to mentor junior engineers. A senior engineer may not hold one-on-one with juniors, but he will guide them through code review, understanding of the system, and making critical decisions in system design as well in code. He will also showcase his own leadership skills when the team needs guidance. If a team is struggling, there is a large role a senior engineer has to fill in.  If a team is doing well, a large credit goes to the senior engineer as well.

Overall, a senior engineer is a cheerleader of the team, he boosts the morale of the team. A senior engineer also guides the new developers who join the team. A senior engineer actually showcases the values the company has adapted.

Engineering Initiatives

A key skill a senior engineer possesses is to look at any system and find the pain points. A senior engineer understands that the team is the customer and she must solve the painful problem. A senior engineer can go out of her way to solve some of these problems and make the team better performing.

She also keeps herself up to date with the new challenges and changes in technology. Foresightedness is a skill, but it only comes with experience. A senior engineer finds the problem in the system and solves them. Example – How to use a circuit breaker in rest call.

Leadership

A senior engineer is a subject matter expert of the system he has worked on. If there is an issue, he doesn’t have to visit the code every time to know where the issue is. Usually, his knowledge of the system is so strong that he can fix the issue quickly. But, there can be situations where there is no solution and a senior engineer takes that as a leader to communicate to the business. Convincingly, he also leads the efforts to implement any new features. A senior engineer is a leader and he finds his way to remove obstacles to the team’s progress.

Conclusion

In conclusion, a senior engineer is the glue that holds a team. A manager usually gives a free hand to senior engineers in many aspects because of their high agency character as well as leadership qualities.

If you enjoyed this post, you can subscribe to my blog here. Also, if you are interested to learn more about Spring Security, you can buy my book Simplifying Spring Security.