Though I hardly heard the term ‘monolithic’ when I started my career in 2004, I contributed to applications which followed monolithic architectural pattern. I and my teammates worked on the same codebase, each owned 1 or 2 modules, packed everything in an archive (typically a war/ jar) and deployed to an application server like jBoss.
In 2006, service-oriented architecture (SOA) took the IT industry by storm. Numerous companies adopted this architecture style as a way to bring better business by communicating and collaborating with each other. A plethora of third-party products were in the market to help companies move on with SOA. We used WSDL, Apache CXF, Axis 1/2, SOAP Protocol, Messaging with ActiveMQ, RabbitMQ etc… The service publisher and consumer contract was well defined. In short, it standardises reusing and exposing services to outside world using WS-* standards and connecting the components using ESBs.
In recent past I got an opportunity to technically lead a client engagement where I had to decompose a monolithic application into different microservices and deploy them in docker containers in AWS cloud platform. And thought its worth sharing it
1. Monolithic Architecture
To know about anything new, it would be good to analyse it with its precursor and here it is the ‘Monolithic Architecture’. Here is the fair picture of the monolithic application (considering few modules) which was serving the customer for years.
- A single Java WAR file or single directory hierarchy of Rails or NodeJS code
- Applications written in this style are extremely common. They are simple to develop since our IDEs and other tools are focused on building a single application.
- These kinds of applications are also simple to test. We can implement end-to-end testing by simply launching the application and testing the UI with a testing package such as Selenium.
- Monolithic applications are also simple to deploy. We just have to copy the packaged application to a server.
- We can also scale the application by running multiple copies behind a load balancer.
1.1 Issues being Monolithic:
All these advantages we discussed worked well in the early stages. The application started expanding by adding more and more features. After some time it did work the way business stakeholders wanted as it grew too big to manage.
- The large monolithic code base intimidates developers, especially ones who are new to the team. The application can be difficult to understand and modify. As a result, development typically slows down. Moreover, because it can be difficult to understand how to correctly implement a change. The quality of the code declines over time. For example, when I joined in my previous client assignment, I was new and the team was also relatively new. When we got single liner requirements, the team had clue on how to implement but had no clue of the complete impact of these features with respect to the existing modules. And the same issue started repeating. Being a technical lead, I had to vet the features with lot of difficulties and failures. That’s when we wanted to bring the change in the architectural level.
- Overloaded IDE – the larger the code base the slower the IDE. Some of the modules of the monstrous application were opened once in blue moon in our favorite IDE.
- Overloaded web container – the larger the application the longer it takes to start up which considerably affects the development and deployment time.
- Continuous Delivery is difficult – A large monolithic application is also an obstacle to frequent deployments. In order to add a feature to one component, we have to redeploy the entire application. This will interrupt background tasks (e.g. Quartz jobs in a Java application), regardless of whether they are impacted by the change. There is also a chance that components that haven’t been updated will fail to start correctly. As a result, the risk associated with redeployment increases, which discourages frequent updates. Production deployment happens with extreme care.
- Scaling the application can be difficult – We cannot scale each component independently. Monolithic applications can also be difficult to scale when different modules have conflicting resource requirements. For example, one module might implement CPU-intensive image processing logic and would ideally be deployed in Amazon EC2 Compute Optimised instances. Another module might be an in-memory database and best suited for EC2 Memory-optimised instances. In this case how would you select your instance type? – Difficult to answer!.
- Affects Reliability – Because all modules are running within the same process, a bug in any module, such as a memory leak, can potentially bring down the entire process. Moreover, since all instances of the application are identical, that bug will impact the availability of the entire application
- Requires a long-term commitment to a technology stack – a monolithic architecture forces developers to be married to the technology stack which was chosen at the start of development . In my team, I was not able to retain good engineers as the application used a framework that became obsolete when we started working on it. Then it was even more challenging to incrementally migrate the application to a newer stack. It’s possible to adopt a newer platform framework but we have to rewrite the entire application, which is a risky undertaking and business stakeholders showed red signal many times.
2. Microservices the rescue:
Have seen almost all of the mentioned disadvantages when I and my teammates worked on it. Microservice is an architectural style that structures an application as a collection of loosely coupled services, which implement business capabilities. Instead of building a single monstrous, monolithic application, the idea is to split our application into set of smaller, interconnected services. Each functional area of the application is now implemented by its own microservice.
We get the below points from the diagram,
- Each bubble represents a single microservice
- There are more bubbles of same service – they are multiple copies of the same service to ensure high availability
- All the bubbles are loosely connected meaning they communicate with each other.
- All the backend services expose REST APIs
- Web application is also split into a set of simpler web applications such as one for notifications UI and other for authentication
- Continuous Delivery – is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released to production at any time. It aims at building, testing, and releasing software faster and more frequently. Since the risk associated with deployment is less – (meaning if there is an issue, only one service would get affected but the system as a whole is up for accepting requests) it is done with extreme care as we do in monolithic application. If something goes wrong, we followed roll(back) forward. Note: It is sometimes confused with ‘Continuous Deployment’ which means that every change is automatically deployed to production. Continuous delivery means that the team ensures every change can be deployed to production but may choose not to do it, usually due to business reasons.
- Increased Team Autonomy – Each service implements distinct features or functionality such as authentication, notification etc…It facilitates autonomy by covering one feature. Therefore a team can fully own it instead of it having to be owned by multiple teams. I am able to recollect the quote “too many cooks spoil the broth” which suits well here. Adding features in one service and deploying to production sooner without worrying about the internals of other services (unless it is an API change) is what agile software development is all about.
- Product Mentality Over Project – It is about a cultural change within our team. In project-based work, a team typically gets one shot to complete a large chunk of work — that’s a lot of pressure to get it right! On top of that, if the project was considered a failure, the accountability fell mostly on the shoulders of the project manager. In product-based work, the entire team is accountable. We get one shot for a short distance jump, review happens and multiple shots to cover the long distance with the quality in place. We’re all committed — yippee!
- Scale Independently – Microservices Architecture pattern enables each service to be scaled independently. We can deploy just the number of instances of each service that satisfy its capacity and availability constraints. Moreover, we can use the hardware that best matches a service’s resource requirements.
- Localised Error – A single micorservice is not working for X amount of time for any good reason – that’s fine if the whole system is not down. Adrain Cockcroft ( director of cloud architecture at Netflix when it started moving to microservices) explains that these API services would break at different times causing smaller, localized problems, rather than losing “the whole system at once”, which, in turn, resulted in less manual intervention.
- Flexible Tech Choice – This freedom means that developers are no longer obligated to use the possibly obsolete technologies that existed at the start of a new project. When writing a new service, we, developers have the option of using trendy tech tools and framework. It also enables an organisation to evolve its technology stack. With this in place, I was able to retain and engage my engineers happily in the team. Yippee!
2.1 Decomposed Microservices in cloud:
Let us see, in reality how these micro services are connected in cloud environment. Though it is not the exact situation, this is the pilot for us to grasp the tonic quick.
- Access to the backend services is mediated by an intermediary known as an API Gateway/ Proxy Server. The API Gateway is responsible for tasks such as load balancing, caching, access control, API metering, and monitoring, and can be implemented effectively using NGINX/HAProxy etc…
- Then comes the naming server which does the DNS (Domain Name Service) operation to resolve the endpoints. In AWS, Route53 does this job. No hard coded host and port information.
- Each service instance is a Docker container.
- In order to be highly available, the containers are running on multiple instances in front of a load balancer such as Nginx, ELB etc…
- Rather than sharing a single database schema with other services, each service has its own database.
2.2 NO Hard Rules:
Though the microservices architecture has matured good enough, there are no hard and fast rules to follow unlike SOA. It is formed by extracting all good berries from several known concepts and connecting them to support the current business needs. There are several inspirations but I would like to highlight few of them.
- Single Responsibility Principle (SRP) – In OOP (Object Oriented Programming), it is a principle that states that every class should have responsibility over a single part of the functionality and that responsibility should be entirely encapsulated by the class. So it makes sense to apply the SRP to (micro) service design as well and design services that are cohesive and implement a small set of strongly related functions.
- Common Closure Principle (CCP) – In OOP, it is another principle which states that classes that change for the same reason should be in the same package. The application should be decomposed in a way that most new and change requirements only affect a single service. For example you need to add ‘forgot password’ feature. Where do you go and add it? Since it has strong relation with authentication, will have to add it in ‘authentication’ service.
- SOA: “Microservices is SOA, for those who know what SOA is.”It is pretty much service orientation without few complicated WS-* standards, ESBs etc.
- REST APIs – Instead of writing complicated WSDLs by following WS-* standards and XML object as a medium of transfer, REST APIs are simple to follow as we are living with hypermedia standards for years and years.
2.3 Drawbacks of Microservices:
Microservices are not silver bullets. Like every other technology, this pattern also has several drawbacks of which I would like to highlight some of them.
- Increased Resource Usage – Each microservice will have multiple runtime instances. In contrast to monolithic application, a microservice application typically consists of a large number of services. For example, Hailo has 160 different services and Netflix has more than 600, according to Adrian Cockcroft. If a service needs to be highly available minimal requirement is having 2 instances of the service running in front of a load balancer. Just think about the number of instances that these giants need. So the initial investment to run these applications is high because all the independently running components need their own runtime containers with more memory and CPU.
- Queries that join data from multiple services – The databases are segregated and as a result, it is no longer straightforward to implement queries that join data from multiple services. These kinds of transactions are trivial to implement in a monolithic application because there is a single database. Chris Richardson comes with ‘Saga’ to the rescue.
- Consistency in distributed data management – In a microservices based application, we end up having to use an eventual consistency-based approach, which is more challenging for developers. Because developers also have to write code to handle partial failure, since if one transaction fails, the previously committed transactions should be reverted.
- Testing Complexity – Testing a microservices application is also much more complex. Let us consider a trivial example, ‘User Signup’ use-case. A QA engineer has to test if the user gets created by testing the ‘user management service’. But it does not end with single microservice. He needs to test if the ‘signup successful’ email/ SMS notification is sent by testing the ‘notification service’. Also, he has to test whether all the events ‘USER_CREATED’, ‘MAIL_SENT’ etc.. are captured in sequence in ‘eventing service’. Somewhere I read, loading Amazon homepage takes help of more than 150 microservices. So, it increases the testing complexity.
- Dev-Ops Complexity – We need to have a mature DevOps team to handle the complexity involved in maintaining microservices based applications as deploying a microservices based application is much more complex. A monolithic application is deployed on a set of identical servers behind a traditional load balancer. We as developers handle the deployment with minimal ant/ maven script knowledge. In case of microservices, the infrastructure should be managed by code as the number of moving parts are more and more and complicated. In addition, we need to implement a service discovery mechanism which enables a service to discover other services without feeding the the locations (hosts and ports) of any other services in code/ configuration files as they are already fed in the infrastructure.
- Team Communication & Formal Documentation Overhead – Teams need to make sure that updates in one team’s service does not break another’s team functionality. We find this problem in monolith architecture applications, too. Every individual running a component application needs to keep updated interface documents like swagger all the time. The documentation should talk for your service and not the developer. Since my engineers were busy in implementation, I started documenting the microservices (though is an overhead)
- Increased Network Communication – Independently running components interact with each other using the network. Such systems require reliable and fast network connections. Say we need data which depends on 2 different services. Service A communicates with Service B where the data transfer goes thru marshalling and un-marshalling processes. This is not the case with monolithic as both the services are in the single process which share the resources.
2.4 Best Practices for Microservices:
We in agile world, learn from failures faster and move on. There are few guidelines which are written after seeing the failures by many organizations. It is good to follow them so that we do not get caught in known pits and will try to solve a set of new problems.
- Separate Data Store for each microservice – Using Spring Boot/Docker doesn’t mean we are doing microservices. We need the team for each microservice to choose the database that best suits the service.Need to take a hard look at the domain, define the data boundaries with lot of attention to details and segregate them to enable loose coupling. Christian Posta (Chief Architect @ Red Hat) in one of his articles quotes “each microservice should own and control its own database and no two services should share a database.”
- Expect Eventual Consistency across data store – No matter how you look at it, consistency is hard in a distributed system. Instead of fighting for it, a better approach is eventual consistency. No ACID (Atomicity, Consistency, Isolation & Durability) only BASE (Basically Available, Soft state & Eventually consistent). Consider the two services – user registration and evening service. The eventing service gets an asynchronous feed of updates from other services like user registration service. This means that whenever an update happens in the user registration service, the eventing service finds the data a few seconds later. That’s fine and we can live with that the delay.
- Do a separate build for each microservice – So that it can pull resource files from the repository at the revision level appropriate to the service. Deploying microservices in containers is important because it means we just need one tool to deploy everything. As long as the microservice is in a container, the tool knows how to deploy it. That said, Docker seems very quickly to have become the de facto standard for containers.
- Treat servers as stateless and Enable Monitoring – I heard from one of my colleagues saying, this server is for image processing and that server is for in-memory database operation etc… by this we are building sentiments around the hardwares which should not be the case. Servers should be interchangeable members of a group. Similar servers perform the similar functions, so we don’t need to be concerned about them individually. Our concern should be that the servers are responsive in nature. We can use auto-scaling to adjust the number of servers up and down based on the demand. If one stops working, it’s automatically replaced by another one. Cockcroft’s analogy is that we need to think of servers like cattle, not pets. If we have a machine in production that performs a specialized function, and we know it by name, and everyone gets sad when it goes down, it’s a pet. Instead we should think of the servers like a herd of cows. What we care about is how many gallons of milk we get.
- No to nano-services – Nanoservice is an anti-pattern where a service is too fine grained. A nanoservice is a service whose overhead like communications, maintenance etc… outweighs its utility. Say for example, would you really go for separate services for implementing the functionalities like ‘authentication’, ‘forgot password’, etc… The answer is ‘NO’. We should collate them in a single service otherwise, it would be considered as ‘nano-service’.
- Standard REST APIs & Tools to Automate documentation – The APIs should follow standards. REST APIs really do not impose any restriction but by following the standards other services will be able to happily communicate with our services. Convention wins!. For example, all accounts management operations should be categorised under ‘/accounts’ namespace with GET/POST/PUT HTTP methods. No to ‘accounts/getAccounts, accounts/updateAccount etc….Follow good articles from enterprises like apigee, WSO2 to know more about the standards.
- Effective Logging with Correlation ID – Every entry and exit points to & from any micro service must be logged to monitor, analyse and debug. This helps to localize and locate issues even in production. Since we recommend to use container for a microservice, it is also a notable point that we should not lose the microservice application level logs when we destroy the up and running container for any good reason like new service deployment. Need to persist the service level logs outside our container; there are good tools available in the market to centralize the logs like SumoLogic, LogEntries etc…and process them. Moreover, we discussed that each request is served by more than one microservices. In such cases, how do we track a request which is served by Service A which internally invoke Service-B which internally invoke Service-C. Meaning at any point of time, how do you trace a request? Create an unique identifier and attach it to the request when it hits the first service (say Service A) and then carry forward the identifier along with your request. This identifier, correlation ID helps to trace a request at any given point of time.
2.5 Success Stories adapting to Microservices:
Well known internet services such as Netflix, AWS, GroupOn etc… initially had a monolithic architecture and successful with microservices. Now it is story time!.
- Netflix – After a single missing semicolon led to a major database corruption in 2008, Netflix understood they had to change their approach. Cockcroft understood Netflix needed to push away from monolithic vertically-scaled datacenters and move towards cloud-based microservices architecture. By segmenting their data into a network of horizontally connected API services, Netflix could minimise the frequency and severity of code errors and related issues. This is known as separation of concerns, or as Cockcroft calls it, “Loosely coupled service oriented architecture with bounded contexts”. Netflix created 30+ separate engineering teams capable of independently developing and implementing new services on a variety of new platforms. Netflix structured its microservice architecture to ensure its data would be secure, persistent, and not susceptible to singular points of failure. By refactoring its legacy database into a microservice-based architecture, Netflix was able to mitigate the risk of database failure. And by utilizing stateless application states, Netflix remains able to respond to sudden scaling issues while also having the ability to manage cluster resources across their stack through making use of features such as AWS EC2’s Auto-Scaling.Rather than rolling out large changes which take months to build, Netflix adopted an agile development pipeline where its developers deploy many smaller changes into production over the course of a day. When working with microservices, continuous integration and continuous deployment allow developers to automate their workflow based on the language, use case, and type of service they are running
- Groupon – Groupon started as a single web page written in Ruby on Rails that showed one deal each day to people in Chicago. Groupon started expanding internationally. As Groupon continued to evolve and new products were launched, the frontend Ruby codebase grew larger. There were too many developers working in the same codebase. It got to the point where it was difficult for developers to run the application locally. Test suites slowed down and flakey tests became a real problem. And since it was a single codebase, the entire application had to be deployed at once. When a production issue required a rollback, everyone’s changes would get rolled back instead of just the broken feature. In short, they had all the problems of a monolithic codebase that had grown too large. Then it decided to split each major feature of the website into a separate web application. At the end of this program, it was up with new Node.js stack based microservices with substantial results.
- Amazon – Started as one application talking to a back end. Written in C++. It grew. For years the scaling efforts at Amazon focused on making the back-end databases scale to hold more items, more customers, more orders, and to support multiple international sites. In 2001 it became clear that the front-end application couldn’t scale anymore. The databases were split into small parts and around each part and created a services interface that was the only way to access the data. The databases became a shared resource that made it hard to scale-out the overall business. SOA gave them the service isolation. Not enough for the scalability that business wants – Rewrite – Services are the independent units delivering functionality within Amazon. It’s also how Amazon is organized internally in terms of teams. Teams are made small. They are assigned authority and empowered to solve a problem as a service in anyway they see fit. With lots of lessons learnt, Amazon becomes more elastic and scalable than we think now.
- Client – Most of the examples I have selected to describe a problem or solution in this presentation is from the experience I gained in decomposing the monolithic application which was serving the client as a core product for years. Hope, we discussed most of it on the way before we reach this point..
Moral of the stories – The journey to microservices is a journey – with lot of ups and downs and it will be different for different company. There are no hard and fast rules, only tradeoffs. So move on depending upon your business needs.
3. Interesting Articles:
It is really worth taking inputs from the articles written by these architects, speakers & authors before we dive deep into the microservices ocean.
- Adrian Cockcroft – He oversaw Netflix transition from a traditional development model with 100 engineers producing a monolithic DVD‑rental application to a microservices architecture with many small teams responsible for the end‑to‑end development of hundreds of microservices that work together to stream digital entertainment to millions of Netflix customers every day. Now he is with AWS.
- Chris Richardson – Experienced software architect, author of POJOs in Action and the creator of the original CloudFoundry.com
- Christian Posta – Chief Architect, cloud application development @ Red Hat, author of Microservices for Java Developers, open-source enthusiast
- Martin Fowler – author, speaker… essentially a loud-mouthed pundit in software development who works at ThoughtWorks
P.S. Originally published in my linked articles space