This is a 2-part series on how microservices architecture pattern can enable insurance and insurtech organisations to build resilient and scalable software applications and gives them the ability to be responsive to market, regulatory and customer needs while maintaining high levels of software quality. Part 1 will provide an overview, high level benefits and disadvantages of microservices architecture.
Have you ever thought about how the human body works? It’s a beautiful piece of machinery where different organs (heart, kidneys, liver, nervous system etc.) work in harmony to keep you fit and healthy so that you can occasionally moan about the weather. Each organ has a specific function or service and they are coordinated by an autonomic nervous system that routes the signals from the brain to the organs.
The microservices architecture is analogous to how the human body works (if we were to stretch the analogy by comparing the human body to an application). It is an approach to developing applications that are composed of fine-grained, (usually) single purpose services that are built around business capabilities and are independently deployable and scalable by small autonomous teams. The following definition from Martin Fowler’s seminal article on this topic provides a great summary.
In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
— James Lewis and Martin Fowlerhttps://martinfowler.com/articles/microservices.html
Typically, organisations have been building applications as a monolith, meaning all components of an application run in one process and to change one part of the application would require you to build, test and deploy the entire application. There are benefits to this approach and if designed well can be considered a good architecture. It is safe to say that most insurance companies still develop software in this fashion. In contrast, microservices proposes an approach to building an application as a set of fine-grained and cohesive (do one thing and one thing only) services which communicate with one another with a technology agnostic and lightweight protocol (REST-ish HTTP). You can think of a service as a component of an application that is built using technology, language and platform best suited for serving the function it provides. This allows us to parallelise development by enabling small autonomous teams to develop, deploy and scale their respective services independently.
Brief history of microservices architecture
One might say that the seeds of microservices architecture style were sown when the Agile Manifesto was published back in 2001. The manifesto and the agile movement it spurred revolutionised how we build software. The progression of this movement led to a culture of continuous delivery which in turn enabled the devops approach of building software where developers and operations team work hand in hand to deliver software faster and to react to customer and market needs. As the complexity and scale of their business needs increased, they soon realised that the monolithic approach to application architecture hindered their ability to be agile. This in turn led to the idea of splitting an application into multiple cohesive mini-applications (aka services) that are built around business capabilities and can be scaled, deployed and managed independently. This led to the birth of microservices architecture pattern that was pioneered by web native companies like Amazon, Netflix and Soundcloud. At a fundamental level, microservices pattern makes your enterprise architecture agile and allows you to quickly respond to changing market, regulatory and customer demands.
This was not a new idea. The industry had already tried something similar in the past called Service Oriented Architecture (or SOA) which also promotes the idea of building an application using discrete and cohesive services. Microservices might be considered a variant of SOA, and (in theory) they look almost identical due to an overlap of many of its principles and concepts. In practice, however, SOA was implemented using bloated middleware(like Enterprise Service Bus aka ESB) built and maintained by large integration teams. The governance around SOA was onerous with the unintended side-effect of reducing IT-agility and increasing integration costs. In the past, as a business user, if you’ve asked for a simple change to a form (like adding or removing a field) and if your IT team responded with an estimate of 2 months, then you’ve seen and felt the impact of a bloated SOA architecture first hand. Microservices, in contrast, emphasises the principle of smart endpoints and dumb pipes. This means using lightweight (REST-ish HTTP) protocols and messaging to interact with different services that expose fine-grained and smart interfaces.
A well designed microservice architecture (where services are cohesive, loosely-coupled and organised around business capability) will allow you to create cross functional teams who can build, test and deploy individual services in parallel. As a consequence, you can speed up development and are able to add new features rapidly to individual (micro)services. This agility gives you a strategic edge over your competition because you can respond to customer needs expeditiously.
It also gives you the flexibility to select the best programming language, platform, database etc. that suits the needs of the service being implemented. For example, if one service only stores key-value pairs, then you can choose to use a non-relational no-sql database. If another service has stringent performance requirements, then you can choose to use C or C++ to implement its business logic.
You also have the option to rip out an implementation of a service and replace it with something different and modern without affecting other services and the overall application as long as the interface definition stays the same.
Hopefully, you’re convinced of the merits of microservices by now and are chomping at the bit to get started. Before you embark on this transformation programme you should be aware of the following challenges that you may encounter.
Implementing inter-service calls over a potentially unreliable network can be complex and is more costly in terms of network latency and message processing time than in-process calls within a monolithic service process. This cost however may be acceptable given the other benefits of adopting this pattern.
Although testing an individual service might be easier, testing the entire application / system end to end will be more complicated and will require you to invest time and effort in building an end to end test suites.
If you add managing the service interfaces, protocols and lightweight messaging infrastructure to the list, when you convert a monolithic app into a bunch of microservices you may end up with runtime overhead and operational complexity that can cancel out the benefits of the approach.
I hope I was able to paint a high level picture of what microservices are and their potential in building an app that is resilient to architecture erosion. In the next part I will talk about strategies to migrate a monolithic application to microservices. Until then, adios.