How to start your journey into Microservices Part -2 (DDD)

In the last blog we learned that there are various areas of designing microservices.

In this blog we will work on decomposition of a larger problem into smaller microservices and look at the subtle but very important aspects on why to breakup and how simple it could be.

For this we will be using DDD. We will not go in detail of what exactly is ddd , we will work on the strategic design part of DDD, but lets still iterate over some of the principles of DDD.

One principle behind DDD is to bridge the gap between domain experts and developers by using the same language to create the same understanding. You must have seen cases where it becomes difficult for the product managers / experts to iteratively add features as the language between the pm and dev is different.

Another principle is to reduce complexity by applying object oriented design and design patters to avoid reinventing the wheel.

But what is a Domain? A Domain is a “sphere of knowledge”, for instance the business the company runs. A Domain is also called a “problem space”, so the problem for which we have to design a solution.

Lets choose a business problem like building a E-Commerce site<Amazon> on which we will try to apply DDD. We will not be able to go in complete depth of the business problem. I am writing very basic features required for it to work:

  1. User searches for a product
  2. User place a order for that product
  3. User pays online
  4. Delivery Management processes that order and start the delivery
  5. Delivery update the Order Status

Now we will try to design this in traditional way, lets create the schema:

  • Tables
    • Product
      • id
      • name
      • image url
      • price
      • count available
      • ….
    • Product Packaging Info
      • id
      • product id
      • size
      • isFragile
      • …..
    • Order
      • product id
      • user id
      • delivery address details
      • paid
    • Order Delivery Status
      • id
      • order id
      • delivery company name
      • delivery company id
      • delivery company code
    • Delivery Company
      • id
      • name
      • ….
    • User
      • id
      • name
      • address details
    • User Preferences
      • id
      • name
      • preferences

We can create a table structure like this in our system<lot more tables will be there>. I think by looking at those tables you could understand not all tables to be understood by every dev, eg: someone working on delivery management might not be interested in UserPreferences <used for searching> and someone working on searching might not be interested in OrderDeliveryStatus.

By this you can understand that we need to break the structure in smaller areas.

To design this in a way which helps to put more business context and smaller structure to manage . Lets put DDD.

As Domain in DDD means we are talking about a specific set of area <knowledge> . In larger sense E-commerce domain can be classified internally by various subdomain like:

  1. Identity Management
    1. User
  2. Inventory Management
    1. Product
  3. Product Search
    1. Product
    2. UserPreferences
  4. Order
    1. Order
    2. Order Delivery Status
  5. Delivery Management
    1. Order
    2. Product Packaging Info
    3. Delivery Company
    4. Order Deliver Status
  6. Billing

The separated Domain can easily be visualized. In DDD terms this is called a Context Map, and it is the starting point for any further modeling.Essentially what we have done is breakup the larger problem into smaller interconnected ones.

Now we need to align the Subdomain aka problem space to our solution design, we need to form a solution space. A solution space in DDD lingo is also called a Bounded Context, and it is the best to align one problem space/Subdomain with one solution space/Bounded Context.

In this we can think of each sub domain as a diff. microservice. Microservices are not complete without their dependencies . lets see them in context of bounded context:

  • Product Search – dependent on – Inventory Management
  • Delivery Management – dependent on – Order
  • Product Search – dependent on – User
  • Billing – dependent on – order
  • … so on

you can see that their is dependency between order and billing service and they talk via a common shared objects model which both of them can represent separately rather that using a single complete model which is cumbersome eg: order object in order service<care about status> is different from order in billing service<care about amount> . Now this is benefit of breaking them into smaller and separated domains.

To define such contracts one can also use ContextMapper.

There are certain outcomes one should take out of this:

  • Breaking into smaller pieces is very important so that it becomes easy for people to work on diff. part of the business
  • It is very simple when we are clear about the business sub domains .

After this i recommend you guys to go in more depth of DDD and look one more example here.

In next we will look about authentication mechanisms.

How to start your journey into Microservices Part -1

Architecting an application using Microservices for the first timers can be very confusing. This article is very relevant if

  • You are you beginning to develop an application that can scale like Amazon, Facebook, Netflix, and Google.
  • You are doing this for the first time.
  • You have already done research and decided that microservices architecture is going to be your secret sauce.

Microservices architecture is believed to be the simplest way of scaling without limits. However, when you get started, a lot of considerations are going to confuse you. Questions arose as I spent time learning about it online or discussing it with a team:

  1. What exactly is a microservice?
    1. Some said it should not exceed 1,000 lines of code.
    2. Some say it should fit one bounded context (if you don’t know what a bounded context is, don’t bother with it right now; keep reading).
  2. Even before deciding on what the “micro”service will be, what exactly is a service?
  3. Microservices do not allow updating multiple entities at once; how will I maintain consistency between entities
  4. Should I have a single database cluster for all my microservices?
  5. What is this eventual consistency thing everyone is talking about?
  6. How will I collate data which is composed of multiple entities residing in different services?
  7. What would happen if one service goes down? How would the dependent services behave?
  8. Should I make a sync invocation between microservices to always get consistent data?
  9. How will I manage version upgrades to a few or all microservices? Is it always possible to do it without downtime?
  10. And the last unavoidable question – how do I test the entire application as an integrated application?

Hmm… All of the above questions must be answered to able to be understand and deploy applications based on microservices.

Lets first list down all things we should cover to understand Microservices :

  • Decomposition – Breaking of system in Microservices and contracts between them
  • Authentication – how authentication info is passed from service to service , how diff. services validate the session
  • Service Discovery – hard coded , external system based like consul , eureka , kubernetes
  • Data Management – Where to store the data , whether to share the DB or not
  • Auditing – very important in business applications to have audit information about who updated or created something
  • Transactional Messaging – when you require very high consistency between DB operation and its event passed onto diff. services
  • Testing – What all to test , Single Service , Cross Service Concerns
  • Deployment Pattern – serverless , docker based
  • Developer Environment Setup – All Services running on developer machine , or single setup
  • Release Upgrades – How to do zero downtime release upgrades , blue green deployments
  • Debugging – Pass tracing id between services , track time taken between services , log aggregation
  • Monitoring – API time taken , System Health Check
  • UI Development – Single Page Applications or Micro Front Ends and client side composition
  • Security – for internal users

The First Thing we should learn is how to Decompose or build the Services:

Domain Driven Design:

While researching the methodology to break up the application into these components and also define the contract between them, we found the Domain Driven Design philosophy to be the most convincing. At a high level, it guided us on

  • How to break a bigger business domain into smaller bounded contexts(services).
  • How to define contracts between them(services).

These smaller components are the microservices. At a finer level, domain driven design (aka DDD) provides tactical methods to help us with

  • How to write code in a single bounded context(services).
  • How to break up the entities(business objects within the service).
  • How to become eventually consistent(as our data is divided into multiple services we cannot have all of them consistent every moment).

After getting the answer to “how to break the application into microservices,” we needed a framework for writing code on these guidelines. We could come up with a framework of our own, but we chose to not to reinvent the wheel. Lagom , Axon, Eventuate are all java based , frameworks which provides all the functionality that we require to do microservices, like

  1. Writing services using DDD.
  2. Testing services.
  3. Integration with Kafka, Rabbit Mq …, the messaging framework for message passing between services.

This is Part 1 in a series of articles. I have explained how we got our direction on getting started with microservices. In the next article, we will discuss about a sample Application and breakup of that using DDD .

Recommended References

Thanks to the blogs by Vaugh VernonUdi DahanChris Richardson, and Microsoft. A few specific references:

  1. Youtube for Event Sourcing, CQRS, Domain Driven Design, Lagom.
  2. https://msdn.microsoft.com/en-us/library/jj554200.aspx
  3. http://cqrs.nu/
  4. Domain Driven Design Distilled and implementing Domain-Driven Design, by Vaugh Vernon.
  5. http://microservices.io/ by Chris Richardson.
  6. Event Storming http://eventstorming.com/, by Alberto.