Launching Stacks

Stacks help developers launch multiple microservices in a single click

Launching Stacks

How did we get here?

Over the course of building Cohesive in the last 4 months, we got the opportunity to speak with multiple organisations - from small startups to large enterprises. One common pattern we observed is that there is a pivotal moment in any organisation’s history where the current infrastructure setup starts deteriorating developer velocity, like:

  • The ability for multiple developers to ship new features without having to step on each other's toes,
  • Making sure every developer has the latest stable changes available in their developer environments,
  • And, the learning curve involved in managing developer infrastructure.

This is the moment where engineering leaders start thinking about what the developer experience in their organisation should look like moving forward. When someone is starting up, the primary goal is to find Product Market Fit (PMF), improving developer velocity is secondary. However, once they hit PMF and start scaling up with 25+ developers, improving developer velocity takes the steering wheel.

A 2020 report from McKinsey shows that companies with higher developer velocity outperform lower ones by five times when it comes to market performance (Revenue CAGR).

This is where Cohesive comes in. We help you focus on innovation by eliminating the hurdles involved in managing, controlling, and keeping your developer environments in sync with the latest changes. Industry reports from 2019 point out that developers spend only half a day or less writing code, thereby hampering their ability to innovate rapidly. Our mission is to give this time back to developers so they can focus on innovation rather than figuring out configurations.

Today, we are pushing our platform even further so it can cater to extremely complex microservices architectures. We are launching the ability to do end to end microservices testing within developer and staging environments.

What is "Stacks"?

Stacks helps templatize complex dependencies that microservices in your system have amongst themselves and enables you to provision and configure the entire infrastructure through a single click. To help you understand the complex problems that Stacks can solve, we are going to showcase how Stacks works with an example. We opted to use Airbnb's latest microservices architecture (SOA v2) as a reference to help you digest how Stacks can help build and deploy complex developer environments in less than a minute.

(Before we step through the rest of the blog, we wish to give credits to Jessica Tai at Airbnb for sharing intricate details of Airbnb's architecture.)

The Airbnb Architecture (2016 to Present)

By 2020, Airbnb's engineering team grew to 2000+ engineers compared to 500+ in 2016, this led to reshaping their microservices architecture to a macro + micro-services architecture. This helped in standardization of service to service calls, low fetch latency between services and avoided redundant data requests. Below is a user facing example of the services that gets called both in the old and new architecture whenever a user visits an Airbnb reservation page.

Airbnb's reservation service call structure across both V1 and V2 versions of their architecture

The new SOA v2 architecture (in the right) has three new elements a) Presentation services (red box), it's responsible for aggregating the data from multiple services and then presenting it to user facing apps. b) Data aggregator (green box), it's responsible for the business logic, translates the incoming requests from presentation layer and decides which downstream service to call. c) Service Blocks (black boxes), is simply an abstraction of multiple services with a common dimension that's condensed into one service block.

The Problem

The migration from v1 to v2 however has led to slow down in build and deploy times for new changes. Let's say if a developer (called Developer A) is working on applying the latest JPEG-XL image compression to all host and guest images that appear on reservations page shown above (iPhone Screenshot). They would test the changes on a staging environment that's shared with Home Service team. Everything is good as long no one makes new changes to the staging environment.

Developer A has a seamless testing experience as long as no one else is making changes to their environment

Now, let's say a new developer (called Developer B) from the Home Service team makes a change to the same staging environment without co-ordinating with Developer A. This is where things go wrong. If a breaking change is made by Developer B, then Developer A will have to debug why the page is breaking. This could either be via emails to Home Service team OR via reading through code commits to the staging environment.

Things start breaking once a non co-ordinated change goes out to the same test environment

The Solution

What if both developers had a way to provision an isolated microservices environment that stacked all services they care about one over the other? Cohesive stacks does just that. The Platform engineering team at Airbnb can simply define what the developer stack should look like for each team and templatize them as a Cohesive Stack. Airbnb developers can simply click a button and launch an isolated environment OR simply push their code and get a preview link to their own isolated environment.

Cohesive stacks offers an uninterrupted microservices testing experience by offering an end to end microservices stack to each developer

Here are the features that Stacks offer today

  1. Helps templatize the configuration of complex microservices within your systems - Airbnb Platform Engineering team can define what services are needed and include them inside a Stack that developers from the User Service Platform team will require for their testing needs.

  1. Auto provisions an Isolated and stable environments on every git commit - Developer A from the User Service Platform team can now confidently commit new code into their own dev branch, and immediately Cohesive will use that team's Stacks config to spin up a new environment.

  1. Create previewable developer environments - Developer A can now choose to hand out a publicly available preview links to product managers and designers to get feedback on the image compression that they are implementing.

  1. Safely incorporate new changes from another service to test - Cohesive also offers Developer A the capability to pull in the latest change that Developer B from the Home Service Platform team just did by simply mentioning the code branch that needs to be incorporated while spinning up a new environment.

We are barely scratching the surface on what we can do to deliver a great developer experience, Cohesive Stacks is just the first step.

If you are currently looking for an end to end microservices testing solution, then Cohesive Stacks is for you. Join the waitlist below for early access to Cohesive.

August 8, 2022