Building for Scale: App Architecture Styles

So far we've looked at what containers are, the difference between a container and a virtual machine, and how to set up one or multiple containers using Docker. In this article we're going to explore different styles of architecture for web applications. But what's the motivation for changing our style of architecture and going to the effort of setting up all our containers in the first place? The most likely answer is scaling.

Scaling

Scaling a web app is giving it additional resources to work with so that it can handle increased load. There are different motivations for scaling depending on the specifics of what you're doing. For example, if you're a video streaming service like Twitch, you might want to offer higher resolution video streams, but this means that you need to let each stream use more processing power. On the other hand, for a simple web app you might just want to handle more users at the same time, but again you need more computing power to make this happen.

Typically scaling up means adding one of the following resources:

CPU cores - for doing more stuff at the same time. Often this just means creating more virtual machines, but it can also mean allowing existing virtual machines greater access to the host machine.
RAM - to allow one instance to do more intensive stuff. This can be very significant in data processing and database usage, for example.
Disk space - lets you store more stuff. Typically if you need disk space to scale, the answer is to use a service like EBS or Google Cloud Storage.
GPU capacity - for complex, computationally expensive tasks like machine learning or video transcoding
Locations - this sounds strange, but there are many situations you might want new locations for your app. If the servers you're using are closer on a network to an end user, then the latency will be lower.

It's also possible that you might want to scale down. Hiring server space is expensive, and if demand for a service drops (e.g. YouTube at 3am in the US) then it might be worth scaling down the resources your app is using to save money.

Scaling can be static (i.e. you do it manually by buying servers, for example) or dynamic (i.e. it happens in response to a change in demand for resources).

Architecture

The architecture of an app is its overall design and structure. On the most basic level this is just how you lay out your code, especially in object-oriented development paradigms.

However, software architecture can range in complexity and scale from software architecture (a single service) to solutions architecture (multiple services - this is what we're looking at in this article) to enterprise architecture (designing all technical systems and use of them in a typically large company).

For obvious reasons, designing and taking responsibility for a whole company's technology stack often has a strong overlap with the role of a CTO or a very senior technical consultant.

Monolith Architecture

A monolith architecture is where all your services are in a single app with a single codebase. While this model is relatively easy to think about and develop with a small team (since it only has a single codebase), it can become difficult when you want to make your app more complex or service a large number of users.

If you've tried to write a complex web app before this is probably what you've used for your backend, and you may already have encountered some of the problems with it.

However, it does have some advantages, especially for small teams and individual developers:

Easy to write (one codebase, don't need to worry how different services communicate)
Easy to deploy (there's only one service)
Easy to test and debug (no looking for problems in and between various different services, as in other architectures)
Work just fine for small teams

The well-known startup accelerator Y Combinator recommends that new products should be built with monolith or a relatively monolithic architecture so that they're easy to iterate on when you're looking for users and revenue. That said, monolithic architectures do have significant drawbacks for larger organisations.

💡 In 2008 Netflix experienced a major failure due to relying on a monolithic architecture, which became unable to handle the increased number of users as they became more popular. As a result, they changed their approach and have since become one of the most prolific users of microservices.

Here are some of the disadvantages of monolithic architectures:

Development speed is inversely proportional to the app size. As apps grow bigger they become harder to understand. Is it my NetworkManager that handles incoming client requests, or my SocketManager?

Difficult to scale. Since there's only one app, you need to either deploy multiple instances of the same app (which is a bad use of resources) or pay ever-increasing server costs for more powerful hardware. At a certain point a single server, no matter how powerful it is, will be unable to handle a large number of users.

Lacks robustness. If one part of the app fails, it all fails.

Difficult to make changes. Since the codebase itself is locked into a given set of technologies, when those technologies change it might require re-engineering a significant amount of code.

If you're starting a new company, then, or building a prototype, perhaps monolith is the way forward. However, you likely don't work at an early stage startup and want to know how to build services that scale well across a larger number of users, or understand your company's existing architecture better. The most common style of architecture we can use for this, and the most relevant to the work we've done with Docker, is the microservices architecture.

Microservices Architecture

Microservices architecture is a style which aims to overcome many of the traditional issues of a monolithic architecture. In microservices architectures, we create many small services with relatively few responsibilities, then connect them together by letting them directly communicate with each other. This could happen in different ways, like reading a shared file or communicating by HTTP request (remember last class?).

Many modern apps that you use every day are microservices-based, and cloud platforms like Google Cloud, Microsoft Azure, and AWS offer extensive support for this style, including offering many services which provide parts of a microservices solution for you. At an extreme, you can write an entire solution using cloud products and just glue them together with a few AWS lambda functions.

Advantages of a microservices architecture are significant:

Easier to scale. Because your app is broken into many parts, you can make use of software like Kubernetes to selectively scale services that need it. For example, on a video streaming site you might need to radically increase the number of CPUs available to you in response to increases in traffic.

Development speed is more linear. It's a bit harder to set up and use a microservices architecture, but once you do, it's easier to add new parts to it relative to a monolith architecture. The cognitive load needed to understand a single microservice is far less than that needed to understand an entire monolith application.

Can be more robust. If implemented well, one service or even one whole part of your app failing shouldn't affect the rest of it. For example, if we sell widgets to people on a repeating basis and have a separate Order service and Billing service, the Order service failing is less of a problem: we are still able to collect payments.

Easy to change. If implemented well, you can swap out one service for a completely different one using the same API and have everything work well.

Works well for large development teams. Each microservice can be managed by a small team of 5-10 developers, with other devops or cross-functional teams in charge of making sure each microservice can talk to each other properly.

Work well with technologies like VMs and containers. If you need a new microservice, you can just spin up a new VM, container, or Kubernetes cluster for it.

For these reasons, microservices have become the de-facto standard for writing apps, especially in large organisations with a strong engineering culture. However, they do have some problems, largely associated with the sheer number of microservices that are involved in a typical app.

Can become difficult to test and debug. If there are 50 different microservices involved in a single user action, it becomes difficult to figure out which of them broke and caused a particular bug. This means that we need to add extra services to help detect and fix bugs, adding even more complexity to our solution.

Deployment is challenging. Since there are a lot of different services with complex relationships (at worst n^2 where n is the number of services), someone needs to understand and manage those relationships in order to deploy them successfully. This implies, at minimum, having someone whose main job is devops, if not a whole team.

Nobody understands how it works. Granted this is a problem in all development, but microservices can make it worse. Since each service is cut off from the others, it can be hard for people who don't work on a given service to understand its importance to the overall product.

With this in mind, a microservices architecture is a great choice if you need it. This might be because your app has a lot of users, or because you need the ability to scale up and down different parts of the app in response to load.

Other Architectural Styles

Some large enterprises, especially those which (unlike Netflix) don't employ a large number of engineers, will use other styles of architecture to overcome the disadvantages of monolithic style and to streamline their services relative to microservice architecture.

One of these is the enterprise service bus architecture. This is an extension of the microservices architecture described above. In this style, there are many services (like our microservices architecture), but rather than talking to each other directly, they communicate through a message or event bus which is another service which helps other services communicate, such as Amazon EventBridge or Apache Kafka. The event bus "translates" between different services and makes sure each app receives the information it needs when it needs it. It can also help manage load by keeping the flow of information between two services running at a consistent rate and adjusting for peaks and dips in demand.

Implementing an event bus based architecture can be very helpful for large organisations who need the robustness that being able to translate between many different systems provides, and can help reduce the latency associated with microservices, but it can also become complex and expensive to maintain.

Another possible architectural style is the multi-tier or layered architecture. This is similar to a microservices architecture in that it has several different services which are connected directly. However, rather than being divided by function, they are divided by level of abstraction. For example, a backend might include an API layer which receives and replies to client message and a data access layer which provides information to the API layer from several different data sources (databases, local files, etc.)

The multi-tier architecture is a middle ground between the chaos that can be brought about by unchecked use of microservices and the difficulties that having a single monolithic app creates. In fact, you've probably already used a multi-tier architecture: most web apps consist of a client and a server which are two separate services.

Knowledge Check

If you want to deal with more users on a basic web app, what should you do?
If you're writing a game and want lower latency to players in Asia, what should you do?
If you're making a video upload site and want processing of new video uploads to go faster, what should you do?
If nobody is using your video upload site at 4am, what should you do?
In what situation might it be helpful to use a monolithic architecture?
Why wouldn't you want to use a monolithic web app for a video streaming site such as Twitch?
Name one benefit and one drawback of using a microservices architecture.
Do you think the organisation you work for (if any) would need to use microservices? Where and why?
What's the difference between a plain microservices architecture and an event bus architecture?
What's the difference between a multi-tier architecture, a microservices architecture, and a monolithic architecture?

Discussion Questions

Let's say we're building Justin.tv (the original version of Twitch). This website needs to be able to display a live chat on the same page as streaming a video and also display basic information about an ongoing stream. How could we go about implementing this and make sure that it doesn't crash when too many users try to access it at the same time?
Let's say we're building a piece of blogging software (like Ghost, which this is hosted on). This software needs to let admins write their articles using the markdown plain-text format. When people visit the website, they should see the markdown articles rendered in a nice format with a front page showing the articles ordered by their last update time. How can we implement this?
Let's say we're building a social media platform. This needs to be able to hold data about a vast number of threads and the comments associated with them. It also needs to be able to scan the content of different threads and perform different actions, from deleting unsafe or illegal content to promoting popular threads on the frontpage or /r/funny. How could we go about building this? What problems can you foresee?
Let's say we're building a search engine. This needs to spawn a number of crawlers to navigate the web via links on websites. It needs to return the metadata and how many links were crawled to a data analysis product so that we can use this information to rank search results. Finally, it needs to present a sorted list of results to users who visit a search frontend. How can we implement this?

Assignment

Write three apps: a frontend which accepts image files for upload (and may display which files are available), a backend which takes those files and writes them into a shared docker volume, and a processing app which creates rescaled versions of those image files. In Python you can use the Pillow library for this. In Node.js you can use the sharplibrary. In either you could also invoke imagemagick using the command line.
Install Minikube from their website. This is a tool which lets you create and run a single Kubernetes cluster on your local computer for learning and experimentaton purposes.