Vertical to Horizontal Scaling, Part 2: Vertical Scaling @ Stanza

Preface

In Part 1, we explored why under explosive growth Stanza switched from vertical to horizontal scaling. In this post we’ll go over background on architecture before the switch to better understand the transition.

Microservice Architecture

From the beginning our backend (MEAN) was built around a microservice architecture. Well-defined, self-contained services allowed us to scale each part of our backend very independently even on a monolithic server. Endpoints rendering different frontend products were kept separate from each other and endpoints for APIs. In addition to scaling benefits, this allowed for easier code management.

Traffic Routing/Load Balancing

Because different services ran on the same physical machine, we used a reverse proxy NGINX to route traffic based on request path.

Each service was scaled by starting additional processes running the same code listening on different ports. NGINX load balanced traffic for a particular service to its processes via upstreams in a round-robin fashion.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# nginx.conf
http {
# round-robin load balancing requests to the
# cloned processes listening on different ports
upstream service_1_upstream {
server localhost:8081;
server localhost:8082;
server localhost:8083;
}
upstream service_3_upstream {
server localhost:9091;
server localhost:9092;
}
server {
listen 80;
location /service_1 {
proxy_pass http://service_1_upstream;
}
location /service_3 {
proxy_pass http://service_3_upstream;
}
}
}

Deploying

Deployment was handspun and based on sending requests to a custom deploy service on the server. The service would then run scripts to:

  1. fetch latest code from the repository (of the service to deploy)
  2. checkout intended branch
  3. install dependencies
  4. restart cloned proccesses to run new code (we used forever)

Logging

Logs for each service were written to files, one per process (eg service_1_8081.log, service_1_8082.log). In our case, we piggybacked forever’s --logFile option.

Since logs were on the same physical machine, simple UNIX commands were sufficient for search.

1
2
3
# searching last 5000 lines of all service_1 processes'
# logs for "Uh oh."
tail -n 5000 service_1*.log | grep --line-buffered "Uh oh."

Security

SSL terminated at NGINX layer before routing to individual processes.

Where do we go from here?

Though we were vertical scaling at server level, underneath the hood each service was horizontally scaled via multiple processes. Luckily for us this model was a good headstart to horizontal scaling as the principles were transferrable. In Part 3, we’ll look at the specific architecture and tools we use under horizontal scaling.