Designing a system that supports millions of users is challenging, and it is a journey that requires continuous refinement and endless improvement. In this chapter, we build a…
- 01.01Scale from zero to millions of usersDesigning a system that supports millions of users is challenging, and it is a journey that requires continuous refinement and endless improvement. In this chapter, we build a…
- 01.02Single server setupA journey of a thousand miles begins with a single step, and building a complex system is no different. To start with something simple, everything is running on a single…
- 01.03DatabaseWith the growth of the user base, one server is not enough, and we need multiple servers: one for web/mobile traffic, the other for the database (Figure 1-3). Separating…
- 01.04Which databases to use?You can choose between a traditional relational database and a non-relational database. Let us examine their differences. Relational databases are also called a relational…
- 01.05Vertical scaling vs horizontal scalingVertical scaling, referred to as “scale up”, means the process of adding more power (CPU, RAM, etc.) to your servers. Horizontal scaling, referred to as “scale-out”, allows you…
- 01.06Load balancerA load balancer evenly distributes incoming traffic among web servers that are defined in a load-balanced set. Figure 1-4 shows how a load balancer works. As shown in Figure…
- 01.07Database replicationQuoted from Wikipedia: “Database replication can be used in many database management systems, usually with a master/slave relationship between the original (master) and the…
- 01.08CacheA cache is a temporary storage area that stores the result of expensive responses or frequently accessed data in memory so that subsequent requests are served more quickly. As…
- 01.09Cache tierThe cache tier is a temporary data store layer, much faster than the database. The benefits of having a separate cache tier include better system performance, ability to reduce…
- 01.10Considerations for using cacheHere are a few considerations for using a cache system: Decide when to use cache. Consider using cache when data is read frequently but modified infrequently. Since cached data…
- 01.11Content delivery network (CDN)A CDN is a network of geographically dispersed servers used to deliver static content. CDN servers cache static content like images, videos, CSS, JavaScript files, etc. Dynamic…
- 01.12Considerations of using a CDNCost: CDNs are run by third-party providers, and you are charged for data transfers in and out of the CDN. Caching infrequently used assets provides no significant benefits so…
- 01.13Stateless web tierNow it is time to consider scaling the web tier horizontally. For this, we need to move state (for instance user session data) out of the web tier. A good practice is to store…
- 01.14Stateful architectureA stateful server and stateless server has some key differences. A stateful server remembers client data (state) from one request to the next. A stateless server keeps no state…
- 01.15Stateless architectureFigure 1-13 shows the stateless architecture. In this stateless architecture, HTTP requests from users can be sent to any web servers, which fetch state data from a shared data…
- 01.16Data centersFigure 1-15 shows an example setup with two data centers. In normal operation, users are geoDNS-routed, also known as geo-routed, to the closest data center, with a split…
- 01.17Message queueA message queue is a durable component, stored in memory, that supports asynchronous communication. It serves as a buffer and distributes asynchronous requests. The basic…
- 01.18Logging, metrics, automationWhen working with a small website that runs on a few servers, logging, metrics, and automation support are good practices but not a necessity. However, now that your site has…
- 01.19Database scalingThere are two broad approaches for database scaling: vertical scaling and horizontal scaling.
- 01.20Vertical scalingVertical scaling, also known as scaling up, is the scaling by adding more power (CPU, RAM, DISK, etc.) to an existing machine. There are some powerful database servers.…
- 01.21Horizontal scalingHorizontal scaling, also known as sharding, is the practice of adding more servers. Figure 1- 20 compares vertical scaling with horizontal scaling. Sharding separates large…
- 01.22Millions of users and beyondScaling a system is an iterative process. Iterating on what we have learned in this chapter could get us far. More fine-tuning and new strategies are needed to scale beyond…
References
Millions of users and beyond
- Hypertext Transfer Protocol: https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
- Should you go Beyond Relational Databases?: https://blog.teamtreehouse.com/should-you-go-beyond-relational-databases
- Replication: https://en.wikipedia.org/wiki/Replication_(computing)
- Multi-master replication: https://en.wikipedia.org/wiki/Multi-master_replication
- NDB Cluster Replication: Multi-Master and Circular Replication: https://dev.mysql.com/doc/refman/5.7/en/mysql-cluster-replication-multi-master.html
- Caching Strategies and How to Choose the Right One: https://codeahoy.com/2017/08/11/caching-strategies-and-how-to-choose-the-right-one/
- R. Nishtala, "Facebook, Scaling Memcache at," 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’13).
- Single point of failure: https://en.wikipedia.org/wiki/Single_point_of_failure
- Amazon CloudFront Dynamic Content Delivery: https://aws.amazon.com/cloudfront/dynamic-content/
- Configure Sticky Sessions for Your Classic Load Balancer: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-sticky-sessions.html
- Active-Active for Multi-Regional Resiliency: https://netflixtechblog.com/active-active-for-multi-regional-resiliency-c47719f6685b
- Amazon EC2 High Memory Instances: https://aws.amazon.com/ec2/instance-types/high-memory/
- What it takes to run Stack Overflow: http://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack-overflow
- What The Heck Are You Actually Using NoSQL For: http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosqlfor.html