In a network system, a rate limiter is used to control the rate of traffic sent by a client or a service. In the HTTP world, a rate limiter limits the number of client requests…
- 04.01Design a rate limiterIn a network system, a rate limiter is used to control the rate of traffic sent by a client or a service. In the HTTP world, a rate limiter limits the number of client requests…
- 04.02Step 1 - Understand the problem and establish design scopeRate limiting can be implemented using different algorithms, each with its pros and cons. The interactions between an interviewer and a candidate help to clarify the type of…
- 04.03Step 2 - Propose high-level design and get buy-inLet us keep things simple and use a basic client and server model for communication.
- 04.04Where to put the rate limiter?Intuitively, you can implement a rate limiter at either the client or server-side. Client-side implementation. Generally speaking, client is an unreliable place to enforce rate…
- 04.05Algorithms for rate limitingRate limiting can be implemented using different algorithms, and each of them has distinct pros and cons. Even though this chapter does not focus on algorithms, understanding…
- 04.06High-level architectureThe basic idea of rate limiting algorithms is simple. At the high-level, we need a counter to keep track of how many requests are sent from the same user, IP address, etc. If…
- 04.07Step 3 - Design deep diveThe high-level design in Figure 4-12 does not answer the following questions: How are rate limiting rules created? Where are the rules stored?How to handle requests that are…
- 04.08Rate limiting rulesLyft open-sourced their rate-limiting component [12]. We will peek inside of the component and look at some examples of rate limiting rules: domain: messaging descriptors: -…
- 04.09Exceeding the rate limitIn case a request is rate limited, APIs return a HTTP response code 429 (too many requests) to the client. Depending on the use cases, we may enqueue the rate-limited requests…
- 04.10Detailed designFigure 4-13 presents a detailed design of the system. Rules are stored on the disk. Workers frequently pull rules from the disk and store them in the cache.When a client sends…
- 04.11Rate limiter in a distributed environmentBuilding a rate limiter that works in a single server environment is not difficult. However, scaling the system to support multiple servers and concurrent threads is a…
- 04.12Performance optimizationPerformance optimization is a common topic in system design interviews. We will cover two areas to improve. First, multi-data center setup is crucial for a rate limiter because…
- 04.13MonitoringAfter the rate limiter is put in place, it is important to gather analytics data to check whether the rate limiter is effective. Primarily, we want to make sure: The rate…
- 04.14Step 4 - Wrap upIn this chapter, we discussed different algorithms of rate limiting and their pros/cons. Algorithms discussed include: Token bucketLeaking bucketFixed windowSliding window…
References
Step 4 - Wrap up
- Rate-limiting strategies and techniques: https://cloud.google.com/solutions/rate-limitingstrategies-techniques
- Twitter rate limits: https://developer.twitter.com/en/docs/basics/rate-limits
- Google docs usage limits: https://developers.google.com/docs/api/limits
- IBM microservices: https://www.ibm.com/cloud/learn/microservices
- Throttle API requests for better throughput: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-requestthrottling.html
- Stripe rate limiters: https://stripe.com/blog/rate-limiters
- Shopify REST Admin API rate limits: https://help.shopify.com/en/api/reference/restadmin-api-rate-limits
- Better Rate Limiting With Redis Sorted Sets: https://engineering.classdojo.com/blog/2015/02/06/rolling-rate-limiter/
- System Design — Rate limiter and Data modelling: https://medium.com/@saisandeepmopuri/system-design-rate-limiter-and-data-modelling- 9304b0d18250
- How we built rate limiting capable of scaling to millions of domains: https://blog.cloudflare.com/counting-things-a-lot-of-different-things/
- Redis website: https://redis.io/
- Lyft rate limiting: https://github.com/lyft/ratelimit
- Scaling your API with rate limiters: https://gist.github.com/ptarjan/e38f45f2dfe601419ca3af937fff574d#request-rate-limiter
- What is edge computing: https://www.cloudflare.com/learning/serverless/glossary/whatis-edge-computing/
- Rate Limit Requests with Iptables: https://blog.programster.org/rate-limit-requests-withiptables
- OSI model: https://en.wikipedia.org/wiki/OSI_model#Layer_architecture