notexOPENOverview
Notex/Systems/Design a rate limiter

Design a rate limiter

Always free to read

In a network system, a rate limiter is used to control the rate of traffic sent by a client or a service. In the HTTP world, a rate limiter limits the number of client requests…

04

DESIGN CASE STUDIES

Design a rate limiter

14 LESSONS · 24 MIN
  1. 04.01Design a rate limiterIn a network system, a rate limiter is used to control the rate of traffic sent by a client or a service. In the HTTP world, a rate limiter limits the number of client requests…READING1 MIN
  2. 04.02Step 1 - Understand the problem and establish design scopeRate limiting can be implemented using different algorithms, each with its pros and cons. The interactions between an interviewer and a candidate help to clarify the type of…READING1 MIN
  3. 04.03Step 2 - Propose high-level design and get buy-inLet us keep things simple and use a basic client and server model for communication.READING1 MIN
  4. 04.04Where to put the rate limiter?Intuitively, you can implement a rate limiter at either the client or server-side. Client-side implementation. Generally speaking, client is an unreliable place to enforce rate…READING2 MIN
  5. 04.05Algorithms for rate limitingRate limiting can be implemented using different algorithms, and each of them has distinct pros and cons. Even though this chapter does not focus on algorithms, understanding…READING8 MIN
  6. 04.06High-level architectureThe basic idea of rate limiting algorithms is simple. At the high-level, we need a counter to keep track of how many requests are sent from the same user, IP address, etc. If…READING1 MIN
  7. 04.07Step 3 - Design deep diveThe high-level design in Figure 4-12 does not answer the following questions: How are rate limiting rules created? Where are the rules stored?How to handle requests that are…READING1 MIN
  8. 04.08Rate limiting rulesLyft open-sourced their rate-limiting component [12]. We will peek inside of the component and look at some examples of rate limiting rules: domain: messaging descriptors: -…READING1 MIN
  9. 04.09Exceeding the rate limitIn case a request is rate limited, APIs return a HTTP response code 429 (too many requests) to the client. Depending on the use cases, we may enqueue the rate-limited requests…READING1 MIN
  10. 04.10Detailed designFigure 4-13 presents a detailed design of the system. Rules are stored on the disk. Workers frequently pull rules from the disk and store them in the cache.When a client sends…READING1 MIN
  11. 04.11Rate limiter in a distributed environmentBuilding a rate limiter that works in a single server environment is not difficult. However, scaling the system to support multiple servers and concurrent threads is a…READING2 MIN
  12. 04.12Performance optimizationPerformance optimization is a common topic in system design interviews. We will cover two areas to improve. First, multi-data center setup is crucial for a rate limiter because…READING1 MIN
  13. 04.13MonitoringAfter the rate limiter is put in place, it is important to gather analytics data to check whether the rate limiter is effective. Primarily, we want to make sure: The rate…READING1 MIN
  14. 04.14Step 4 - Wrap upIn this chapter, we discussed different algorithms of rate limiting and their pros/cons. Algorithms discussed include: Token bucketLeaking bucketFixed windowSliding window…READING2 MIN

References

Step 4 - Wrap up
  1. Rate-limiting strategies and techniques: https://cloud.google.com/solutions/rate-limitingstrategies-techniques
  2. Twitter rate limits: https://developer.twitter.com/en/docs/basics/rate-limits
  3. Google docs usage limits: https://developers.google.com/docs/api/limits
  4. IBM microservices: https://www.ibm.com/cloud/learn/microservices
  5. Throttle API requests for better throughput: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-requestthrottling.html
  6. Stripe rate limiters: https://stripe.com/blog/rate-limiters
  7. Shopify REST Admin API rate limits: https://help.shopify.com/en/api/reference/restadmin-api-rate-limits
  8. Better Rate Limiting With Redis Sorted Sets: https://engineering.classdojo.com/blog/2015/02/06/rolling-rate-limiter/
  9. System Design — Rate limiter and Data modelling: https://medium.com/@saisandeepmopuri/system-design-rate-limiter-and-data-modelling- 9304b0d18250
  10. How we built rate limiting capable of scaling to millions of domains: https://blog.cloudflare.com/counting-things-a-lot-of-different-things/
  11. Redis website: https://redis.io/
  12. Lyft rate limiting: https://github.com/lyft/ratelimit
  13. Scaling your API with rate limiters: https://gist.github.com/ptarjan/e38f45f2dfe601419ca3af937fff574d#request-rate-limiter
  14. What is edge computing: https://www.cloudflare.com/learning/serverless/glossary/whatis-edge-computing/
  15. Rate Limit Requests with Iptables: https://blog.programster.org/rate-limit-requests-withiptables
  16. OSI model: https://en.wikipedia.org/wiki/OSI_model#Layer_architecture
KEEP EXPLORING

One note leads to another.

Back to the library →
⌕
Search across the entire library ⌘ K