Logo
Logo
Home
Archive
AI Agent Notes
Advertise
YouTube
Login
Sign Up
Logo
  • Home
  • Posts
  • 🦥Rate Limiting For Dummies

🦥Rate Limiting For Dummies

Nov 26, 2024

Hello friends!

Welcome to this week’s Sloth bytes. I hope you had a great week.

🦥 No selling out today

I am genuinely considering selling feet pics, so if you work at a company with a marketing budget please forward this to your boss immediately or the feet come out.

Save yourself and learn more about sponsoring

Two-fingered and three-fingered sloths are not related.

Although two-fingered and three-fingered sloths look and act similar they are not closely related to each other. Instead, they are examples of convergent evolution, which is when two different animals evolve similar traits to adapt to the same niche in an ecosystem.

Rate Limiting

Ever wondered why you get those "Too Many Requests" errors or want to stop people from spamming your APIs/servers? This is where rate limiting comes in.

What is Rate Limiting?

Rate limiting controls how quickly a client can consume some resource—requests, login attempts, tokens, expensive jobs, etc. The key might be an authenticated user, API key, tenant, IP address, or a combination. Think of it like a nightclub:

  • The door is your API endpoint

  • The bouncer is your rate limiter

  • The guest list is your authenticated users

  • The maximum capacity is your rate limit

Why You Need Rate Limiting

  1. Prevent abuse: Slow brute force, scraping, spam, and runaway clients.

  2. Resource management: Keep one customer or bot from consuming everything.

  3. Cost control: Protect expensive APIs, AI calls, database queries, and downstream services.

  4. Reliability: Shed or shape excess traffic before it overwhelms a dependency.

Rate limiting helps with abuse and overload, but it is not a replacement for DDoS mitigation. Large distributed or volumetric attacks are usually handled at the network/CDN/WAF edge before they reach your application.

Common Rate Limiting Algorithms

1. Fixed Window (I’ve actually had to implement this in an interview)

import time
from collections import defaultdict

class FixedWindowRateLimiter:
    def __init__(self, max_requests, window_seconds):
        self.max_requests = max_requests
        self.window_seconds = window_seconds
        self.windows = defaultdict(lambda: {"window": None, "count": 0})

    def is_allowed(self, user_id):
        now = time.time()
        # Integer bucket number: every request in the same bucket shares a counter.
        window_id = int(now // self.window_seconds)
        state = self.windows[user_id]

        if state["window"] != window_id:
            state["window"] = window_id
            state["count"] = 0

        if state["count"] >= self.max_requests:
            return False

        state["count"] += 1
        return True
  1. Token Bucket

import time

class TokenBucket:
    def __init__(self, capacity, refill_rate_per_second):
        self.capacity = capacity
        self.tokens = float(capacity)
        self.refill_rate = refill_rate_per_second
        self.last_refill = time.monotonic()

    def get_token(self):
        now = time.monotonic()
        elapsed = now - self.last_refill
        self.tokens = min(
            self.capacity,
            self.tokens + elapsed * self.refill_rate
        )
        self.last_refill = now

        if self.tokens >= 1:
            self.tokens -= 1
            return True
        return False

Token buckets allow short bursts up to the bucket capacity while enforcing an average refill rate. Using a monotonic clock avoids weirdness if the system wall clock changes.

Real-World Implementation

For one small Express process, middleware can handle the mechanics. Production gets harder once you have multiple app instances, proxies, authenticated users, and shared quotas.

Here’s a current express-rate-limit example. Its in-memory store is fine for a single process; multiple servers or workers need a shared store so every instance sees the same counters.

import { rateLimit } from 'express-rate-limit';

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  limit: 100,
  standardHeaders: 'draft-8',
  legacyHeaders: false,
  message: { error: 'Too many requests, please try again later.' },
});

// Apply globally...
app.use(limiter);

// ...or create stricter limiters for sensitive routes such as login/reset endpoints.

Best Practices

  1. Clear communication

Prefer standardized RateLimit/RateLimit-Policy headers when your framework supports them, and return HTTP 429 Too Many Requests when a request exceeds the policy. A Retry-After header can tell clients when trying again makes sense.

  1. Multiple tiers (if you’re doing a SaaS)

const freeTierLimit = rateLimit({
  windowMs: 60 * 60 * 1000,
  limit: 100,
});

const proTierLimit = rateLimit({
  windowMs: 60 * 60 * 1000,
  limit: 1000,
});

For authenticated products, keying quotas by user/API key/tenant is often fairer than IP alone. IP limits can accidentally group many users behind one NAT or proxy, and attackers can rotate addresses.

  1. Error handling (still using express)

You usually do not need a separate Express error handler just to produce a 429—the rate limiter can return the response itself. The important application logic is deciding which identity you limit, which operations are expensive, and what should happen if the counter store is unavailable.

Distributed rate limiting gets trickier

An in-memory dictionary works in an interview. It fails as a global quota once your API runs on five servers, because each server has its own counter. Production limiters often use Redis, a gateway, or another shared/edge store with atomic updates and expiration.

Also watch for the fixed-window boundary problem: a client can send 100 requests at 12:00:59 and another 100 at 12:01:00 while technically respecting “100 per minute.” Sliding-window and token-bucket algorithms smooth that behavior.

The lazy way to handle this stuff

If you don’t want to do all this, I understand. There’s a lot of tools/services that handle this for us.

  • Redis/shared-store rate limiting — useful when several app instances must share counters.

  • Nginx / reverse-proxy limiting — reject or shape traffic before it reaches the app process.

  • CDN/WAF/provider controls — Cloudflare, AWS WAF, API gateways, and similar services can enforce limits closer to the network edge.

There’s a lot more, so feel free to do your own research.

If you want to keep learning

  • Redis explained — a common building block for distributed rate limiters and fast counters.

  • Message queues explained — another way large systems absorb traffic spikes without melting.

  • Microservices explained — where rate limiting gets more interesting once services start multiplying.

  • 5 system design resources — go deeper on scalability, databases, caching, and distributed systems.

Introducing the Model Context Protocol (4 minute read)

The Model Context Protocol is an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools. Its aim is to help frontier models produce better, more relevant responses.

React Router v7 (3 minute read)

React Router v7 brings all the great things you love from Remix back to React Router

Ai2 OpenScholar: Scientific literature synthesis with retrieval-augmented language models (10 minute read)

Ai2’s & UW’s OpenScholar, a retrieval-augmented LM, helps scientists navigate and synthesize scientific literature.

Tailwind CSS v4.0 Beta 1 (2 minute read)

After a long alpha period, we're excited to release the first public beta of Tailwind CSS v4.0.

Try the internet’s easiest file API

  • Easy file uploads and retrieval in minutes

  • No complex setup or infrastructure needed

  • Focus on building, not configurations.

Try It here!

*A message from our sponsor.

How to improve search without looking at queries or results (12 minute read)

How we improved Canva’s private design search while respecting the privacy of our community.

A Single Korean-Speaking Intern Saved Valve From Going Bankrupt (5 minute read)

Steam, Portal, Left 4 Dead, Team Fortress 2 – all of those wouldn't have been possible if it wasn't for a pure stroke of luck.

Thank you to everyone who submitted 😃 

ddat828, clsmv, pyGuy152, agentNinjaK, JamesHarryT, dropbearII, ravener, mc-milo, codiling, and taypham88

Splitting Up Numbers

Create a function that takes a number num and returns each place value in the number.

Examples

num_split(39)
output =[30, 9]

num_split(-434)
output = [-400, -30, -4]

num_split(100)
output =[100, 0, 0]

How To Submit Answers

Reply with

  • A link to your solution (github, twitter, personal blog, portfolio, replit, etc)

  • or if you’re on the web version leave a comment!

Video should be coming out within these 2 days

Hopefully…

That’s all from me!

Have a great week, be safe, make good choices, and have fun coding.

If I made a mistake or you have any questions, feel free to comment below or reply to the email!

See you all next week.

Keep Reading

Read all
arrow-right
envelope-simple

Join 50k+ developers and become a better programmer and stay up to date in just 5 minutes.

© 2026 Sloth Bytes.
beehiivPowered by beehiiv