The Nginx rate limiting config I wish I set up sooner

A few months back one of my sites got hit with a traffic spike. Not even a deliberate attack, just a viral moment that sent requests through the roof. My VPS handled it fine for about 30 seconds, then MySQL started throwing too many connections and everything went down.

The fix wasn't upgrading the server. It was adding proper rate limiting in Nginx.

Here's the actual config I'm running now:

# Define rate limiting zones
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=general_limit:10m rate=30r/s;
limit_conn_zone $binary_remote_addr zone=conn_limit:10m;

server {
    # General rate limit for all requests
    limit_req zone=general_limit burst=50 nodelay;

    # Max 20 concurrent connections per IP
    limit_conn conn_limit 20;

    location /api/ {
        # Stricter for API endpoints
        limit_req zone=api_limit burst=20 nodelay;

        # Return 429 instead of 503 when rate limited
        limit_req_status 429;
    }

    # Custom error page for rate limited requests
    error_page 429 = @rate_limited;

    location @rate_limited {
        default_type application/json;
        return 429 '{"error":"Too many requests","retry_after":1}';
    }
}

Key things I learned the hard way:

The burst parameter is more important than the rate itself. Without burst, even normal page loads with multiple assets can trigger the limit. A browser loading a page with 15-20 resources would get blocked on request #11 without burst.

nodelay matters. Without it, requests in the burst queue get delayed instead of processed immediately. Users would see random slowness and think the server is broken.

The $binary_remote_addr variable uses less memory than $remote_addr since it doesn't store the full IP string. With the 10m zone size you can track about 160k different IPs.

limit_conn is separate from limit_req. Request rate limiting controls how many requests per second. Connection limiting controls how many simultaneous connections. You want both.

For APIs specifically, returning a proper 429 status code with a JSON body is way better than the default 503. It lets clients implement proper retry logic and it doesn't skew your error monitoring.

I also added this to my log format so I can track rate limiting events:

log_format detailed '$remote_addr - $request_time $status '
                    'upstream_response_time=$upstream_response_time '
                    'request_limit=$limit_req_status '
                    '$request';

This lets me grep for rate limited requests and see patterns. If the same IP keeps hitting the limit, that's worth looking into.

The whole setup took maybe 15 minutes and it's saved me from going down twice since. Once from a Reddit hug of death and once from what looked like an actual scraping bot.

What rate limiting setup are you running? Do you handle it at the reverse proxy level or somewhere else like Cloudflare?