Introduction to Rate Limiting with Redis

Rate limiting is a crucial technique used in API management to control the amount of incoming requests from users. By implementing rate limiting, you can prevent abuse, ensure fair usage, and maintain the performance of your services. Redis, an in-memory data structure store, is particularly effective for this purpose due to its speed and efficiency. Below are three practical examples illustrating how to implement rate limiting using Redis.

Example 1: Simple Token Bucket Algorithm

In this example, we will use the Token Bucket algorithm, which allows a user to make a fixed number of requests over a period of time. Redis will store the current token count for each user.

The context for this use case could be an API that allows users to fetch data. Each user can make a maximum of 100 requests per hour.

import redis
import time

# Connect to Redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)

def rate_limit(user_id):
    bucket_key = f"rate_limit:{user_id}"
    tokens = r.get(bucket_key)
    if tokens is None:
        # First request, initialize tokens
        r.set(bucket_key, 100, ex=3600)  # 100 tokens with 1 hour expiry
        tokens = 100
    else:
        tokens = int(tokens)

    if tokens > 0:
        r.decr(bucket_key)  # Use a token
        return True  # Request allowed
    else:
        return False  # Rate limit exceeded

# Simulating requests
user_id = "user123"
for i in range(105):  # Simulate 105 requests
    if rate_limit(user_id):
        print(f"Request {i + 1} allowed")
    else:
        print(f"Request {i + 1} denied")
        time.sleep(1)  # Wait before retrying

Notes:

• The r.set command initializes the token bucket with 100 tokens and sets an expiration time of 3600 seconds (1 hour).
• You can adjust the limit and time frame as per your requirements.

Example 2: Fixed Window Counter

This example demonstrates a fixed window rate limiting approach, where requests are counted in discrete time windows. This is useful for APIs that require strict limits within specific intervals.

Imagine an API allowing users to post comments, and you want to limit each user to 5 comments per minute.

import redis
import time

# Connect to Redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)

def rate_limit_fixed_window(user_id):
    current_time = int(time.time())
    window_key = f"window:{user_id}:{current_time // 60}"
    count = r.get(window_key)
    if count is None:
        r.set(window_key, 1, ex=60)  # 1 comment allowed, set expiry to 1 minute
        return True  # Request allowed
    elif int(count) < 5:
        r.incr(window_key)  # Increment count
        return True  # Request allowed
    else:
        return False  # Rate limit exceeded

# Simulating requests
user_id = "user456"
for i in range(10):  # Simulate 10 comments
    if rate_limit_fixed_window(user_id):
        print(f"Comment {i + 1} allowed")
    else:
        print(f"Comment {i + 1} denied")
        time.sleep(1)  # Wait before retrying

Notes:

• The window_key is based on the current minute, allowing for a reset every minute.
• The limits can be modified based on your application’s needs.

Example 3: Sliding Log Algorithm

The sliding log algorithm provides a more dynamic rate limiting method, where requests are logged in real-time. This approach is beneficial for scenarios requiring a more granular control over request traffic.

Consider a scenario where users can make 10 requests every 10 minutes.

import redis
import time

# Connect to Redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)

def rate_limit_sliding_log(user_id):
    log_key = f"log:{user_id}"
    current_time = int(time.time())
    expiry_time = current_time - 600  # 10 minutes ago

    # Remove outdated entries
    r.zremrangebyscore(log_key, 0, expiry_time)
    request_count = r.zcard(log_key)

    if request_count < 10:
        r.zadd(log_key, {current_time: current_time})  # Add current request time
        r.expire(log_key, 600)  # Set expiry to 10 minutes
        return True  # Request allowed
    else:
        return False  # Rate limit exceeded

# Simulating requests
user_id = "user789"
for i in range(15):  # Simulate 15 requests
    if rate_limit_sliding_log(user_id):
        print(f"Request {i + 1} allowed")
    else:
        print(f"Request {i + 1} denied")
        time.sleep(1)  # Wait before retrying

Notes:

• The zremrangebyscore command removes requests older than 10 minutes to maintain an accurate count.
• This method allows for a smoother rate limiting experience, especially during burst traffic.

By utilizing these examples of rate limiting with Redis, you can effectively manage API traffic, ensuring that your services remain reliable and performant.