scalable-api

Design Scalable API - Serving Millions of Users

A scalable API system capable of serving millions of concurrent users with low latency, high availability, and graceful degradation. This educational project focuses on building production-grade API infrastructure with load balancing, caching, rate limiting, and observability.

Codebase Stats

Metric	Value
Total SLOC	7,541
Source Files	48
.js	3,188
.md	2,460
.tsx	826
.sql	397
.ts	271

Architecture Overview

                    ┌─────────────────┐
                    │   Frontend      │
                    │  (React + TS)   │
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │   API Gateway   │
                    │   (Port 8080)   │
                    │ - Rate Limiting │
                    │ - Auth          │
                    │ - Routing       │
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │  Load Balancer  │
                    │   (Port 3000)   │
                    │ - Health Checks │
                    │ - Circuit Break │
                    └────────┬────────┘
           ┌─────────────────┼─────────────────┐
           ▼                 ▼                 ▼
    ┌────────────┐    ┌────────────┐    ┌────────────┐
    │ API Server │    │ API Server │    │ API Server │
    │ (Port 3001)│    │ (Port 3002)│    │ (Port 3003)│
    └──────┬─────┘    └──────┬─────┘    └──────┬─────┘
           │                 │                 │
           └────────────┬────┴────────────────┘
                        │
         ┌──────────────┼──────────────┐
         ▼                             ▼
    ┌──────────┐                 ┌──────────┐
    │ PostgreSQL│                │   Redis  │
    │ (Port 5432)│               │(Port 6379)│
    └──────────┘                 └──────────┘

Key Features

1. High Availability

• Multiple API server instances with load balancing
• Health checks with automatic failover
• Circuit breakers for dependency protection

2. Performance

• Two-level caching (local + Redis)
• Connection pooling
• Response compression

3. Traffic Management

• Sliding window rate limiting
• Per-tier rate limits (anonymous, free, pro, enterprise)
• Request queuing and graceful degradation

4. Security

• Session-based authentication
• API key management
• Request validation and input sanitization

5. Observability

• Request/response logging with request IDs
• Prometheus-compatible metrics
• Real-time admin dashboard

Quick Start

Prerequisites

• Node.js 18+
• Docker and Docker Compose
• npm or yarn

Option 1: Docker (Recommended for Full Stack)

Start the entire stack with Docker:

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop all services
docker-compose down

Access points:

• Frontend: http://localhost:80
• API Gateway: http://localhost:8080
• Load Balancer: http://localhost:3000
• API Servers: http://localhost:3001, :3002, :3003

Option 2: Native Development (Recommended for Development)

Step 1: Start Infrastructure Services

# Start PostgreSQL and Redis only
docker-compose -f docker-compose.dev.yml up -d

Step 2: Install Backend Dependencies

cd backend
cp .env.example .env
npm install

Step 3: Run Backend Services

Open multiple terminal windows and run:

# Terminal 1: API Gateway
npm run dev:gateway

# Terminal 2: Load Balancer
npm run dev:lb

# Terminal 3: API Server 1
npm run dev:server1

# Terminal 4: API Server 2
npm run dev:server2

# Terminal 5: API Server 3
npm run dev:server3

# Or run all at once (requires concurrently)
npm run dev:all

Step 4: Run Frontend

cd frontend
npm install
npm run dev

Access points:

• Frontend: http://localhost:5173
• API Gateway: http://localhost:8080
• Load Balancer: http://localhost:3000

API Endpoints

Authentication

# Login
curl -X POST http://localhost:8080/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email": "admin@example.com", "password": "admin123"}'

# Response: {"token": "...", "user": {...}}

# Use token for authenticated requests
curl http://localhost:8080/api/v1/me \
  -H "Authorization: Bearer <token>"

Resources API

# List resources
curl http://localhost:8080/api/v1/resources

# Get single resource
curl http://localhost:8080/api/v1/resources/1

# Create resource (requires auth)
curl -X POST http://localhost:8080/api/v1/resources \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"name": "New Resource", "type": "document"}'

Admin Endpoints (requires admin role)

# Get dashboard data
curl http://localhost:8080/api/v1/admin/dashboard \
  -H "Authorization: Bearer <token>"

# View circuit breakers
curl http://localhost:8080/api/v1/admin/circuit-breakers \
  -H "Authorization: Bearer <token>"

# Clear cache
curl -X POST http://localhost:8080/api/v1/admin/cache/clear \
  -H "Authorization: Bearer <token>"

Health & Metrics

# Health check
curl http://localhost:8080/health

# Load balancer status
curl http://localhost:3000/lb/status

# Prometheus metrics
curl http://localhost:8080/metrics

Configuration

Environment Variables

Copy .env.example to .env in the backend directory:

# Server Configuration
PORT=3001
NODE_ENV=development
INSTANCE_ID=api-1

# PostgreSQL Configuration
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DB=scalable_api
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres

# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379

# Gateway Configuration
GATEWAY_PORT=8080

# Load Balancer Configuration
LB_PORT=3000
API_SERVERS=http://localhost:3001,http://localhost:3002,http://localhost:3003

# Rate Limiting
RATE_LIMIT_WINDOW_MS=60000
RATE_LIMIT_MAX_REQUESTS=100

# Cache TTL (seconds)
CACHE_TTL=300

# Circuit Breaker
CIRCUIT_FAILURE_THRESHOLD=5
CIRCUIT_RESET_TIMEOUT=30000

Rate Limit Tiers

Tier	Requests/Minute
Anonymous	100
Free	1,000
Pro	10,000
Enterprise	100,000

Testing the System

Test Rate Limiting

# Send many requests to trigger rate limiting
for i in {1..150}; do
  curl -s http://localhost:8080/api/v1/status | jq -r '.status // .error'
done

Test Load Balancing

# Observe requests being distributed across instances
for i in {1..10}; do
  curl -s http://localhost:3000/api/v1/status | jq -r '.instanceId'
done

Test Circuit Breaker

# Call external service endpoint (randomly fails)
for i in {1..20}; do
  curl -s http://localhost:8080/api/v1/external | jq -r '.data // .error'
  sleep 0.5
done

# Check circuit breaker state
curl http://localhost:8080/api/v1/admin/circuit-breakers \
  -H "Authorization: Bearer admin-token-dev"

Test Caching

# First request (cache miss)
time curl -s http://localhost:8080/api/v1/resources/1 > /dev/null

# Second request (cache hit - should be faster)
time curl -s http://localhost:8080/api/v1/resources/1 > /dev/null

# View cache stats
curl http://localhost:8080/api/v1/admin/cache \
  -H "Authorization: Bearer admin-token-dev"

Default Credentials

User	Email	Password	Role
Admin	admin@example.com	admin123	admin
User	user@example.com	user123	user

Project Structure

scalable-api/
├── backend/
│   ├── api-server/           # Individual API server instances
│   │   └── src/
│   │       └── index.js      # API server entry point
│   ├── gateway/              # API Gateway with rate limiting
│   │   └── src/
│   │       └── index.js      # Gateway entry point
│   ├── load-balancer/        # Load balancer with health checks
│   │   └── src/
│   │       └── index.js      # Load balancer entry point
│   ├── shared/               # Shared utilities and services
│   │   ├── config/           # Configuration management
│   │   ├── middleware/       # Express middleware (auth, logging)
│   │   ├── services/         # Core services (cache, circuit breaker, etc.)
│   │   └── utils/            # Utility functions
│   ├── package.json
│   └── .env.example
├── frontend/
│   ├── src/
│   │   ├── components/       # React components
│   │   ├── services/         # API client
│   │   └── stores/           # Zustand stores
│   └── package.json
├── database/
│   ├── schema.sql            # Database schema
│   └── migrations/           # Database migrations
├── docker-compose.yml        # Full stack deployment
├── docker-compose.dev.yml    # Development infrastructure
├── architecture.md           # System design documentation
└── README.md                 # This file

Technical Challenges Addressed

1. Horizontal Scaling: Stateless API servers behind a load balancer
2. Rate Limiting: Distributed sliding window using Redis
3. Caching: Two-level cache (local + Redis) with cache invalidation
4. Circuit Breakers: Per-dependency failure isolation
5. Observability: Metrics, logging, and real-time dashboard

Implementation Status

• API server framework with Express
• Load balancer with health checks
• API Gateway with routing
• Rate limiting (sliding window)
• Caching layer (local + Redis)
• Circuit breakers
• Authentication (session-based)
• Admin dashboard with metrics
• Docker deployment
• Database request logging
• Distributed tracing
• Prometheus/Grafana integration

Development Notes

See claude.md for development insights and design decisions. See architecture.md for detailed system design documentation.

References & Inspiration

• Building and Deploying Netflix API Gateway - Netflix Zuul API gateway architecture
• Kong API Gateway - Open-source API gateway and microservices management
• Rate Limiting Strategies and Techniques - Google Cloud's comprehensive guide to rate limiting
• Stripe's Rate Limiting - How Stripe implements rate limiting at scale
• Circuit Breaker Pattern - Martin Fowler on the circuit breaker pattern
• Resilience4j Circuit Breaker - Modern fault tolerance library patterns
• Load Balancing Algorithms - AWS guide to load balancing strategies
• NGINX Load Balancing Guide - Production load balancer configuration
• 12-Factor App Methodology - Best practices for building scalable, maintainable services
• Caching Best Practices - AWS caching strategies for web applications
• API Design Patterns - Google's API design guide for scalable APIs
• Envoy Proxy - Modern edge and service proxy for cloud-native apps