Add nginx load balancing support for multi-instance Streamlit deployments (#336)

t0mdavid-m · claude · t0mdavid-m · commit 20cf2de00b6e · 2026-02-09T18:35:44.000+02:00
* Add nginx load balancer for scaling Streamlit in a single container When STREAMLIT_SERVER_COUNT > 1, the entrypoint dynamically generates an nginx config and launches multiple Streamlit instances on internal ports (8510+), with nginx on port 8501 using ip_hash sticky sessions for WebSocket compatibility. Default (STREAMLIT_SERVER_COUNT=1) preserves existing behavior with no nginx overhead. https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Fix nginx config: create /etc/nginx directory before writing config https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Fix nginx: use absolute path /usr/sbin/nginx The mamba environment activation shadows system binaries on the PATH. https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Switch nginx from ip_hash to least_conn for load balancing ip_hash pins all users behind the same NAT/VPN/reverse-proxy to a single backend, defeating the load balancer. least_conn distributes new connections to the instance with fewest active connections, and once a WebSocket is established it stays on that backend for the session lifetime, so sticky sessions are not needed. https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Fix file uploads: disable nginx client_max_body_size limit nginx defaults to 1MB max body size, which blocks Streamlit file uploads with a 400 error. Set to 0 (unlimited) to let Streamlit enforce its own 200MB limit from config.toml. https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Fix file uploads: switch to hash-based sticky sessions least_conn routes each HTTP request independently, so the file upload POST (/_stcore/upload_file) can land on a different backend than the WebSocket session, causing a 400 error. Use hash $remote_addr$http_x_forwarded_for consistent instead: - Provides session affinity so uploads hit the correct backend - Behind a reverse proxy: XFF header differentiates real client IPs - Direct connections: falls back to remote_addr (like ip_hash) - "consistent" minimizes redistribution when backends are added/removed https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 * Implement cookie-based sticky sessions for nginx load balancer Replace ip_hash/hash-on-IP with cookie-based session affinity using nginx's built-in map and $request_id: - map $cookie_stroute $route_key: if browser has a "stroute" cookie, reuse its value; otherwise fall back to $request_id (a unique random hex string nginx generates per-request) - hash $route_key consistent: route based on the cookie/random value - add_header Set-Cookie on every response to persist the routing key This ensures each browser gets its own sticky backend regardless of source IP, fixing both: - File uploads (POST must hit the same backend as the WebSocket session) - Load distribution when all users share the same IP (NAT/VPN/proxy) No new packages required - uses only built-in nginx directives. https://claude.ai/code/session_018VEL5xKZfe4LCcUa8iUHJ9 --------- Co-authored-by: Claude <noreply@anthropic.com>
diff --git a/Dockerfile b/Dockerfile
@@ -58,6 +58,7 @@ RUN mamba install pip
 RUN python -m pip install --upgrade pip
 RUN python -m pip install -r requirements.txt
 
+RUN apt-get update && apt-get install -y --no-install-recommends nginx
 
 # create workdir and copy over all streamlit related files/folders
 WORKDIR /app
@@ -81,11 +82,56 @@ COPY clean-up-workspaces.py /app/clean-up-workspaces.py
 # add cron job to the crontab
 RUN echo "0 3 * * * /root/miniforge3/envs/streamlit-env/bin/python /app/clean-up-workspaces.py >> /app/clean-up-workspaces.log 2>&1" | crontab -
 
-# create entrypoint script to start cron service and launch streamlit app
-RUN echo "#!/bin/bash" > /app/entrypoint.sh
-RUN echo "source /root/miniforge3/bin/activate streamlit-env" >> /app/entrypoint.sh && \
-    echo "service cron start" >> /app/entrypoint.sh && \
-    echo "streamlit run app.py" >> /app/entrypoint.sh
+# Set default worker count (can be overridden via environment variable)
+ENV RQ_WORKER_COUNT=1
+ENV REDIS_URL=redis://localhost:6379/0
+
+# Number of Streamlit server instances for load balancing (default: 1 = no load balancer)
+# Set to >1 to enable nginx load balancer with multiple Streamlit instances
+ENV STREAMLIT_SERVER_COUNT=1
+
+# create entrypoint script to start cron, Redis, RQ workers, and Streamlit
+RUN echo -e '#!/bin/bash\n\
+set -e\n\
+source /root/miniforge3/bin/activate streamlit-env\n\
+\n\
+# Start cron for workspace cleanup\n\
+service cron start\n\
+\n\
+# Load balancer setup\n\
+SERVER_COUNT=${STREAMLIT_SERVER_COUNT:-1}\n\
+\n\
+if [ "$SERVER_COUNT" -gt 1 ]; then\n\
+    echo "Starting $SERVER_COUNT Streamlit instances with nginx load balancer..."\n\
+\n\
+    # Generate nginx upstream block\n\
+    UPSTREAM_SERVERS=""\n\
+    BASE_PORT=8510\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        UPSTREAM_SERVERS="${UPSTREAM_SERVERS}        server 127.0.0.1:${PORT};\\n"\n\
+    done\n\
+\n\
+    # Write nginx config\n\
+    mkdir -p /etc/nginx\n\
+    echo -e "worker_processes auto;\\npid /run/nginx.pid;\\n\\nevents {\\n    worker_connections 1024;\\n}\\n\\nhttp {\\n    client_max_body_size 0;\\n\\n    map \\$cookie_stroute \\$route_key {\\n        \\x22\\x22      \\$request_id;\\n        default \\$cookie_stroute;\\n    }\\n\\n    upstream streamlit_backend {\\n        hash \\$route_key consistent;\\n${UPSTREAM_SERVERS}    }\\n\\n    map \\$http_upgrade \\$connection_upgrade {\\n        default upgrade;\\n        \\x27\\x27 close;\\n    }\\n\\n    server {\\n        listen 8501;\\n\\n        location / {\\n            proxy_pass http://streamlit_backend;\\n            proxy_http_version 1.1;\\n            proxy_set_header Upgrade \\$http_upgrade;\\n            proxy_set_header Connection \\$connection_upgrade;\\n            proxy_set_header Host \\$host;\\n            proxy_set_header X-Real-IP \\$remote_addr;\\n            proxy_set_header X-Forwarded-For \\$proxy_add_x_forwarded_for;\\n            proxy_set_header X-Forwarded-Proto \\$scheme;\\n            proxy_read_timeout 86400;\\n            proxy_send_timeout 86400;\\n            proxy_buffering off;\\n            add_header Set-Cookie \\x22stroute=\\$route_key; Path=/; HttpOnly; SameSite=Lax\\x22 always;\\n        }\\n    }\\n}" > /etc/nginx/nginx.conf\n\
+\n\
+    # Start Streamlit instances on internal ports (localhost only)\n\
+    for i in $(seq 0 $((SERVER_COUNT - 1))); do\n\
+        PORT=$((BASE_PORT + i))\n\
+        echo "Starting Streamlit instance on port $PORT..."\n\
+        streamlit run app.py --server.port $PORT --server.address 127.0.0.1 &\n\
+    done\n\
+\n\
+    sleep 2\n\
+    echo "Starting nginx load balancer on port 8501..."\n\
+    exec /usr/sbin/nginx -g "daemon off;"\n\
+else\n\
+    # Single instance mode (default) - run Streamlit directly on port 8501\n\
+    echo "Starting Streamlit app..."\n\
+    exec streamlit run app.py\n\
+fi\n\
+' > /app/entrypoint.sh
 # make the script executable
 RUN chmod +x /app/entrypoint.sh
 
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -12,6 +12,9 @@ services:
       - 8501:8501
     volumes:
       - workspaces-streamlit-template:/workspaces-streamlit-template
-    command: streamlit run openms-streamlit-template/app.py
+    environment:
+      # Number of Streamlit server instances (default: 1 = no load balancer).
+      # Set to >1 to enable nginx load balancing across multiple Streamlit instances.
+      - STREAMLIT_SERVER_COUNT=1
 volumes:
   workspaces-streamlit-template:
