Hiding Downtime Behind a Proxy

The Mechanism in One Sentence

A reverse proxy with proxy_request_buffering on (nginx's default) owns the TCP connection to the client independently of the connection to the upstream. The upstream can die mid-request and the client notices nothing: the proxy already holds the bytes.

This post covers how that works down to the kernel socket buffers, then builds a demo: a Node.js upload server, an nginx proxy, a 500 MB transfer, and a kill -9 on the backend halfway through.

Architecture

Two independent TCP connections. This is not a transparent passthrough but a terminating proxy that decouples the client's session from the upstream's.

What nginx Actually Does with a Large Upload

A buffered upload moves through three phases. The upstream can be dead for the first two and the client never notices.

Phase 1, Client body buffering

When a POST /upload arrives, nginx checks proxy_request_buffering. With the default value of on:

1. nginx reads the body from conn A into client_body_buffer_size of memory (default 8k/16k), spilling to client_body_temp_path on disk (default /var/lib/nginx/tmp/client_body on Debian) once that fills.
2. It keeps draining conn A at the client's send rate, ACK-ing segments, while the upstream is not contacted yet.

Phase 2, Upstream connection

Only after the entire client body is buffered does nginx open conn B, reconstruct the request from the temp file, and stream it upstream. This is why a 9 GB upload keeps “uploading at realistic speeds” while the upstream is offline: the bytes are going to the proxy's disk, not the application.

Phase 3, Upstream recovery

When the upstream returns, nginx opens a fresh connection, replays the buffered request from the temp file, then discards the file and forwards a single contiguous response. The client never saw a TCP reset or a 502.

Kernel-Level Details

Socket receive buffers

Every TCP socket has a kernel receive buffer sized by net.core.rmem_default and net.ipv4.tcp_rmem. As long as nginx keeps calling recv() to drain it, the kernel keeps ACK-ing the client's segments and the window stays open, so the upload runs at full speed.

Check current buffer sizes:

sysctl net.ipv4.tcp_rmem
# output: net.ipv4.tcp_rmem = 4096  131072  6291456
# min / default / max in bytes

sendfile and splice

Forwarding the temp file to the upstream, nginx uses sendfile(2) (or splice(2) when sendfile is off) to move data between a file descriptor and a socket without a userspace copy, a kernel zero-copy path:

No data traverses userspace during the relay, which is why the replay is fast even on a low-power device.

inode lifecycle of the temp file

nginx creates the temp file with O_TMPFILE (or open() + unlink()), so it is unlinked immediately and the inode persists only via a file descriptor held by the worker. If the worker dies, the kernel drops the fd and reclaims the inode. No cleanup step is needed.

Hands-On Demo

Prerequisites

# nginx
sudo apt install nginx

# Node.js (upload server)
node --version  # >= 18

# tools
which curl pv  # pv gives you a live transfer rate meter

Step 1, The upstream upload server

// server.js
import http from 'http';
import fs from 'fs';
import path from 'path';
import { pipeline } from 'stream/promises';

const UPLOAD_DIR = '/tmp/uploads';
fs.mkdirSync(UPLOAD_DIR, { recursive: true });

const server = http.createServer(async (req, res) => {
  if (req.method === 'POST' && req.url === '/upload') {
    const filename = `upload-${Date.now()}.bin`;
    const dest = path.join(UPLOAD_DIR, filename);
    const out = fs.createWriteStream(dest);

    let received = 0;
    req.on('data', chunk => {
      received += chunk.length;
      process.stdout.write(`\r[upstream] received ${(received / 1024 / 1024).toFixed(1)} MB`);
    });

    try {
      await pipeline(req, out);
      console.log(`\n[upstream] write complete: ${dest}`);
      res.writeHead(200, { 'Content-Type': 'application/json' });
      res.end(JSON.stringify({ ok: true, file: filename, bytes: received }));
    } catch (err) {
      console.error('\n[upstream] stream error:', err.message);
      res.writeHead(500);
      res.end();
    }
    return;
  }

  if (req.url === '/health') {
    res.writeHead(200);
    res.end('OK\n');
    return;
  }

  res.writeHead(404);
  res.end();
});

const PORT = 3000;
server.listen(PORT, '127.0.0.1', () => {
  console.log(`[upstream] listening on 127.0.0.1:${PORT} (pid ${process.pid})`);
});

fs.writeFileSync('/tmp/upstream.pid', String(process.pid));

Start it:

node server.js
# [upstream] listening on 127.0.0.1:3000 (pid 12345)

Step 2, nginx configuration

# /etc/nginx/sites-available/upload-proxy

client_max_body_size        0;          # no size limit; default is 1m
client_body_buffer_size     128k;       # in-memory before spilling to disk
client_body_temp_path       /tmp/nginx_client_temp 1 2;

proxy_request_buffering     on;         # THE key directive
proxy_read_timeout          3600s;
proxy_send_timeout          3600s;
proxy_connect_timeout       75s;

upstream app_backend {
    server 127.0.0.1:3000;
    keepalive 4;
}

server {
    listen 8080;
    server_name localhost;

    location /upload {
        proxy_pass         http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header   Connection "";
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   Content-Type $content_type;
    }

    location /health {
        proxy_pass http://app_backend;
    }

    error_page 502 503 504 /offline.html;
    location = /offline.html {
        root /var/www/html;
        internal;
    }
}

sudo mkdir -p /tmp/nginx_client_temp/{1,2}
sudo chown -R www-data:www-data /tmp/nginx_client_temp

sudo ln -s /etc/nginx/sites-available/upload-proxy /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

Step 3, Generate a test file

# 500 MB of pseudorandom data
dd if=/dev/urandom bs=1M count=500 of=/tmp/testfile.bin status=progress

Step 4, Upload through the proxy, kill the upstream midway

Terminal 1, start the upload and watch with pv:

pv /tmp/testfile.bin | curl \
  -X POST \
  http://localhost:8080/upload \
  -H "Content-Type: application/octet-stream" \
  --data-binary @-

pv prints a live rate (e.g. 247MB 0:00:12 [19.8MB/s]). Wait until roughly half the file has transferred, then in Terminal 2 kill the upstream:

kill -9 $(cat /tmp/upstream.pid)

Observe Terminal 1: the pv rate does not drop. The transfer continues. The bytes are going into nginx's temp path, not to the dead Node.js process.

Watch the temp file grow on disk:

watch -n1 'ls -lah /tmp/nginx_client_temp/**/*'
# You'll see a temp file growing in real time

Restart the upstream:

node server.js &

After a few seconds nginx opens a new connection, replays the buffered body, and curl finally receives a 200 OK with the JSON response.

Verifying the Behavior at the TCP Level

Trace the TCP connections while the upload runs:

ss -tnp | grep -E '8080|3000'

You will see two distinct connections:

State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port  Process
ESTAB   0       0       127.0.0.1:8080      127.0.0.1:XXXXX    nginx
ESTAB   0       0       127.0.0.1:YYYYY     127.0.0.1:3000     nginx

After kill -9 on the upstream, the second connection disappears but the first stays ESTAB. To watch the ACK stream directly:

sudo tcpdump -i lo -n 'tcp port 8080' -l | grep -E 'flags \[.\]'
# You will see continuous ACK flags from nginx → client during the buffering phase

The proxy_request_buffering off Case

With proxy_request_buffering off, nginx becomes a streaming proxy: it opens the upstream connection immediately and pipes bytes both ways concurrently. No temp file, lower first-byte latency, but the isolation is gone. If the upstream drops before the client finishes, the ECONNRESET propagates straight to the client as an abrupt mid-transfer termination.

# With this setting the isolation guarantee is gone
proxy_request_buffering off;

This is correct for streaming protocols (WebSockets, gRPC, SSE, large video where you cannot buffer the whole body). For file uploads where resilience matters, leave it at the default on.

The Graceful Error Path

Buffering hides an upstream that dies mid-upload. A fresh request arriving while the upstream is fully down is different: nginx cannot connect and returns 502 Bad Gateway, which the error_page directive intercepts to serve a static page instead.

This is the other half of the story: the Raspberry Pi running nginx stayed up through a power failure. Even with the upstream fully down, any new request got a clean 200 OK offline page rather than a raw 502.

error_page 502 503 504 =200 /offline.html;
# =200 rewrites the status code, useful for UX
# Omit it if you want to preserve the 5xx for monitoring

Put a self-contained HTML file at /var/www/html/offline.html, served without hitting the upstream (hence internal;).

Configuring the Temp Path for Production

The default client_body_temp_path on most Linux systems is /var/lib/nginx/tmp/client_body. For production:

# Fast SSD, survives reboots
client_body_temp_path /mnt/fast-ssd/nginx/client_temp 1 2;

# tmpfs, low-latency, but buffered uploads are lost on reboot
client_body_temp_path /dev/shm/nginx_client_temp;

The 1 2 arguments create a two-level hashed directory structure, avoiding thousands of files in one directory (a performance problem at scale). Ensure the nginx worker user owns it:

install -d -o www-data -g www-data -m 0700 /mnt/fast-ssd/nginx/client_temp

What About Multipart and Chunked Uploads?

Transfer-Encoding: chunked

nginx fully supports chunked bodies. It dechunks the stream while buffering, then re-frames it as a normal Content-Length request upstream, so the upstream never needs to know the client used chunked encoding.

Multipart (multipart/form-data)

Browser uploads via <input type="file"> send multipart/form-data. nginx buffers the whole body the same way without parsing the parts: the temp file is the raw MIME body, and the upstream receives it intact.

Resumable uploads (TUS protocol)

TUS uses PATCH requests with Upload-Offset headers to resume at a byte offset, and nginx buffers each small PATCH individually. That is more resilient than a monolithic PUT: if the upstream dies mid-PATCH, only that patch fails and the client retries from the last offset.

Timeouts to Tune

Buffering large files, the default nginx timeouts are too short:

# How long nginx waits for the client to send the body (per read, not total)
client_body_timeout    60s;   # default; fine for fast LANs, too short for slow WAN

# How long nginx waits for the upstream to accept the connection
proxy_connect_timeout  75s;   # fine

# How long nginx waits between successive reads from the upstream response
proxy_read_timeout     3600s; # default is 60s, INCREASE for large uploads
                              # covers the time the upstream spends writing to disk

# How long nginx waits between successive writes to the upstream
proxy_send_timeout     3600s; # default is 60s, INCREASE for the relay phase

The critical one is proxy_read_timeout. Relaying a 9 GB body, the upstream may spend minutes writing to disk; if the timeout fires first, nginx returns a 504 Gateway Timeout even though the upload finished. Increase it aggressively for upload endpoints.

Monitoring the Proxy Buffer State

Expose stub status to watch active connections:

location /nginx_status {
    stub_status;
    allow 127.0.0.1;
    deny all;
}

curl -s http://localhost/nginx_status
# Active connections: 3
# server accepts handled requests
#  1234 1234 1567
# Reading: 1 Writing: 2 Waiting: 0

Reading: N is the number of clients whose request bodies nginx is reading. Elevated while the upstream is down, those are active uploads buffering to disk.

Log the buffer size per request:

log_format upload_log '$remote_addr - $request_length bytes '
                      '[$time_local] "$request" $status '
                      'upstream_response_time=$upstream_response_time';
access_log /var/log/nginx/uploads.log upload_log;

$request_length is the total request size including body, useful for correlating temp file growth with uploads.

Practical Implications

Homelab (Raspberry Pi as a proxy in front of a home server)

• Point client_body_temp_path to a USB SSD, not the SD card, whose write endurance a 9 GB upload burns through.
• Set client_max_body_size 0 to remove the default 1 MB body size cap.
• With more than one backend, proxy_next_upstream error timeout retries the next healthy server automatically.

Production (Kubernetes ingress, cloud load balancers)

• AWS ALB and GCP HTTPS LBs both buffer request bodies by default, identical to nginx proxy_request_buffering on.
• The nginx ingress controller defaults to proxy-request-buffering: "on", but the pod's temp path is ephemeral: if the ingress pod restarts mid-upload, the buffer is gone.
• For that reason, large Kubernetes uploads should go through TUS or a presigned S3 URL rather than the ingress.

Summary

A proxy does not merely route requests. Configured correctly, it owns each side independently, decoupling client and upstream at the TCP level. Upstream restarts, OOM kills, and power failures become invisible to the client as long as the proxy stays up and has disk to absorb the upload.

Further Reading

• nginx proxy_request_buffering docs
• nginx client_body_buffer_size docs
• Linux sendfile(2) man page
• TUS resumable upload protocol
• RFC 7230, HTTP/1.1 Message Syntax and Routing