How to Fix Bad Gateway 504 Errors and Prevent Them

You're on call, the dashboard is red, and the error budget is burning while users keep refreshing the same page. The response in the logs says 504 Gateway Timeout, which usually means the request made it to a gateway or proxy, but the upstream didn't answer before the clock ran out. That makes this a coordination problem, not a browser mystery.

For a quick runbook on gateway failures, keep Sota Proxy 503 error fix handy as a companion reference, because 503 and 504 often get confused under pressure.

What a 504 Bad Gateway Error Means

A 504 Bad Gateway is an HTTP server error that means a gateway or proxy did not get a timely response from the upstream server. The key distinction is that this is not the same as a 502, where the upstream response is invalid or the connection fails in a different way. With a 504, the upstream may still be alive, it just did not finish before the timeout window closed, as described in the HTTP standard and MDN's reference on 504 Gateway Timeout.

That distinction matters during an outage. A browser can be healthy, the proxy can still be connected, and the application can still time out because the request path is slower than the gateway allows. Netdata notes that NGINX's proxy_read_timeout defaults to 60 seconds, so backend work that crosses that line can surface as a 504 even while the origin keeps grinding away in the background. That is why 504s usually point to backend latency, not client-side failure.

The common triggers are familiar to anyone who has watched a production stack slow down under load, slow database queries, thread pool exhaustion, long-running computations, and overloaded upstream servers. Visual Sentinel's traffic sample reinforces the operational value of that signal, because it saw 4 genuine 502 Bad Gateway responses and zero 504s across 2.5 million HTTP checks in its sample, and it described 504s as rarer than 502s in the wild in its own analysis at What is 504 Gateway Timeout. When a 504 does show up, treat it as a latency-management failure that deserves attention, not a routine hiccup.

Practical rule: if the gateway timed out, start by asking which hop got slow, not whether the browser cache is stale.

A cleaner mental model views 504s as a coordination failure between a proxy and the service behind it. The request path exists, but the response missed the deadline.

In practice, that means you look at timeout matching across the chain, not just one config file. If the edge waits longer than the reverse proxy, or the reverse proxy waits longer than the application server, you can create false timeouts in either direction. The same is true deeper in the stack, including private links and internal services, so private network routing and timeout paths need to be part of the review when the upstream lives behind another hop.

For operators, the trade-off is simple. Longer timeouts can hide brief slowdowns and reduce noisy failures, but they also keep bad requests and stuck workers tied up longer. Shorter timeouts fail faster and free resources sooner, but they will surface 504s sooner if the backend is still doing legitimate work. That is why matching the timeout budget to the actual request path matters more than picking a number that feels safe.

Tracing the Request Path From Browser to Upstream

The full request path is where the diagnosis starts. The browser sends the request, then the CDN or edge, the load balancer, the reverse proxy, the application server, and finally the database or downstream microservice all take a turn at the clock. A 504 is the symptom the user sees when one of those hops waits too long for the next one to answer.

CloudFront is a good example at the edge. AWS says CloudFront returns a 504 when its origin fetch gets an HTTP 504 from the origin or when the origin doesn't respond before the request expires, and blocked traffic from a firewall, security group, or lack of internet access can produce the same result, according to AWS CloudFront 504 guidance. Cloudflare similarly treats 502 or 504 as a sign it can't establish contact with the origin web server, which helps separate proxy-to-origin problems from a broken client, as noted in its HTTP 5xx error troubleshooting.

Default Timeouts and Config Knobs by Layer

One useful internal reference for hop-by-hop isolation is Private Network considerations, because origin reachability, not just app code, can determine whether a gateway ever gets a response.

Targeted Fixes for Nginx, Apache, HAProxy, ELB, CloudFront, and API Gateway

Nginx is usually the first place people look, and for good reason. proxy_connect_timeout, proxy_send_timeout, and proxy_read_timeout control how long Nginx will wait at different stages of the upstream conversation, and the symptom they target is the classic “upstream was connected, but too slow to finish.” Raising proxy_read_timeout can buy time for long operations, but it's a trap if the backend is already saturated. You're masking delay, not removing it.

Apache's ProxyTimeout plays the same game at the front door, while worker settings influence how many requests can even progress at once. If the worker pool is tight, requests back up and time out before they reach useful work. That's why a bigger timeout alone can make the outage look quieter without making the system faster.

HAProxy gives you a sharper set of controls with timeout connect, timeout server, and related HTTP request timing directives. These matter when a backend connection is established but the response drags. The fix targets slow upstreams, but the caution is the same, don't stretch the timeout so far that bad latency becomes invisible.

AWS ELB and CloudFront both have timeout behavior that can create 504s when the origin is slow or unreachable. AWS also notes blocked traffic from a security group or firewall can produce the same outcome at the CloudFront layer, so don't waste time tuning app code until origin reachability is checked. A useful internal companion for the NGINX side of this is Forbidden 403 NGINX, because access-control and timeout failures often sit near each other in the same stack.

Apigee's 504 path is a little different. It returns 504 when the client doesn't get a timely response from the Edge Gateway or backend during API execution, which means the gateway's own processing time matters too, not just your service code. For long-running endpoints, increase integration timeouts only where the request is meant to take that long, and verify the before-and-after timing with logs so the change doesn't become a permanent workaround for a slow dependency.

Timeouts should match the slowest legitimate endpoint, not the slowest endpoint you've learned to tolerate.

Useful Commands and Logs for Fast Triage

A live 504 needs evidence fast, not theory. Start with curl and timing flags so you can see where the request spent its budget, then compare the direct backend path with the proxied path. If the backend is quick directly but slow through the proxy, you've localized the bottleneck to the gateway layer or a hop before it.

Nginx logs are the next stop. Error logs will show whether the failure came from reading a response header, sending a request upstream, or waiting on the upstream itself. Access logs are even better when they include upstream timing fields, because they let you line up request time with upstream response time and spot the requests that consistently burn the whole budget.

HAProxy and cloud logs round out the picture. show stat tells you whether an upstream pool is unhealthy or stuck, CloudWatch can expose whether the load balancer is dropping or timing out requests, and CloudFront edge logs show whether the failure happened before the origin ever answered. Apigee operators should inspect Message Processor or Router logs for slow backend calls, because the gateway can time out on its own path too.

Confirm scope, isolate the layer, check upstream health, then inspect config diffs.

A short triage sequence usually beats a long blame discussion. If you can prove the request is slow only when it crosses a proxy, you've already narrowed the problem to timeout alignment, backend saturation, or a network-path issue.

Monitoring and Alerting That Catches 504s Early

Alerting on 5xx counts alone is too late. By the time a proxy emits a 504, latency has already crossed a hard threshold, so the warning sign is upstream latency drifting toward that line. Watch upstream P95 latency, origin response time, and proxy-upstream queue depth together, because those signals show stress before the gateway has to give up.

The most useful threshold is the one that leaves room. If origin latency is climbing to the point where it regularly sits near the proxy timeout, the system is already living too close to the edge. Queue depth rising sharply is another bad sign, because it usually means work is stacking up faster than the upstream pool can clear it.

Build the dashboard to show the gap between current P95 and the configured timeout, not just whether an error happened. That makes it obvious when a service is operating with too little headroom. It also helps operators decide whether to fix capacity, tune a timeout, or change the request pattern.

For a practical discussion of dashboard design, the internal guide operational dashboard is a useful complement. And if retries are part of your design, Tagada's smart retry tool is a good reference point for thinking about retry behavior without turning transient slowness into extra load.

Visual Sentinel's sample is a reminder that 504s are uncommon, with zero 504s across 2.5 million HTTP checks in that dataset, so when one appears it deserves a real incident, not a shrug. Make the alert rare, specific, and tied to latency behavior that predicts the timeout instead of just counting the timeout after it happens.

Preventing 504s With Capacity, Retries, and Smart Scheduling

The cheapest 504 is the one that never happens. Capacity planning is the first line of defense, because upstream pools need to handle peak traffic, not the average day that looks nice on a graph. If you only size for average load, the system will punish you the first time work piles up.

Retries can help, but only when they're disciplined. Exponential backoff with jitter keeps one slow call from turning into a storm of duplicate pressure, and idempotency makes those retries safe when the first attempt might still be working. A retry policy that ignores safety just creates a second outage with better intentions.

Maintenance timing matters too. Reboots, resizes, and patching should land in low-traffic windows so planned work doesn't look like an origin failure to the proxy. That's a scheduling problem as much as a systems problem, and it's one of the easiest ways to avoid self-inflicted 504s.

For broader capacity strategy, capacity planning software is a relevant internal read because the same planning discipline that protects spend also protects latency. Blanket timeout increases are the wrong default. Tune timeouts to the slowest endpoint that needs the headroom, then fix the endpoint so the timeout isn't doing silent damage control forever.

Operational Checklist and How Server Scheduler Helps

A clean 504 runbook starts with scope, then moves to the layer, then the upstream. Confirm whether the failure is isolated to one route or one region, check whether the proxy or gateway is the thing timing out, and then inspect the health of the backend service and database. After that, match the timeout chain so one hop is not stricter than the rest of the path can realistically satisfy.

The prevention side is operational too. Schedule maintenance deliberately, keep retries safe, and watch latency before it crosses the gateway deadline. If reboots and resize actions happen in the middle of peak demand, the proxy sees a timeout even when the operator only meant to do routine work.

Server Scheduler fits that discipline because predictable maintenance is part of reliability, not separate from it. When teams define start, stop, resize, and reboot windows ahead of time, they cut down on surprise restarts that can look like backend slowness and force gateways into 504s. For teams running AWS environments, Server Scheduler gives those actions a visual time grid, which helps keep planned changes out of the traffic path and keeps the on-call queue quieter.

Predictable maintenance is a form of 504 prevention.