← Glossary / HTTP 521 Web Server Down (Cloudflare)

What is HTTP 521 Web Server Down (Cloudflare)?

HTTP 521 Web Server Down (Cloudflare) is an edge error indicating that Cloudflare successfully resolved the target's DNS, but the origin server actively refused the TCP connection. For scraping pipelines, this usually means the target infrastructure has crashed under load, or the origin's local firewall has aggressively dropped connections. It is a hard infrastructure failure, meaning immediate retries will only exacerbate the target's outage.

CloudflareOrigin ErrorTCP RefusedInfrastructureCircuit Breaker
// 02 — definitions

The origin
refused.

Cloudflare is up, the DNS is correct, but the actual server hosting the data slammed the door on the edge node.

Ask a DataFlirt engineer →

TL;DR

An HTTP 521 error means the origin server is offline or actively rejecting Cloudflare's connection attempts. Unlike a 503 (Service Unavailable) which is an HTTP-level response, a 521 is a TCP-level refusal. For data pipelines, it signals a critical need to halt requests and allow the target time to recover.

01Definition & structure
An HTTP 521 is a Cloudflare-specific status code. It occurs when Cloudflare's edge node attempts to establish a TCP connection with the target's origin server, but the origin responds with a TCP RST (Reset) packet, actively refusing the connection. To the scraper, the request fails instantly, but the failure point is behind the edge.
02Why it happens during scraping
In the context of data extraction, a 521 usually means your pipeline has overwhelmed the target. If you send 50 concurrent requests to a poorly optimized Magento or WordPress site, the origin's PHP workers or database connections max out. The origin's web server process crashes, and the operating system begins refusing new connections on port 443, triggering the 521 at the Cloudflare edge.
03The origin firewall misconfiguration
Sometimes, a 521 is caused by the target's own security software (like Fail2Ban or iptables) mistakenly identifying Cloudflare's IPs as an attack vector. Because Cloudflare proxies all traffic, the origin sees massive request volume coming from a handful of Cloudflare IPs. If the origin isn't configured to trust Cloudflare, it blocks them, taking the entire site offline and returning 521s to all visitors, including your scraper.
04How DataFlirt handles it
We treat 521s as a critical infrastructure signal. Our orchestration layer uses distributed circuit breakers. If a target returns a cluster of 521s, we immediately halt the queue for that domain. We do not rely on standard retries, which would only keep the target offline. Instead, we trickle a single probe request every 30 seconds until the origin stabilizes, ensuring our pipelines are good citizens of the web.
05521 vs 522 vs 520
Cloudflare's 52x series all indicate origin issues, but the mechanics differ. 521 means the connection was actively refused (TCP RST). 522 means the connection timed out (packets dropped, no response). 520 is a catch-all for when the origin connects but returns an empty, malformed, or protocol-violating response that Cloudflare cannot parse.
// 03 — backoff math

How to back off
when the origin dies.

When a target throws 521s, aggressive retries keep it offline. DataFlirt's circuit breakers use jittered exponential backoff to allow the origin time to recover without abandoning the crawl.

Exponential backoff = T = base · 2attempt + jitter
Standard retry delay calculation to prevent thundering herd problems. Network Reliability Engineering
Circuit breaker trip threshold = E521 / Rtotal > 0.05
If >5% of requests return 521 in a 60s window, the queue is paused. DataFlirt pipeline SLO
Recovery probe rate = Rprobe = 1 req / 30s
The trickle rate used to test if the origin has come back online. DataFlirt scheduler config
// 04 — edge trace

When the origin
drops the handshake.

A trace showing a scraper hitting a Cloudflare edge node, which then attempts and fails to establish a TCP connection with the origin server.

Cloudflare EdgeTCP RSTCircuit Breaker
edge.dataflirt.io — live
CAPTURED
// scraper -> cloudflare edge
GET /api/v1/inventory HTTP/2
Host: target-retailer.com

// cloudflare -> origin (internal)
TCP SYN -> 192.0.2.44:443
TCP RST <- 192.0.2.44:443 // connection refused

// edge response to scraper
HTTP/2 521 Web Server Down
cf-ray: 8daaf6152771b0da-LHR

// pipeline intervention
circuit_breaker: tripped
action: pause_target_queue
retry_scheduled: +120s
// 05 — root causes

Why the origin
refuses connections.

A 521 is a TCP-level rejection. Here is what typically causes the origin to actively refuse Cloudflare's connection during a high-volume scrape.

ERROR CLASS ·  ·  ·  ·    Infrastructure
RETRY SAFE? ·  ·  ·  ·    No (Backoff req)
UPDATED ·  ·  ·  ·  ·  ·  2026-05-19
01

Origin process crashed

OOM / CPU exhaustion · The web server daemon (Nginx/Apache) died under load.
02

Origin firewall blocking CF

Misconfiguration · Target's iptables blocked Cloudflare IPs by mistake.
03

Port 443/80 closed

Deployment error · Server is up, but not listening on web ports.
04

SSL/TLS daemon failure

Handshake failure · Origin cannot negotiate the internal TLS connection.
05

Bad Cloudflare DNS routing

Stale IP · Cloudflare is pointing to a decommissioned origin IP.
// 06 — pipeline resilience

Don't kick a server,

when it's already down.

A 521 error is a hard infrastructure failure. Continuing to send requests at your normal concurrency budget will only ensure the target stays offline longer. DataFlirt implements automatic circuit breaking at the edge: if a target returns >5% 521s in a one-minute window, we pause the queue, drop to a single probe request every 30 seconds, and only resume normal concurrency when the origin proves it has recovered. Sustainable scraping means knowing when to stop.

Circuit Breaker State

Live status of a pipeline hitting 521 errors on a fragile target.

target.domain fragile-catalog.com
error.rate_60s 12.4% (521s)
breaker.status OPENqueue paused
probe.interval 30s
probe.last_result 521 Web Server Down
concurrency.current 0 workers
alert.dispatched true

Stay ahead of the pipeline

Data engineering
intel, weekly.

Anti-bot shifts, scraping infrastructure updates, dataset delivery patterns, and business outcomes from our pipelines. Short, technical, no fluff.

By submitting, you acknowledge our Privacy Notice.

// 07 — FAQ

Common
questions.

Common questions about Cloudflare 521 errors, origin failures, and how to handle target downtime in production pipelines.

Ask us directly →
What is the difference between a 521 and a 522 error? +
A 521 (Web Server Down) means the origin actively refused the connection — it sent a TCP RST packet back to Cloudflare. A 522 (Connection Timed Out) means the origin silently dropped the packets or took too long to respond, resulting in a timeout. 521 is an active rejection; 522 is a lack of response.
Can my scraper's IP address cause a 521 error? +
No. A 521 error occurs between Cloudflare and the origin server. Your scraper's IP connects to Cloudflare. If Cloudflare blocks your IP, you will receive a 403 Forbidden or a 429 Too Many Requests. A 521 strictly means the target's own server is refusing Cloudflare's internal connection.
How long should I wait before retrying a 521 error? +
Implement exponential backoff starting at 30 seconds. Because a 521 usually indicates the target server has crashed or is rebooting, immediate retries will only add to the connection queue and delay the server's recovery. Never retry a 521 in a tight loop.
How does DataFlirt handle 521s at scale? +
We use distributed circuit breakers. If a target domain starts throwing 521s above a 5% threshold, the entire fleet pauses requests to that domain. We then send a single probe request every 30 seconds. Once the probe returns a 200 OK, the circuit breaker half-closes, slowly ramping concurrency back to the baseline.
Is it possible the origin is only blocking scrapers but allowing normal users? +
For a 521, this is highly unlikely. Cloudflare pools connections to the origin, meaning the origin sees Cloudflare's IPs, not yours. If the origin firewall blocks Cloudflare, the entire site goes down for everyone. If they wanted to block just you, they would configure Cloudflare WAF to return a 403.
What if the 521 only happens on specific heavy endpoints? +
This is common when scraping expensive search or export endpoints. The specific backend worker handling that route crashes (e.g., runs out of memory executing a massive database query), causing the local reverse proxy to refuse the connection. The solution is to reduce your pagination depth or add strict rate limits to those specific paths.
$ dataflirt scope --new-project --target=http-521-web-server-down-(cloudflare) READY

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue dump or a continuous feed across millions of records — we scope, build, and operate the pipeline.

Start a pipeline → View pricing
hello@dataflirt.com  ·  Bengaluru  ·  IST  ·  typical reply < 4h
// 08 — keep reading

Related glossary terms

Scraping Errors

HTTP 522 Connection Timed Out (Cloudflare)

When the origin server silently drops packets instead of actively refusing them.

Read →
Scraping Errors

HTTP 520 Unknown Error (Cloudflare)

When the origin returns an empty, malformed, or unexpected response to the edge.

Read →
Scraping Errors

HTTP 503 Service Unavailable

The standard HTTP response when a server is overloaded or down for maintenance.

Read →
Anti-Scraping

Cloudflare Error 1015 (Rate Limited)

When Cloudflare blocks your scraper directly for exceeding request thresholds.

Read →