← Glossary / Request Batching

What is Request Batching?

Request batching is the technique of grouping multiple distinct data queries into a single HTTP payload to reduce network overhead, connection setup time, and API rate limit consumption. Instead of opening 100 separate TCP connections to fetch 100 product prices, a scraper sends one array of 100 SKUs to a GraphQL or REST endpoint. When implemented correctly, it drastically lowers egress costs and scraper latency; when abused, it triggers payload size limits and silent partial failures that corrupt downstream datasets.

Network LayerGraphQLAPI ScrapingThroughputLatency Optimization

// 02 — definitions

Pack it
tight.

Why sending one large payload is operationally superior to sending a thousand small ones, and how modern APIs handle the load.

Ask a DataFlirt engineer →

TL;DR

Request batching collapses multiple logical fetches into a single physical HTTP request. It is the primary throughput optimization for API scraping, bypassing the TCP handshake tax and connection pooling limits. However, it shifts the failure domain from the network layer (HTTP 500) to the application layer (HTTP 200 with partial errors in the JSON body).

01Definition & structure

Request batching is an application-layer optimization where a client groups multiple distinct data requirements into a single HTTP payload. Instead of making 50 separate GET /api/product/1 requests, the client makes one POST /api/products request with a JSON body containing [1, 2, ..., 50].

This approach drastically reduces network overhead by paying the TCP, TLS, and HTTP header tax only once per batch. It is highly prevalent in GraphQL APIs and modern REST endpoints designed for bulk data retrieval.

02REST vs GraphQL batching

In REST, batching is usually implemented via query parameters (?ids=A,B,C) or custom POST endpoints. It requires the backend developer to explicitly build support for array inputs.

In GraphQL, batching is often supported out-of-the-box by the server framework (like Apollo). A scraper can send a JSON array of query objects, and the server will execute them concurrently and return an array of results. This makes GraphQL targets exceptionally efficient to scrape if you understand the schema.

03The partial failure trap

The most dangerous aspect of batching is error handling. When you send 100 items, and 1 item causes a database error, the server will typically return an HTTP 200 OK. The error is buried inside the JSON payload (e.g., in an errors array).

Naive scrapers that only check response.status_code == 200 will silently drop the failed item. Production pipelines must parse the response, map the results back to the input array, and selectively requeue the missing items.

04How DataFlirt handles it

We treat batching as a dynamic variable, not a static config. Our extraction workers use an adaptive controller that scales batch sizes up until it detects latency degradation or WAF pushback, then settles at the optimal threshold.

Every batched response passes through a strict reconciliation layer. We assert that the number of returned records matches the number of requested inputs. Any delta triggers an automatic unbatched retry for the missing IDs, ensuring 100% data completeness without sacrificing bulk throughput.

05Did you know: WAF payload limits

Many scrapers assume they can batch 10,000 items into a single request. However, edge networks like Cloudflare and AWS WAF inspect request bodies for malicious payloads. If your batched JSON exceeds the WAF's inspection buffer (often around 128 KB), the request will be outright rejected with an HTTP 413 or 400 error, regardless of whether the backend API could have handled it.

// 03 — throughput math

Why batching
wins at scale.

The latency advantage of batching comes from eliminating the TCP/TLS handshake tax. DataFlirt's API scrapers dynamically tune batch sizes based on target response times and WAF payload limits to maximize throughput without triggering timeouts.

Unbatched latency = L = N × (T_handshake + T_processing)

Every single request pays the network setup penalty. Standard HTTP/1.1 model

Batched latency = L = T_handshake + (N × T_processing)

The handshake tax is paid exactly once per batch. Batching optimization

Optimal batch size = B = min(Limit_API, Limit_{WAF_bytes} / Bytes_query)

Bounded by what the target's infrastructure will accept before dropping the connection. DataFlirt adaptive controller

// 04 — the wire trace

One request,
fifty records.

A live trace of a batched GraphQL query hitting an e-commerce pricing API. Notice how the HTTP status is 200 OK, but the payload contains a mix of successes and application-level errors.

GraphQLHTTP/2JSON

edge.dataflirt.io — live

CAPTURED

// outbound payload
POST /graphql HTTP/2
content-length: 4192
[{"query":"...","variables":{"sku":"A1"}}, {"query":"...","variables":{"sku":"A2"}}, ...]

// inbound response
HTTP/2 200 OK
content-type: application/json

// parsed body
data[0].price: 149.99 // OK
data[1].price: 89.50 // OK
data[2].errors: ["SKU not found"] // WARN
data[49].price: 12.00 // OK

// extraction router
records.extracted: 49
records.failed: 1
action: quarantine failed SKU, write 49 to S3

// 05 — batching constraints

What limits
your batch size.

You cannot batch infinitely. APIs and infrastructure enforce hard limits on payload size, query complexity, and execution time. Exceeding these results in dropped connections or HTTP 413 errors.

AVG BATCH SIZE · · · 50–200 items

WAF LIMIT · · · · · ~128 KB typical

UPDATED · · · · · · 2026-05-19

WAF payload size limits

HTTP 413 · Cloudflare blocks bodies over 128MB on free tiers, often tuned lower

API-enforced item limits

App layer · Hardcoded max array lengths (e.g., max 100 IDs per request)

Database query timeouts

HTTP 504 · Backend takes too long to resolve a massive IN clause

GraphQL query complexity

App layer · AST depth and node count scoring blocks heavy queries

Memory consumption

Worker · Large JSON responses crash lightweight scraper processes

// 06 — our architecture

Dynamic batching,

tuned to the target's breaking point.

Static batch sizes fail when API latency fluctuates. DataFlirt uses an adaptive batching controller that monitors the target's response times and error rates in real time. If a target starts throwing 504 Gateway Timeouts at a batch size of 100, our workers automatically back off to 50, then 25, until the success rate stabilizes. We maximize throughput without crossing the target's operational redline.

Adaptive Batch Controller

Live metrics from a pricing pipeline adjusting batch sizes.

target.host api.retailer.com

batch.current 75 items

latency.p95 1.2s

error.rate 0.01%

waf.status clear

action scale up to 100

Stay ahead of the pipeline

Data engineering
intel, weekly.

Anti-bot shifts, scraping infrastructure updates, dataset delivery patterns, and business outcomes from our pipelines. Short, technical, no fluff.

// 07 — FAQ

Common
questions.

About batching mechanics, partial failures, rate limits, and how DataFlirt optimizes API throughput.

Ask us directly →

What is the difference between request batching and connection pooling? +

Connection pooling reuses a single TCP connection for multiple sequential HTTP requests, eliminating the handshake tax but still sending individual headers and payloads. Request batching groups multiple logical queries into a single HTTP payload. Batching is far more efficient because it reduces application-layer parsing overhead on the server and minimizes egress bandwidth.

How do you handle partial failures in a batched response? +

This is the hardest part of batching. A batched request usually returns an HTTP 200 OK even if 5 out of 100 items failed. You must parse the JSON response, map the returned data back to the original request array, identify the missing or errored items, and push those specific items back into the retry queue. If you just check the HTTP status code, you will silently lose data.

Does batching bypass API rate limits? +

It depends on how the target implements rate limiting. If the API limits by HTTP requests per minute, batching effectively multiplies your throughput. If the API limits by "credits" or "items processed" per minute, batching won't help you bypass the limit, but it will still reduce your network overhead and latency.

Why do I get HTTP 413 Payload Too Large errors when batching? +

You hit a Web Application Firewall (WAF) or reverse proxy limit. Systems like Nginx or Cloudflare have strict client_max_body_size configurations to prevent buffer overflow attacks. When your batched JSON payload exceeds this limit, the edge drops the connection before it even reaches the application server. You must reduce your batch size.

How does DataFlirt determine the optimal batch size? +

We don't guess. Our adaptive controller starts with a conservative batch size (e.g., 10) and scales up linearly while monitoring TTFB (Time to First Byte) and error rates. Once latency spikes non-linearly or the target throws a 5xx error, we record that as the ceiling and back off by 20%. This ensures we operate at maximum safe throughput.

Can you batch REST API requests? +

Yes, if the API supports it. Many REST APIs accept arrays of IDs (e.g., ?ids=1,2,3 or a POST body with an array). However, GraphQL is natively designed for batching, allowing you to send an array of distinct queries in a single POST request. For REST APIs that don't support arrays, you must rely on HTTP/2 multiplexing or connection pooling instead.

$ dataflirt scope --new-project --target=request-batching READY

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue dump or a continuous feed across millions of records — we scope, build, and operate the pipeline.

Start a pipeline → View pricing

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h

What is Request Batching?

Pack ittight.

TL;DR

Why batchingwins at scale.

One request,fifty records.

What limitsyour batch size.

WAF payload size limits

API-enforced item limits

Database query timeouts

GraphQL query complexity

Memory consumption

Dynamic batching,

Adaptive Batch Controller

Stay ahead of the pipeline

Data engineeringintel, weekly.

Commonquestions.

Tell us whatto extract.We do the rest.

Related glossary terms

Multiplexed Requests (HTTP/2)

Connection Pooling

HTTP Pipelining

Request Throttling