← Glossary / Mitmproxy

What is Mitmproxy?

Mitmproxy is an open-source, interactive HTTPS proxy used to intercept, inspect, and modify web traffic on the fly. For scraping engineers, it is the primary diagnostic tool for reverse-engineering undocumented mobile APIs, debugging TLS fingerprinting failures, and capturing the exact request headers a native client sends. If a target app works on your phone but your scraper gets a 403, mitmproxy is how you find the missing token.

DevToolsTraffic InterceptionReverse EngineeringTLS DebuggingMobile APIs

// 02 — definitions

See the
wire.

The essential diagnostic layer between your scraper and the target server, revealing exactly what is being sent before it hits the network.

Ask a DataFlirt engineer →

TL;DR

Mitmproxy acts as a man-in-the-middle, terminating the client's TLS connection and establishing its own with the server. This allows engineers to read encrypted HTTP/2 and WebSocket traffic in plain text, making it indispensable for API discovery and debugging anti-bot blocks.

01Definition & structure

Mitmproxy is a suite of tools for intercepting and modifying HTTP/HTTPS traffic. It consists of three core components: mitmproxy (an interactive command-line interface), mitmweb (a web-based GUI), and mitmdump (a command-line version designed for scripting and automated traffic capture). By acting as a proxy server, it allows engineers to inspect headers, payloads, and timing data that are otherwise hidden by TLS encryption.

02How it works in practice

To intercept HTTPS traffic, you start mitmproxy and configure your client (a browser, a mobile device, or a scraper) to route its traffic through the proxy's IP and port. You then install mitmproxy's generated Certificate Authority (CA) on the client device. Once trusted, mitmproxy decrypts the outbound request, logs it, and re-encrypts it before sending it to the destination server. The server's response is similarly decrypted, logged, and passed back to the client.

03Mobile API reverse engineering

Mobile apps often use undocumented, highly efficient APIs that are much easier to scrape than their web counterparts. Mitmproxy is the standard tool for discovering these endpoints. By routing a physical phone's traffic through mitmproxy, engineers can capture the exact JSON or Protobuf payloads the app uses, identify required authentication headers, and replicate the requests in a stateless scraper.

04How DataFlirt uses it

We use mitmproxy extensively in our lab environment to build and debug pipelines. When a target deploys a new anti-bot challenge, we route a real device through mitmdump to capture the exact telemetry payload the native client generates. We also use mitmproxy's Python scripting interface to inject mock responses into mobile apps, testing how they handle rate limits or malformed data without needing to trigger those states on the live server.

05Did you know?

Mitmproxy isn't just for passive observation. Its Python API allows you to write scripts that modify requests and responses in real-time. You can write a 10-line script to strip out specific tracking headers before they leave your network, or automatically rewrite 403 Forbidden responses to 200 OKs to see how a client-side application reacts to a bypassed block.

// 03 — interception metrics

The cost of
inspection.

Running an intercepting proxy adds latency and alters the TLS signature. DataFlirt engineers account for these shifts when moving from local mitmproxy debugging to production fleet deployment.

Interception Latency = L_total = L_client-proxy + L_proxy-server + T_processing

Mitmproxy adds ~15-40ms of processing overhead per request. Network diagnostic baseline

TLS Signature Shift = JA3_proxy ≠ JA3_client

Mitmproxy negotiates its own TLS with the server, masking the original client's fingerprint. TLS handshake mechanics

DataFlirt Debug Ratio = 1 mitmdump capture : 10,000 production requests

We use traffic captures to generate baseline headers for high-volume pipelines. Internal engineering SLO

// 04 — mitmdump trace

Intercepting a
mobile API call.

A live mitmdump trace capturing an undocumented GraphQL endpoint from a retail app, revealing the hidden authorization headers required for scraping.

mitmdumpHTTP/2GraphQL

edge.dataflirt.io — live

CAPTURED

// start transparent interception
$ mitmdump -s extract_tokens.py --mode transparent
Proxy server listening at http://*:8080
Client connected: 192.168.1.45 (Android Device)

// intercepted request
>> POST https://api.target-retail.com/graphql
x-app-version: "4.12.0"
authorization: "Bearer eyJhbG..." // Token captured
x-device-fingerprint: "a8f93b2c..." // Anti-bot payload

// server response
<< 200 OK 1.2k
content-type: "application/json"

// script execution
Script extract_tokens.py: Saved auth token to local store.

// 05 — diagnostic targets

What we look for
in the trace.

When a scraper fails but a real browser succeeds, the discrepancy is hidden in the traffic. These are the primary signals DataFlirt engineers hunt for using mitmproxy.

DEBUG SESSIONS · · · 1,200+ / month

PRIMARY USE · · · · Mobile APIs

UPDATED · · · · · · 2026-05-19

01

Missing dynamic headers

94% of blocks · X-CSRF, HMAC signatures, device tokens

02

Cookie sequence mismatches

82% of blocks · Missing intermediate set-cookie directives

03

API endpoint discovery

75% of cases · Finding undocumented mobile/web endpoints

04

Payload encoding analysis

60% of cases · Protobuf, GraphQL, custom binary formats

05

WebSocket frame inspection

45% of cases · Real-time data feeds and heartbeat pings

// 06 — pipeline generation

From manual trace,

to automated pipeline.

DataFlirt doesn't just use mitmproxy for manual debugging. We use its Python API (mitmdump) to record baseline traffic from real devices. These captures are fed into our pipeline generator, which automatically extracts the required header sequences, identifies dynamic tokens, and compiles a production-ready fetch configuration that perfectly mimics the native client.

mitmdump.capture.log

Automated analysis of a captured mobile app session.

session.id cap-mob-882

tls.intercepted true

endpoints.discovered 14 unique paths

auth.type Bearer + HMAC signature

anti_bot.detected PerimeterX Mobile SDK

pipeline.generated fetch_config_v2.json

Stay ahead of the pipeline

Data engineering
intel, weekly.

Anti-bot shifts, scraping infrastructure updates, dataset delivery patterns, and business outcomes from our pipelines. Short, technical, no fluff.

// 07 — FAQ

Common
questions.

Common questions about traffic interception, SSL pinning, legal boundaries, and how DataFlirt uses mitmproxy in production.

Ask us directly →

What is the difference between mitmproxy, Charles, and Fiddler? +

Mitmproxy is CLI and Python-first, open-source, and highly scriptable, making it ideal for engineers who want to automate traffic modification. Charles and Fiddler are GUI-heavy and often preferred by QA teams for manual inspection. We default to mitmproxy because its Python API integrates directly into our automated testing pipelines.

How do you intercept HTTPS traffic if it's encrypted? +

Mitmproxy generates a custom Certificate Authority (CA). You install this CA on your client device (browser or phone). The client trusts mitmproxy, allowing mitmproxy to decrypt the traffic, inspect it, and then re-encrypt it to send to the target server. It is a deliberate, controlled man-in-the-middle attack.

What happens if the target app uses SSL pinning? +

SSL pinning hardcodes the expected server certificate inside the app, causing it to reject mitmproxy's CA and drop the connection. To bypass this, you must modify the app—using dynamic instrumentation tools like Frida or Objection on Android/iOS—to hook and disable the pinning checks before intercepting the traffic.

Does mitmproxy alter the TLS fingerprint (JA3) of my scraper? +

Yes. Because mitmproxy terminates the connection, the server sees mitmproxy's TLS fingerprint, not your scraper's or browser's. This is why you should never use mitmproxy in a production scraping flow where anti-bot systems are active—it will flag you immediately. It is strictly a diagnostic tool.

Is it legal to intercept traffic from a third-party app? +

Intercepting traffic on a device you own, for interoperability or research purposes, is generally protected under fair use and reverse-engineering exemptions in the US and EU. However, using the discovered APIs to bypass access controls or harvest protected data carries distinct legal risks. Always consult counsel for your specific use case.

How does DataFlirt scale API discovery? +

We don't run mitmproxy in our production fetch fleet. We use it in our lab environment to capture baseline traffic from real iOS and Android devices. Our automated tooling then translates those mitmdump captures into stateless Go and Rust fetchers that run at scale, perfectly mimicking the native client's header order and payload structure.

$ dataflirt scope --new-project --target=mitmproxy READY

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue dump or a continuous feed across millions of records — we scope, build, and operate the pipeline.

Start a pipeline → View pricing

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h