#Mitmproxy - Python packet capture artifact

Do you often encounter these pain points when doing App crawlers and Web interface debugging?

• Fiddler: Focus on Windows, with average Mac/Linux experience and limited scalability;
• Charles: comprehensive functions but expensive;
• tcpdump/Wireshark: Pure command line operation, manual filtering of HTTP/HTTPS traffic is cumbersome and unintuitive.

Mitmproxy perfectly fills these gaps - it is a free open source, cross-platform, interactive HTTP/HTTPS proxy written in Python that supports deep customization of Python scripts. Whether it is to temporarily capture interfaces or build automated crawling/traffic modification pipelines, it is the first choice artifact for developers and crawlers.

This article will take you from installation, basic mode, to creating an App directed traffic interception script, and finally share some best practices summarized in actual combat, and master the core usage of Mitmproxy in one go.

#1. Installation and three operating modes

Mitmproxy core provides three independent tools, covering different scenarios from visual operations to automated scripts:

#1.1 Installation

Just onepipJust use the command. It is recommended to use Python 3.8 and above:

pip install mitmproxy

#1.2 Mode List

# 模式1：Web可视化模式（新手友好首选）
# 启动后自动打开浏览器访问 http://127.0.0.1:8081 进行操作
mitmweb -p 8081

# 模式2：纯命令行交互模式（适合终端控/临时调试）
# 终端内带彩色UI，支持类Vim快捷键浏览流量
mitmproxy -p 8082

# 模式3：自动化脚本模式（核心功能，本文重点）
# 直接运行自定义Python脚本，批量处理流量
mitmdump -s your_script.py -p 8080

Each of the three modes has its own focus:mitmwebSuitable for novices to quickly view requests;mitmproxyLet veterans operate efficiently in the terminal;mitmdumpIt is the cornerstone of automated traffic processing. Next, we will delve into the third mode through script practice.

#2. Practical script: App directional traffic interception and analysis

The following script is a core tool tailored for App crawlers. It has the following capabilities:

• Automatically identify inclusionsapi、mobile、app、sdk、analyticsRequests for keywords such as
• Completely extract request and response data (including JSON, Headers, Query parameters);
• Automatically save results to local JSON file;
• Count the number of API calls and exception status codes.

Save the code asapp_traffic_analyzer.pyReady to use.

from mitmproxy import http, ctx
import json
import re
import time
import os
from datetime import datetime
from typing import Dict, Any

class AppTrafficAnalyzer:
    """App定向流量拦截与分析器"""

    def __init__(self):
        self.intercepted_data = []
        # 可根据目标应用/网站的域名特征自由添加、删减
        self.target_features = {
            'domains': [r'.*api\..*', r'.*mobile\..*', r'.*app\..*', r'.*sdk\..*'],
            'ua_keywords': ['mobile', 'android', 'ios', 'app'],
            'path_keywords': ['/api/', '/mobile/', '/sdk/', '/v1/', '/v2/']
        }
        self.output_dir = 'mitmproxy_analysis'
        os.makedirs(self.output_dir, exist_ok=True)

    def request(self, flow: http.HTTPFlow) -> None:
        """请求拦截入口"""
        if not self._is_target_flow(flow):
            return

        req_info = {
            'type': 'request',
            'timestamp': datetime.fromtimestamp(flow.request.timestamp_start).isoformat(),
            'method': flow.request.method,
            'url': flow.request.pretty_url,
            'host': flow.request.host,
            'path': flow.request.path,
            'query': dict(flow.request.query) if flow.request.query else {},
            'headers': dict(flow.request.headers),
            'text': flow.request.text if flow.request.content else ''
        }
        self.intercepted_data.append(req_info)
        ctx.log.info(f"🎯 捕获目标请求: {flow.request.pretty_url}")

    def response(self, flow: http.HTTPFlow) -> None:
        """响应拦截入口"""
        if not self._is_target_flow(flow):
            return

        resp_info = {
            'type': 'response',
            'timestamp': datetime.fromtimestamp(flow.response.timestamp_end).isoformat(),
            'status_code': flow.response.status_code,
            'url': flow.response.url,
            'headers': dict(flow.response.headers),
            'text': flow.response.text if flow.response.content else ''
        }
        self.intercepted_data.append(resp_info)
        ctx.log.info(f"📡 捕获目标响应: {flow.response.url} - {flow.response.status_code}")

    def _is_target_flow(self, flow: http.HTTPFlow) -> bool:
        """判断是否为目标流量"""
        # 检查域名
        for pattern in self.target_features['domains']:
            if re.search(pattern, flow.request.host, re.IGNORECASE):
                return True

        # 检查User-Agent
        ua = flow.request.headers.get('User-Agent', '').lower()
        if any(k in ua for k in self.target_features['ua_keywords']):
            return True

        # 检查路径
        path = flow.request.path.lower()
        if any(k in path for k in self.target_features['path_keywords']):
            return True

        return False

    def done(self) -> None:
        """mitmdump停止时自动执行：保存所有拦截数据"""
        output_file = os.path.join(self.output_dir, f"traffic_{int(time.time())}.json")
        with open(output_file, 'w', encoding='utf-8') as f:
            json.dump(self.intercepted_data, f, ensure_ascii=False, indent=2)

        ctx.log.info(f"✅ 分析完成，结果已保存到: {output_file}")

# 必须添加到addons列表，供mitmproxy识别
addons = [AppTrafficAnalyzer()]

#3. Complete usage process

From launching the agent to seeing results, it only takes three steps.

#3.1 Start the script agent and configure the device

#(1) Start mitmdump

Execute the following command in the terminal to load the script you just wrote and listen to port 8080:

mitmdump -s app_traffic_analyzer.py -p 8080

Keep in mind the device IP the agent is running on:

• If you capture browser traffic locally, the IP is127.0.0.1；
• If you want to grab the mobile app, you need to fill in the IP of the computer in the LAN (for Windowsipconfig, for Mac/LinuxifconfigCheck).

#(2) Configure proxy for browser or mobile phone

• Browser: Go to browser settings → Network → Proxy, select manual configuration of HTTP/HTTPS proxy, fill in the IP and port8080。
• cell phone:
  • iOS：设置 → 无线局域网→ Click on the right side of the currently connected Wi-Fiⓘ → 配置代理 → 手动, fill in the IP and port.
  • Android：设置 → WLAN→ Long press the current Wi-Fi →修改网络 → 高级选项 → 代理 → 手动, fill in the IP and port.

#3.2 Install and trust the Mitmproxy CA certificate (required to capture HTTPS)

After configuring the proxy, access it in a browser or mobile browser mitm.it, follow the prompts to download the certificate of the corresponding platform and trust it.

• Special Note for iOS:

1. After installing the description file, go to设置 → 通用 → VPN 与设备管理Trust this profile.
2. Again设置 → 通用 → 关于本机 → 证书信任设置, Full Trust Mitmproxy CA certificate.

• Special Notes for Android 7.0+: The system does not trust user-installed certificates by default. There are two mainstream solutions:

1. Non-Root friendly: Use virtual environments such as VirtualXposed and Taichi to place the target App into the virtual environment and only trust the user certificate of the virtual environment.
2. Root scheme: Put the downloaded certificate file into/system/etc/security/cacerts/Directory, permissions are set to644。

#3.3 Start packet capture and view the results

After the configuration is completed, operate the target App or web page normally.mitmdumpThe terminal will print out the captured request and response logs in real time.

After the packet capture is completed, pressCtrl+CStop the script and go tomitmproxy_analysisDirectory, you can see the JSON file named by timestamp, which contains all hit request and response data for further analysis.

#4. Advanced and best practices

1. Precisely customize target features in scripttarget_featuresCovers common App interface features, but in actual scenarios it is recommended to fine-tune based on specific goals. You can firstmitmwebQuickly capture a wave of traffic, lock the fixed domain name, UA characteristics or path keywords of the target App, and then fill them into the script to avoid interference.

2. Function Extension you canrequestorresponseAdd more logic to the method, for example:

• Dynamically modify request headers or parameters to test interface boundaries;
• Automatically extract key fields in JSON (such as Token, product list, user information);
• Write data into databases such as MongoDB and MySQL in real time to build a continuously accumulated analysis library.

3. Guidelines for mobile packet capture and pitfall avoidance

• The iOS device must be in the same LAN as the agent machine, and certificate trust has been completed;
• If SSL Pinning (certificate binding) is turned on for Android applications, regular proxies will not be able to capture HTTPS requests. At this point, you can try to use Frida to unpack and Hook the SSL Pinning verification function, or use tools such as VirtualXposed to bypass it.

Mastering the above methods, Mitmproxy can become the most flexible network analysis tool in your hand. Whether it is temporary debugging interfaces or building automated crawler pipelines, it can provide powerful and free network traffic analysis capabilities. Open the terminal and use a Python script to unlock your exclusive packet capture experience!