#Python Requests library usage tutorial: Say goodbye to the cumbersomeness of urllib

Are you still using the Python built-inurllibAre you having headaches with complex coding splicing, certificate configuration, and multi-session management? Today we will talk about Requests, the most popular HTTP client library in the Python ecosystem. It uses a minimalist API to encapsulate almost all HTTP functions needed for daily development. The threshold for getting started is extremely low, but the efficiency is very high.

#1. Introduction

The official slogan of Requests is "HTTP for Humans (HTTP library designed for humans)", which is no exaggeration at all:

• Built-in common functions such as automatic encoding and decoding, JSON parsing, status code checking, etc.
• Supports connection retention pooling, cookie persistence, and file upload/download in chunks
• Handling exceptions, setting timeouts, proxies, and authentication all require just one line of parameters

Whether you are calling external APIs, writing crawlers, or doing interface testing, Requests can help you get twice the result with half the effort.

#2. Installation

Just one command to do it (supports Python 3.7+). It is recommended to install in a virtual environment and keep project dependencies clean:

pip install requests

After the installation is complete, you can import it directly in the code and use it:

import requests

#3. Basic usage: Get started in one minute

Requests supports all standard HTTP methods. Let’s start with the most commonly used ones.GETandPOSTGet started, and then take a quick look at other methods.

#Send GET request

callrequests.get()And pass in the target URL, the returnedResponseThe object contains all response information:

import requests

# 发送一个简单的 GET 请求
response = requests.get('https://httpbin.org/get')

# 打印响应文本（自动处理编码，无需手动 decode）
print(response.text)

#Send POST request

When submitting a form or simple data, use it directlydataParameters, Requests will be set automaticallyContent-Type：

# 准备要提交的表单数据
form_data = {'username': 'python_user', 'email': 'user@example.com'}

# 发送 POST 请求
response = requests.post('https://httpbin.org/post', data=form_data)

# 直接解析 JSON 响应（httpbin 会原样返回你发送的请求详情）
print(response.json())

#Other standard HTTP methods

PUT、DELETE、HEAD、OPTIONSThe other methods are equally simple, and their usage is the same asGET / POSTBe consistent:

requests.put('https://httpbin.org/put', data={'key': 'update_value'})
requests.delete('https://httpbin.org/delete')
requests.head('https://httpbin.org/get')        # 只获取响应头
requests.options('https://httpbin.org/get')     # 查看服务器支持的方法

#4. Request parameters: customize your request as you like

In real projects, we often need to splice URL parameters, customize request headers, or send JSON data. Requests uses a few keyword parameters to make these operations extremely refreshing.

#Pass URL parameters

Put the parameters into a dictionary and pass it toparams, Requests will automatically complete URL encoding and splicing:

query_params = {'page': 1, 'limit': 20, 'sort': 'newest'}

response = requests.get('https://httpbin.org/get', params=query_params)

# 查看拼接后的完整 URL
print(response.url)  # 输出：https://httpbin.org/get?page=1&limit=20&sort=newest

💡 Tips: If the parameter value is empty orNone, Requests will ignore this parameter and will not carry it in the URL.

#Custom request header

To simulate browser access, add API Token, set Accept type and other scenarios, just pass in a dictionary toheadersparameter:

custom_headers = {
    'User-Agent': 'My-Python-App/1.0.0',          # 自定义客户端标识
    'Authorization': 'Bearer YOUR_API_TOKEN',     # 常见的 Bearer Token 认证
}

response = requests.get('https://api.example.com/data', headers=custom_headers)

#Send JSON data

More and more APIs require the use of request bodies in JSON format, and Requests specifically providesjsonParameters, no need to do it manuallyjson.dumps()and settingsContent-Type：

# 注意：不需要额外导入 json 库
json_data = {
    'title': 'Python Requests',
    'content': 'This is a tutorial'
}

response = requests.post('https://httpbin.org/post', json=json_data)

#5. Response processing: quickly extract useful information

ResponseThe object is very well encapsulated. According to different needs, you can choose the most appropriate attributes to obtain the response content.

#Get response content

Example:

response = requests.get('https://httpbin.org/get')

# 文本
print("=== 响应文本（前100字符） ===")
print(response.text[:100])

# 二进制（例如保存一张图片）
img_response = requests.get('https://httpbin.org/image/png')
with open('test.png', 'wb') as f:
    f.write(img_response.content)

# JSON
print("\n=== JSON 响应中的 origin ===")
print(response.json()['origin'])

⚠️ If the response is not valid JSON,response.json()will throwjson.JSONDecodeError, it is recommended to placetry-exceptused in blocks.

#Check status code

• response.status_code: Get the digital status code directly
• response.raise_for_status(): Automatically determine the status code and throw it when it is not 2xxHTTPErrorabnormal

response = requests.get('https://httpbin.org/status/404')  # 模拟 404 错误

print(f"状态码：{response.status_code}")

# 结合 try-except 让错误处理更安全
try:
    response.raise_for_status()
except requests.exceptions.HTTPError as e:
    print(f"HTTP 错误：{e}")

#Get response headers

response.headersIt is a dictionary-like object, and the keys are not case-sensitive, so it is very convenient to use:

response = requests.get('https://httpbin.org/get')

print("Content-Type:", response.headers['content-type'])
print("Server:", response.headers.get('Server'))   # 推荐使用 get 方法避免 KeyError

#6. Advanced functions: easily handle complex scenarios

#Session object (Session): reused connections and persistent cookies

When you need to send multiple requests to the same website,SessionThe object can reuse TCP connections, significantly improving performance; at the same time, it will also automatically manage and carry Cookies, eliminating the trouble of manually handling Cookies:

# 创建一个会话
session = requests.Session()

# 设置会话级别的请求头（该会话内所有请求都会自动携带）
session.headers.update({'User-Agent': 'My-Session-App/1.0'})

# 第一次请求：模拟设置 Cookie
login_response = session.post('https://httpbin.org/cookies/set', params={'session_id': '123456'})
print("第一次请求后设置的 Cookie：", login_response.json())

# 第二次请求：自动携带上次的 Cookie
data_response = session.get('https://httpbin.org/cookies')
print("第二次请求时服务器收到的 Cookie：", data_response.json())

# 使用完毕后记得关闭会话
session.close()

✅ You can also usewith requests.Session() as session:syntax to avoid forgetting to close.

#SSL Verification

By default Requests will verify the SSL certificate to ensure a secure connection. If you encounter a self-signed certificate or development environment, you can also flexibly control the verification behavior:

# 临时禁用 SSL 验证（仅用于测试，会抛出一个安全警告）
requests.get('https://self-signed.badssl.com', verify=False)

# 生产环境中可以指定自己的 CA 证书
requests.get('https://internal-api.example.com', verify='/path/to/your/ca-cert.pem')

#Timeout settings

**Be sure to set a timeout! ** Otherwise, the program may wait indefinitely due to network fluctuations. Requests supports two timeout methods:

• tuple(connect_timeout, read_timeout): Set connection timeout and read timeout respectively
• single number: total timeout (connect + read)

# 连接超时 3 秒，读取超时 27 秒
requests.get('https://httpbin.org/delay/5', timeout=(3, 27))

# 总超时 5 秒
try:
    requests.get('https://httpbin.org/delay/10', timeout=5)
except requests.exceptions.Timeout as e:
    print("请求超时：", e)

#7. Error handling: gracefully deal with various accidents

Requests encapsulates common network and HTTP errors into exception classes, using a clear exception tree structure. It is strongly recommended to catch specific exception types rather than just the parent class of all exceptions:

from requests.exceptions import (
    RequestException,   # 所有 Requests 异常的父类，可做兜底
    HTTPError,
    Timeout,
    ConnectionError,
)

try:
    response = requests.get('https://httpbin.org/status/500', timeout=5)
    response.raise_for_status()   # 先检查 HTTP 状态码
except HTTPError as e:
    print(f"HTTP 错误（状态码 >=400）：{e}")
except Timeout as e:
    print(f"请求超时：{e}")
except ConnectionError as e:
    print(f"连接错误（如网络不通、DNS 解析失败）：{e}")
except RequestException as e:
    print(f"其他 Requests 错误：{e}")

This way your program won't crash easily due to a network jitter.

#8. Best Practice: Write Robust and Elegant Code

1. Timeout must be set - to prevent the program from hanging indefinitely, especially in a production environment
2. Priority to use Session - can improve performance and automatically manage cookies in multi-request scenarios
3. Develop and useraise_for_status()Habit - automatically detect 4xx/5xx and use it with exception-handling
4. Enable streaming downloads when processing large files - avoid loading the entire file into memory at once (stream=True）
5. Catch specific exceptions - Don’t just useRequestExceptionOne pot for easy debugging and targeted processing
6. Comply with the website’srobots.txt——Crawler development must be friendly
7. Sensitive information is stored in environment variables - API Tokens, passwords, etc. must not be hard-coded. Useos.getenv()or.envFile management

#9. Summary

Requests is the most recommended HTTP client library in the Python ecosystem. It uses an intuitive API to solve the problem.urlliball pain points. Whether it is a simple interface call or complex communication between crawlers and microservices, Requests allows you to complete the most work with the least code.

If you also want to explore more advanced features (file uploads, streaming requests, proxy settings, OAuth authentication, etc.), please visit Requests 官方文档, which has rich examples and detailed instructions. Now, just usepip install requestsStart your journey to efficient HTTP!