快捷導(dǎo)航

python中httpx庫(kù)的詳細(xì)使用方法及案例詳解

更新時(shí)間：2025年02月27日 09:38:54 作者：數(shù)據(jù)知道

httpx 是一個(gè)現(xiàn)代化的 Python HTTP 客戶端庫(kù),支持同步和異步請(qǐng)求,功能強(qiáng)大且易于使用,它比 requests 更高效,支持 HTTP/2 和異步操作,以下是 httpx 的詳細(xì)使用方法,感興趣的小伙伴跟著小編一起來看看吧

1. 安裝 httpx

首先，確保已經(jīng)安裝了 httpx?？梢酝ㄟ^以下命令安裝：pip install httpx

如果需要支持 HTTP/2，可以安裝額外依賴：pip install httpx[http2]

2. 同步請(qǐng)求

發(fā)送 GET 請(qǐng)求

import httpx

# 發(fā)送 GET 請(qǐng)求
response = httpx.get('https://httpbin.org/get')
print(response.status_code)  # 狀態(tài)碼
print(response.text)         # 響應(yīng)內(nèi)容

發(fā)送 POST 請(qǐng)求

# 發(fā)送 POST 請(qǐng)求
data = {'key': 'value'}
response = httpx.post('https://httpbin.org/post', json=data)
print(response.json())  # 解析 JSON 響應(yīng)

設(shè)置請(qǐng)求頭

headers = {'User-Agent': 'my-app/1.0.0'}
response = httpx.get('https://httpbin.org/headers', headers=headers)
print(response.json())

設(shè)置查詢參數(shù)

params = {'key1': 'value1', 'key2': 'value2'}
response = httpx.get('https://httpbin.org/get', params=params)
print(response.json())

處理超時(shí)

try:
    response = httpx.get('https://httpbin.org/delay/5', timeout=2.0)
except httpx.TimeoutException:
    print("請(qǐng)求超時(shí)")

3. 異步請(qǐng)求

httpx 支持異步操作，適合高性能場(chǎng)景。

發(fā)送異步 GET 請(qǐng)求

import httpx
import asyncio

async def fetch(url):
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        print(response.text)

asyncio.run(fetch('https://httpbin.org/get'))

發(fā)送異步 POST 請(qǐng)求

async def post_data(url, data):
    async with httpx.AsyncClient() as client:
        response = await client.post(url, json=data)
        print(response.json())

asyncio.run(post_data('https://httpbin.org/post', {'key': 'value'}))

并發(fā)請(qǐng)求

async def fetch_multiple(urls):
    async with httpx.AsyncClient() as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        for response in responses:
            print(response.text)

urls = ['https://httpbin.org/get', 'https://httpbin.org/ip']
asyncio.run(fetch_multiple(urls))

4. 高級(jí)功能

使用 HTTP/2

# 啟用 HTTP/2
client = httpx.Client(http2=True)
response = client.get('https://httpbin.org/get')
print(response.http_version)  # 輸出協(xié)議版本

文件上傳

files = {'file': open('example.txt', 'rb')}
response = httpx.post('https://httpbin.org/post', files=files)
print(response.json())

流式請(qǐng)求

# 流式上傳
def generate_data():
    yield b"part1"
    yield b"part2"

response = httpx.post('https://httpbin.org/post', data=generate_data())
print(response.json())

流式響應(yīng)

# 流式下載
with httpx.stream('GET', 'https://httpbin.org/stream/10') as response:
    for chunk in response.iter_bytes():
        print(chunk)

5. 錯(cuò)誤處理

httpx 提供了多種異常類，方便處理錯(cuò)誤。

處理網(wǎng)絡(luò)錯(cuò)誤

try:
    response = httpx.get('https://nonexistent-domain.com')
except httpx.NetworkError:
    print("網(wǎng)絡(luò)錯(cuò)誤")

處理 HTTP 錯(cuò)誤狀態(tài)碼

response = httpx.get('https://httpbin.org/status/404')
if response.status_code == 404:
    print("頁(yè)面未找到")

6. 配置客戶端

可以通過 httpx.Client 或 httpx.AsyncClient 配置全局設(shè)置。

設(shè)置超時(shí)

client = httpx.Client(timeout=10.0)
response = client.get('https://httpbin.org/get')
print(response.text)

設(shè)置代理

proxies = {
    "http://": "http://proxy.example.com:8080",
    "https://": "http://proxy.example.com:8080",
}
client = httpx.Client(proxies=proxies)
response = client.get('https://httpbin.org/get')
print(response.text)

設(shè)置基礎(chǔ) URL

client = httpx.Client(base_url='https://httpbin.org')
response = client.get('/get')
print(response.text)

7. 結(jié)合 Beautiful Soup 使用

httpx 可以與 Beautiful Soup 結(jié)合使用，抓取并解析網(wǎng)頁(yè)。

import httpx
from bs4 import BeautifulSoup

# 抓取網(wǎng)頁(yè)
response = httpx.get('https://example.com')
html = response.text

# 解析網(wǎng)頁(yè)
soup = BeautifulSoup(html, 'lxml')
title = soup.find('title').text
print("網(wǎng)頁(yè)標(biāo)題:", title)

8. 示例：抓取并解析網(wǎng)頁(yè)

以下是一個(gè)完整的示例，展示如何使用 httpx 抓取并解析網(wǎng)頁(yè)數(shù)據(jù)：

import httpx
from bs4 import BeautifulSoup

# 抓取網(wǎng)頁(yè)
url = 'https://example.com'
response = httpx.get(url)
html = response.text

# 解析網(wǎng)頁(yè)
soup = BeautifulSoup(html, 'lxml')

# 提取標(biāo)題
title = soup.find('title').text
print("網(wǎng)頁(yè)標(biāo)題:", title)

# 提取所有鏈接
links = soup.find_all('a', href=True)
for link in links:
    href = link['href']
    text = link.text
    print(f"鏈接文本: {text}, 鏈接地址: {href}")