### Install from source
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Steps to build and install the library from the GitHub repository.
```bash
git clone https://github.com/lexiforest/curl_cffi/
cd curl_cffi
make preprocess
pip install .
```
--------------------------------
### Setup local development environment on macOS
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/dev.md
Configures the necessary directory structure, installs system dependencies via Homebrew, and installs the project in editable mode with development and test dependencies.
```shell
sudo mkdir /Users/runner
sudo chmod 777 /Users/runner
brew install libidn2 zstd
pip install -e .[test]
pip install -e .[dev]
```
--------------------------------
### Install beta releases
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Command to install pre-release versions of the library.
```bash
pip install curl_cffi --upgrade --pre
```
--------------------------------
### Install curl-cffi with CLI extras
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Install the optional 'cli' extra for enhanced features like syntax highlighting and download progress bars.
```bash
pip install 'curl_cffi[cli]'
```
--------------------------------
### Basic GET Request
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Perform a simple GET request to a specified URL.
```bash
curl-cffi get https://httpbin.org/get
```
--------------------------------
### Perform GET requests with impersonation
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Demonstrates basic GET requests using the impersonate parameter to mimic specific browser TLS fingerprints and proxy configurations.
```python
import curl_cffi
url = "https://tls.browserleaks.com/json"
# Notice the impersonate parameter
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome110")
print(r.json())
# output: {..., "ja3n_hash": "aa56c057ad164ec4fdcb7a5a283be9fc", ...}
# the js3n fingerprint should be the same as target browser
# To keep using the latest browser version as `curl_cffi` updates,
# simply set impersonate="chrome" without specifying a version.
# Other similar values are: "safari" and "safari_ios"
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome")
# http/socks proxies are supported
proxies = {"https": "http://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome110", proxies=proxies)
proxies = {"https": "socks://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome110", proxies=proxies)
```
--------------------------------
### Install curl-cffi with pip
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Install the curl_cffi package using pip. This includes the CLI functionality.
```bash
pip install curl_cffi
```
--------------------------------
### Perform Asynchronous HTTP Requests
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Demonstrates basic and concurrent GET requests using AsyncSession.
```python
from curl_cffi import AsyncSession
async with AsyncSession() as s:
r = await s.get("https://example.com")
```
```python
import asyncio
from curl_cffi import AsyncSession
urls = [
"https://google.com/",
"https://facebook.com/",
"https://twitter.com/",
]
async with AsyncSession() as s:
tasks = []
for url in urls:
task = s.get(url)
tasks.append(task)
results = await asyncio.gather(*tasks)
```
--------------------------------
### GET Request with Custom Header
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Make a GET request and include a custom header in the request.
```bash
curl-cffi get https://httpbin.org/get X-My-Header:value
```
--------------------------------
### Implement Synchronous and Asynchronous WebSockets
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Provides examples for connecting to WebSockets using both blocking Session and non-blocking AsyncSession interfaces.
```python
from curl_cffi import Session, WebSocket
def on_message(ws: WebSocket, message):
print(message)
with Session() as session:
ws = session.ws_connect(
"wss://api.gemini.com/v1/marketdata/BTCUSD",
on_message=on_message,
)
ws.run_forever()
# asyncio
import asyncio
from curl_cffi import AsyncSession
async with AsyncSession() as session:
async with session.ws_connect("wss://echo.websocket.org") as ws:
await asyncio.gather(*[ws.send_str("Hello, World!") for _ in range(10)])
async for message in ws:
print(message)
```
--------------------------------
### Basic GET Request
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Perform a simple GET request to a URL. Impersonates Chrome by default.
```bash
curl-cffi get https://httpbin.org/get
```
--------------------------------
### Python Example for Proxy Credential Reuse Control
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/psk.md
A placeholder for a Python example demonstrating how to enable or manage the `proxy_credential_no_reuse` option in curl_cffi, which binds TLS session caches to proxy credentials to prevent issues with rotating proxies.
```python
# Python example to be added.
# We might enable this by default when proxies are used.
```
--------------------------------
### Use curl-cffi CLI
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Examples of using the command line interface to perform requests with impersonation.
```bash
curl-cffi get tls.browserleaks.com/json
# curl-cffi can be hard to type, use an alias if you want
alias imp=curl-cffi
imp get tls.browserleaks.com/json --impersonate chrome
```
--------------------------------
### Doctor Command Example Output
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/doctor.md
This is an example of the diagnostic information printed by the `doctor` command. It includes details about your Python environment, platform, curl_cffi version, and configuration.
```text
curl-cffi doctor
----------------
python: 3.10.20
executable: /usr/bin/python3
platform: macOS-15.4-arm64-arm-64bit
machine: arm64
curl_cffi: 0.15.0b4
libcurl: libcurl/8.15.0
api_root: https://api.impersonate.pro/v1
config_path: /Users/you/.config/impersonate/config.json
config_present: True
api_key_configured: True
fingerprint_path: /Users/you/.config/impersonate/fingerprints.json
fingerprint_present: True
fingerprint_count: 42
```
--------------------------------
### Manage sessions
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Example of using the Session object to persist cookies across requests.
```python
s = curl_cffi.Session()
# httpbin is a http test website, this endpoint makes the server set cookies
s.get("https://httpbin.org/cookies/set/foo/bar")
print(s.cookies)
# ]>
# retrieve cookies again to verify
r = s.get("https://httpbin.org/cookies")
print(r.json())
# {'cookies': {'foo': 'bar'}}
```
--------------------------------
### Perform requests with impersonation
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Examples of using the high-level API to perform requests with various browser impersonation and proxy configurations.
```python
import curl_cffi
# Notice the impersonate parameter
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome")
print(r.json())
# output: {..., "ja3n_hash": "aa56c057ad164ec4fdcb7a5a283be9fc", ...}
# the js3n fingerprint should be the same as target browser
# To keep using the latest browser version as `curl_cffi` updates,
# simply set impersonate="chrome" without specifying a version.
# Other similar values are: "safari" and "safari_ios"
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome")
# Use http/3 with impersonation
r = curl_cffi.get(
"https://fp.impersonate.pro/api/http3",
http_version="v3",
impersonate="chrome"
)
# To pin a specific version, use version numbers together.
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome124")
# To impersonate other than browsers, bring your own ja3/akamai strings
# See examples directory for details.
r = curl_cffi.get("https://tls.browserleaks.com/json", ja3=..., akamai=...)
# http/socks proxies are supported
proxies = {"https": "http://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome", proxies=proxies)
proxies = {"https": "socks://localhost:3128"}
r = curl_cffi.get("https://tls.browserleaks.com/json", impersonate="chrome", proxies=proxies)
```
--------------------------------
### Install curl-cffi via Homebrew
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Install curl-cffi on macOS using Homebrew. This makes the 'curl-cffi' command available in your shell.
```bash
brew install lexiforest/tap/curl-cffi
```
--------------------------------
### HTTP/3 Request
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Execute a GET request using the HTTP/3 protocol by specifying the --http3 flag.
```bash
curl-cffi get --http3 https://fp.impersonate.pro/api/http3
```
--------------------------------
### Custom Header
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Include a custom header in the GET request.
```bash
curl-cffi get https://httpbin.org/get X-My-Header:value
```
--------------------------------
### Impersonate Safari
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Make a GET request and explicitly impersonate Safari using the -i flag.
```bash
curl-cffi get -i safari tls.browserleaks.com/json
```
--------------------------------
### Verbose Output
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Execute a GET request with verbose output enabled (-v), showing request and response headers and body.
```bash
curl-cffi get -v https://httpbin.org/get
```
--------------------------------
### Akamai Fingerprint String Format Example
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/customize.md
The Akamai HTTP/2 fingerprint string encodes client-controlled protocol parameters: SETTINGS, WINDOW_UPDATE, PRIORITY, and Pseudo-Header Order.
```default
1:65536;4:131072;5:16384|12517377|3:0:0:201|m,p,a,s
```
--------------------------------
### Initialize and use AsyncSession
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Demonstrates the recommended context manager usage for AsyncSession, as well as manual instantiation.
```python
from curl_cffi.requests import AsyncSession
# recommended.
async with AsyncSession() as s:
r = await s.get("https://example.com")
s = AsyncSession() # it also works.
```
--------------------------------
### Connect and Communicate with AsyncWebSocket
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/websockets.md
Demonstrates establishing a connection, sending text, and iterating over incoming messages using the async context manager.
```python
import asyncio
from curl_cffi import AsyncSession
async def main():
async with AsyncSession() as session:
# Connect using the standard context manager syntax
async with session.ws_connect("wss://echo.websocket.org") as ws:
# Send a text message
await ws.send_str("Hello, World!")
# Receive a text message
msg = await ws.recv_str()
print(f"Received: {msg}")
# Iterate over messages
async for message in ws:
print(f"Stream: {message}")
asyncio.run(main())
```
--------------------------------
### Batch Execution with .http Files
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/vs_others.md
Shows how to define multiple HTTP requests in a human-readable .http file format, separated by '###', and execute them using curl-cffi's run command.
```text
### Get users
GET https://api.example.com/users
### Create user
POST https://api.example.com/users
Content-Type: application/json
{"name": "Alice"}
```
```bash
curl-cffi run requests.http
```
--------------------------------
### Handle URL parameters
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Shows how to pass query parameters as dictionaries, including support for lists.
```python
import curl_cffi
>>> params = {"foo": "bar"}
>>> r = curl_cffi.get("http://httpbin.org/get", params=params)
>>> r.url
'http://httpbin.org/get?foo=bar'
>>> params = {'key1': 'value1', 'key2': ['value2', 'value3']}
>>> import curl_cffi
>>> r = curl_cffi.get('https://httpbin.org/get', params=params)
>>> r.url
'https://httpbin.org/get?key1=value1&key2=value2&key2=value3'
```
--------------------------------
### Create Custom Cache Backend
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/advanced.md
Implement a custom storage backend by subclassing CacheBackend and defining the required storage methods.
```python
from datetime import timedelta
from curl_cffi import CacheBackend, Session
class DictCache(CacheBackend):
def __init__(self):
super().__init__(expires=timedelta(minutes=10))
self.data = {}
def _read_payload(self, key):
return self.data.get(key)
def _write_payload(self, key, payload):
self.data[key] = payload
def _delete_payload(self, key):
self.data.pop(key, None)
def clear(self):
self.data.clear()
with Session(cache=DictCache()) as s:
response = s.get("https://example.com/api")
```
--------------------------------
### curl_cffi.requests.WebSocket.__init__
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Initializes a new WebSocket instance with optional event callbacks and configuration settings.
```APIDOC
## WebSocket.__init__
### Description
Initializes the WebSocket client. Callbacks can be provided to handle connection events and incoming data.
### Parameters
- **autoclose** (bool) - Optional - Whether to close the WebSocket after receiving a close frame.
- **skip_utf8_validation** (bool) - Optional - Whether to skip UTF-8 validation for text frames.
- **debug** (bool) - Optional - Print extra curl debug info.
- **on_open** (callable) - Optional - Callback function: `def on_open(ws)`
- **on_close** (callable) - Optional - Callback function: `def on_close(ws, code, reason)`
- **on_data** (callable) - Optional - Callback function: `def on_data(ws, data, frame)`
- **on_message** (callable) - Optional - Callback function: `def on_message(ws, message)`
- **on_error** (callable) - Optional - Callback function: `def on_error(ws, exception)`
```
--------------------------------
### AsyncSession.__init__
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Initializes a new asynchronous session. Parameters provided here serve as defaults for all requests made within the session.
```APIDOC
## __init__(loop=None, async_curl=None, max_clients=10, **kwargs)
### Description
Initializes an async request session where cookies and connections are reused. Parameters set here can be overridden by individual request methods.
### Parameters
- **loop** (asyncio.AbstractEventLoop) - Optional - The event loop to use.
- **async_curl** (AsyncCurl) - Optional - The AsyncCurl object to use.
- **max_clients** (int) - Optional - Maximum number of curl handles to use in the session.
- **headers** (dict) - Optional - Default headers for the session.
- **cookies** (dict) - Optional - Default cookies for the session.
- **auth** (tuple) - Optional - HTTP basic auth (username, password).
- **proxies** (dict) - Optional - Proxy configuration dictionary.
- **proxy** (str) - Optional - Proxy URL.
- **impersonate** (str) - Optional - Browser version or fingerprint to impersonate.
```
--------------------------------
### Configure WebSocket Connections
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/websockets.md
Shows how to pass network parameters, authentication, and browser impersonation to the ws_connect method.
```python
async with AsyncSession() as session:
# Session cookies are automatically included
session.cookies.set("session_id", "xyz")
async with session.ws_connect(
"wss://api.example.com/v1/stream",
impersonate="chrome",
auth=("user", "pass"),
params={"stream_id": "123"},
timeout=10 # Connection timeout
) as ws:
...
```
--------------------------------
### Localhost Shortcut
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Use a shortcut for localhost requests, defaulting to HTTP.
```bash
curl-cffi get :8000/api/health
```
--------------------------------
### ws_connect
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Establishes a WebSocket connection.
```APIDOC
## ws_connect(url, ...)
### Description
Connects to a WebSocket server with various configuration options for authentication, proxies, and fingerprinting.
### Parameters
- **url** (str) - Required - The WebSocket URL to connect to.
- **autoclose** (bool) - Optional - Whether to automatically close the connection (default: True).
- **params** (dict|list|tuple) - Optional - Query parameters.
- **headers** (HeaderTypes) - Optional - Request headers.
- **cookies** (CookieTypes) - Optional - Cookies to send.
- **auth** (tuple) - Optional - Authentication credentials.
- **timeout** (float|tuple|NotSetType) - Optional - Connection timeout.
- **impersonate** (BrowserTypeLiteral|str|Fingerprint) - Optional - Browser impersonation settings.
### Returns
- **AsyncWebSocketContext** - The context object for the WebSocket connection.
```
--------------------------------
### List fingerprints via CLI
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Command to display the current list of available fingerprints.
```bash
curl-cffi list
```
--------------------------------
### Perform Asynchronous HTTP Requests
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Requires an AsyncSession context manager to execute asynchronous GET requests.
```python
# You must use a session for asyncio
async with curl_cffi.AsyncSession() as s:
r = await s.get("https://example.com")
```
--------------------------------
### Execute HTTP Requests from File
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
Use the `run` subcommand to execute requests defined in a .http or .har file. The format is auto-detected by the file extension.
```bash
curl-cffi run requests.http
```
```bash
curl-cffi run session.har
```
--------------------------------
### Impersonate Default Browser (Chrome)
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Perform a GET request, impersonating Chrome by default, to a site that provides browser information.
```bash
curl-cffi get tls.browserleaks.com/json
```
--------------------------------
### Session.ws_connect
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Establishes a WebSocket connection to the specified URL with optional callback handlers.
```APIDOC
## ws_connect(url, on_message=None, on_error=None, on_open=None, on_close=None, **kwargs)
### Description
Connects to a WebSocket URL. Note: This method is deprecated; use the WebSocket class directly.
### Parameters
- **url** (str) - Required - The WebSocket URL.
- **on_message** (Callable) - Optional - Callback for received messages.
- **on_error** (Callable) - Optional - Callback for errors.
- **on_open** (Callable) - Optional - Callback for connection open.
- **on_close** (Callable) - Optional - Callback for connection close.
```
--------------------------------
### HTTP Authentication with --auth
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/options.md
Provide HTTP authentication credentials using the `--auth` flag in the format `user:password`.
```bash
# Basic auth
curl-cffi get -a user:password https://httpbin.org/basic-auth/user/password
```
--------------------------------
### Run Requests from .http File
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Execute multiple requests defined in an .http or .rest file. Shares session by default.
```bash
curl-cffi run requests.http
```
--------------------------------
### Basic POST Request: curl vs. curl-cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/vs_others.md
Demonstrates the difference in syntax for making a POST request. curl-cffi uses a more readable key-value format instead of flags like -H and -d.
```bash
# curl
curl -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-d '{"name": "Alice", "age": 30}'
```
```bash
# curl-cffi
curl-cffi post https://api.example.com/users name=Alice age:=30
```
--------------------------------
### JA3 String Format Example
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/customize.md
A JA3 string is a comma-separated representation of TLS ClientHello fields: SSL/TLS Version, Cipher Suites, Extension IDs, Supported Groups, and EC Point Formats.
```default
771,4865-4866-4867-49195-49196,0-11-10-35-16-5,29-23-24,0
```
--------------------------------
### Randomize Browser Impersonation Version
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/faq.md
Demonstrates how to select a random browser version for impersonation from a predefined list. It is recommended to use realistic versions rather than generating custom fingerprints to avoid detection.
```python
import random
# Select a random browser version from a list of known valid versions
version = random.choice(["chrome119", "chrome120", "safari17", "firefox120"])
```
--------------------------------
### Manage WebSocket Connections
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Shows how to handle WebSocket messages using a callback function.
```python
from curl_cffi import WebSocket
def on_message(ws: WebSocket, message: str | bytes):
print(message)
ws = WebSocket(on_message=on_message)
ws.run_forever("wss://api.gemini.com/v1/marketdata/BTCUSD")
```
--------------------------------
### Proxy Configuration
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/options.md
Route requests through a proxy server by specifying the proxy URL with the `--proxy` option.
```bash
# Use a proxy
curl-cffi get --proxy http://proxy:8080 https://httpbin.org/get
```
--------------------------------
### Persist cookies using pickle
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cookies.md
Demonstrates how to save and load session cookies using the pickle module to maintain state across different session instances. This method is preferred over manual dictionary dumping due to the complex nature of cookie objects.
```python
import pickle
import curl_cffi
import os
def save_cookies(client):
with open("cookies.pk", "wb") as f:
pickle.dump(client.cookies.jar._cookies, f)
def load_cookies():
if not os.path.isfile("cookies.pk"): return None
with open("cookies.pk", "rb") as f:
return pickle.load(f)
client = curl_cffi.Session()
client.get("https://httpbin.org/cookies/set/foo/bar")
save_cookies(client)
client = curl_cffi.Session()
client.cookies.jar._cookies.update(load_cookies())
print(client.cookies.get("foo"))
```
--------------------------------
### Configure API Key with curl-cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/pro.md
Store your impersonate.pro API key in the local config file. Alternatively, use the IMPERSONATE_API_KEY environment variable.
```bash
curl-cffi config --api-key imp_xxxxxxxx
```
```bash
export IMPERSONATE_API_KEY=imp_xxxxxxxx
```
--------------------------------
### Select HTTP version for requests
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/advanced.md
Configure the HTTP version using the http_version parameter, supporting values like v1, v2, v3, and v3only.
```python
import curl_cffi
curl_cffi.get("https://cloudflare-quic.com", http_version="v3")
```
--------------------------------
### Localhost Shortcut
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/_index.md
Access a local service running on a specific port using the colon shortcut for localhost.
```bash
curl-cffi get :8000/api/health
```
--------------------------------
### Using Sessions for cookies and connections
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Sessions persist cookies and reuse connections across multiple requests.
```python
s = curl_cffi.Session()
# Cookies from server are stored in session
s.get("https://httpbin.org/cookies/set/foo/bar")
print(s.cookies)
# ]>
# Cookies are used in next request
r = s.get("https://httpbin.org/cookies")
print(r.json())
# {'cookies': {'foo': 'bar'}}
# It's preferred to use a context manager
with curl_cffi.Session() as s:
r = s.get("https://example.com")
```
```python
with curl_cffi.Session(headers={"User-Agent": "curl_cffi/0.11"}) as s:
r = s.get("https://example.com")
```
--------------------------------
### Download Response Body to File
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/options.md
Use the `--download` flag to save the response body to a file. The filename is derived from the `Content-Disposition` header or the URL path. Use `-o` to specify a custom output file path.
```bash
# Download a file
curl-cffi get --download https://example.com/file.zip
```
```bash
# Download to a specific path
curl-cffi get --download -o myfile.zip https://example.com/file.zip
```
--------------------------------
### Explicit vs. Implicit POST Method: HTTPie vs. curl-cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/vs_others.md
Highlights the difference in HTTP method specification. curl-cffi requires the method as an explicit subcommand, while HTTPie infers it from the presence of data.
```bash
# HTTPie -- implicit POST because of data fields
http example.com/api name=test
```
```bash
# curl-cffi -- method is always explicit
curl-cffi post example.com/api name=test
```
--------------------------------
### Replay HAR File Entries
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
The `run` command can replay all entries from a HAR (HTTP Archive) file. Ensure sensitive data is included in the HAR export if needed.
```bash
curl-cffi run captured.har
```
--------------------------------
### Headers Only Output
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Print only the response headers.
```bash
curl-cffi get --headers https://httpbin.org/get
```
--------------------------------
### Batch Execution with .har Files
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/vs_others.md
Demonstrates replaying HTTP traffic from a .har file, exported from browser developer tools, using curl-cffi's run command. This is useful for reproducing real-world requests.
```bash
curl-cffi run session.har
```
--------------------------------
### Control Output with Headers and Body Flags
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/options.md
Use `--headers` to print only response headers, `--body` to print only the response body, or `-v` for full verbose output including request and response headers and body.
```bash
# Headers only
curl-cffi get --headers https://httpbin.org/get
```
```bash
# Body only (no headers)
curl-cffi get --body https://httpbin.org/get
```
```bash
# Full verbose output
curl-cffi post -v https://httpbin.org/post name=test
```
--------------------------------
### Package with PyInstaller specifying paths and data
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/faq.md
When packaging with PyInstaller, specify Python paths and include necessary data files, such as DLLs and package data, using `--paths`, `--add-data`, and `--collect-data`.
```default
pyinstaller --noconfirm --onefile --console \
--paths "C:/Users/Administrator/AppData/Local/Programs/Python/Python39" \
--add-data "C:/Users/Administrator/AppData/Local/Programs/Python/Python39/Lib/site-packages/curl_cffi.libs/libcurl-cbb416caa1dd01638554eab3f38d682d.dll;. " \
--collect-data "curl_cffi" \
"C:/Users/Administrator/Desktop/test_script.py"
```
--------------------------------
### Use Session as a Context Manager
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
The Session class supports the context manager protocol for automatic resource cleanup.
```python
from curl_cffi.requests import Session
with Session() as s:
r = s.get("https://example.com")
```
--------------------------------
### Batch Execution Error Summary
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
When running multiple requests, execution continues on failure. A summary of failed requests is printed at the end.
```text
---
--- [1] GET https://api.example.com/users
--- [2] POST https://api.example.com/missing
--- [3] GET https://api.example.com/health
1 request(s) failed.
```
--------------------------------
### Configure curl_cffi for eventlet or gevent
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/community.md
Demonstrates how to enable non-blocking I/O by setting the thread parameter to eventlet or gevent within a curl_cffi Session. This allows the library to work within greenlet-based concurrency environments.
```python
from curl_cffi import requests
s = requests.Session(thread="eventlet")
s.get(url)
```
--------------------------------
### Configure API Key via Python
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/fingerprints.md
Set the API key programmatically in Python using FingerprintManager. This is useful for script-based configurations.
```python
from curl_cffi.fingerprints import FingerprintManager
FingerprintManager.set_api_key("imp_xxxxxxxx")
```
--------------------------------
### Configure API Key via CLI
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/fingerprints.md
Set your API key once using the curl-cffi config command. This configuration is persistent.
```sh
curl-cffi config --api-key imp_xxxxxxxx
```
--------------------------------
### curl_cffi.requests.Session.__init__
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Initializes a new Session object. Parameters provided here are used as defaults for all requests made within the session and can be overridden by individual request methods.
```APIDOC
## class curl_cffi.requests.Session
### Description
A request session where cookies and connections are reused. This object is thread-safe, though using a separate session per thread is recommended.
### Constructor
`__init__(curl=None, thread=None, use_thread_local_curl=True, **kwargs)`
### Parameters
- **curl** (Curl, optional) - Curl object to use. A fresh one is created if not provided or when accessed from another thread.
- **thread** (ThreadType, optional) - Thread engine to use (e.g., 'eventlet', 'gevent').
- **headers** (dict, optional) - Headers to use in the session.
- **cookies** (dict, optional) - Cookies to add to the session.
- **auth** (tuple, optional) - HTTP basic auth as (username, password).
- **proxies** (dict, optional) - Proxy configuration dictionary.
- **proxy** (str, optional) - Proxy URL string.
- **proxy_auth** (tuple, optional) - Proxy basic auth as (username, password).
- **base_url** (str, optional) - Base URL for relative paths.
- **params** (dict, optional) - Default query string parameters.
- **verify** (bool, optional) - Whether to verify HTTPS certificates.
- **timeout** (int, optional) - Seconds to wait before timing out.
- **trust_env** (bool, optional) - Use environment variables like http_proxy. Default: True.
- **allow_redirects** (bool/str/CurlFollow, optional) - Redirect handling policy.
- **max_redirects** (int, optional) - Maximum number of redirects. Default: 30.
- **retry** (int/RetryStrategy, optional) - Number of retries or strategy for failed requests.
- **impersonate** (str, optional) - Browser version or fingerprint to impersonate.
- **ja3** (str, optional) - JA3 string for impersonation.
- **akamai** (str, optional) - Akamai string for impersonation.
- **perk** (str, optional) - Perk string for impersonation.
- **extra_fp** (dict, optional) - Additional fingerprint options.
- **interface** (str, optional) - Interface name or local IP to bind to.
- **doh_url** (str, optional) - DNS-over-HTTPS server URL.
- **default_encoding** (str/callable, optional) - Encoding for response content. Default: 'utf-8'.
- **cert** (tuple, optional) - (cert, key) filenames for client certificates.
- **response_class** (type, optional) - Custom Response subtype.
- **raise_for_status** (bool, optional) - Automatically raise HTTPError for 4xx/5xx status codes.
```
--------------------------------
### Configure Queue Sizes for Backpressure
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/websockets.md
Adjust internal buffer sizes to manage high-throughput streams or specific latency requirements.
```python
# Increase queues for high-throughput streams (e.g., market data)
ws = await session.ws_connect(
url,
recv_queue_size=512,
send_queue_size=256
)
```
```python
# High throughput, fast moving streams (e.g., video)
ws = await session.ws_connect(
url,
recv_queue_size=2048,
send_queue_size=2048
)
```
--------------------------------
### curl_cffi.requests.WebSocket.connect
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
Establishes a connection to the specified WebSocket URL with various configuration options.
```APIDOC
## WebSocket.connect
### Description
Connects to the WebSocket server. Supports headers, cookies, authentication, proxy settings, and browser impersonation.
### Parameters
- **url** (str) - Required - The WebSocket URL to connect to.
- **params** (dict/list/tuple) - Optional - Query string parameters.
- **headers** (dict) - Optional - Headers to send.
- **cookies** (dict) - Optional - Cookies to use.
- **auth** (tuple) - Optional - HTTP basic auth (username, password).
- **timeout** (float) - Optional - Connection timeout in seconds.
- **impersonate** (str) - Optional - Browser version or fingerprint to impersonate.
- **proxy** (str) - Optional - Proxy URL.
- **verify** (bool) - Optional - Whether to verify HTTPS certificates.
```
--------------------------------
### Execute Concurrent Requests with asyncio.gather
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Demonstrates managing multiple concurrent requests by gathering tasks created within an AsyncSession.
```python
import asyncio
from curl_cffi import AsyncSession
urls = [
"https://google.com/",
"https://facebook.com/",
"https://apple.com/",
]
async with AsyncSession() as s:
tasks = []
for url in urls:
task = s.get(url)
tasks.append(task)
results = await asyncio.gather(*tasks)
```
--------------------------------
### List Fingerprints with curl-cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/pro.md
List native and cached fingerprints in a table format. Use the --json flag to output fingerprints as JSON.
```bash
curl-cffi list
```
```bash
curl-cffi list --json
```
--------------------------------
### Using CurlFollow.SAFE for Redirects
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/security.md
Demonstrates how to use `CurlFollow.SAFE` to prevent SSRF by rejecting redirects to internal IP addresses. The string 'safe' is also accepted as a shorthand.
```python
from curl_cffi import CurlFollow, requests
# Recommended for server-side code that fetches user-supplied URLs
r = requests.get(url, allow_redirects=CurlFollow.SAFE)
# The string "safe" is also accepted
r = requests.get(url, allow_redirects="safe")
# Session-level setting
s = requests.Session(allow_redirects=CurlFollow.SAFE)
```
--------------------------------
### Verbose Output
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Display full verbose output, including request and response headers and body.
```bash
curl-cffi post -v https://httpbin.org/post name=test
```
--------------------------------
### Update fingerprints via CLI
Source: https://github.com/lexiforest/curl_cffi/blob/main/README.md
Command to fetch the latest fingerprints from impersonate.pro.
```bash
curl-cffi update
```
--------------------------------
### Advanced fingerprint customization
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Demonstrates how to retrieve and modify a fingerprint object before passing it to the impersonate parameter.
```python
fingerprint = curl_cffi.get_fingerprint("edge_146_macos_26")
fingerprint.headers["User-Agent"] = "..."
r = curl_cffi.get(
"https://httpbin.org/headers",
impersonate=fingerprint,
)
```
--------------------------------
### Include Request Body from File
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
Use the `< filepath` syntax to include the request body from an external file. This is useful for large or complex request bodies.
```text
POST https://api.example.com/upload
Content-Type: application/json
< body.json
```
--------------------------------
### Define Multiple HTTP Requests in a File
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
The .http format allows defining multiple requests in a single file, separated by `###`. Each request can include a method, URL, headers, and body.
```text
### Get user list
GET https://api.example.com/users
### Create a user
POST https://api.example.com/users
Content-Type: application/json
{"name": "Alice", "email": "alice@example.com"}
### Get a specific user
GET https://api.example.com/users/1
```
--------------------------------
### Integrate curl_cffi as a requests adapter
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/community.md
Shows how to use curl-adapter to mount curl_cffi into a standard requests.Session. This enables the use of curl_cffi's impersonation features while maintaining the requests API.
```python
import requests
from curl_adapter import CurlCffiAdapter
session = requests.Session()
session.mount("http://", CurlCffiAdapter())
session.mount("https://", CurlCffiAdapter())
session.get("https://example.com")
```
--------------------------------
### Set API Key via Environment Variable
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/fingerprints.md
Override the configured API key at runtime by setting the IMPERSONATE_API_KEY environment variable. This takes precedence over configured keys.
```sh
export IMPERSONATE_API_KEY=imp_xxxxxxxx
```
--------------------------------
### TLS PSK Extension Flow with Rotating Proxies
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/psk.md
Illustrates a scenario where a TLS PSK identity, associated with one IP address, is reused with a different IP address, leading to a blocked connection. This highlights the problem solved by `proxy_credential_no_reuse`.
```text
┌───────────┐ ┌───────────┐
│ │ │ │
│ │ IP: 10.0.0.1 │ │
│ ┼─────────────TLS─Hello──────────────► │
│ │ │ │
│ ◄─────────────PSK:─xxx───────────────┼ │
│ │ │ │
│ │ │ │
│ │ │ Server │
│ Client │ IP: 10.0.0.2 │ │
│ ┼─────────────TLS─with─PSK───────────► │
│ │ │ │
│ ◄─────────────Blocked────────────────┼ │
│ │ │ │
│ │ PSK: xxx was │ │
│ │ associated with │ │
│ │ 10.0.0.1, not │ │
└───────────┘ 10.0.0.2 └───────────┘
```
--------------------------------
### Configure Reliability and Retries
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/websockets.md
Set up automatic retry strategies and ensure data integrity during network errors.
```python
from curl_cffi import WebSocketRetryStrategy
# Retry transient read errors up to 5 times
retry_policy = WebSocketRetryStrategy(
retry=True,
count=5
)
ws = await session.ws_connect(
url,
drain_on_error=True,
ws_retry=retry_policy
)
```
--------------------------------
### Upload files with multipart
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Use CurlMime to construct multipart requests as the files API is not supported.
```python
mp = curl_cffi.CurlMime()
mp.addpart(
name="attachment", # field name in the form
content_type="image/png", # mime type
filename="image.png", # filename seen by remote server
local_path="./image.png", # local file to upload
data=file.read(), # if you already have the data in memory
)
r = curl_cffi.post("https://httpbin.org/post", data={"foo": "bar"}, multipart=mp)
print(r.json())
```
--------------------------------
### Set HTTP2 Pseudo-Headers Order with CurlOpt
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/customize.md
Use `curl.setopt` with `CurlOpt.HTTP2_PSEUDO_HEADERS_ORDER` to specify the exact order of HTTP/2 pseudo-headers.
```python
import curl_cffi
from curl_cffi import Curl, CurlOpt
c = Curl()
c.setopt(CurlOpt.HTTP2_PSEUDO_HEADERS_ORDER, "masp")
```
--------------------------------
### Toggle default headers
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Compares the default header behavior with the option to disable them using default_headers=False.
```python
>>> r = curl_cffi.get("https://httpbin.org/headers", impersonate="chrome")
>>> print(r.text)
{
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"Accept-Encoding": "gzip, deflate, br, zstd",
"Accept-Language": "en-US,en;q=0.9",
"Host": "httpbin.org",
"Priority": "u=0, i",
"Sec-Ch-Ua": "\"Chromium\";v=\"136\", \"Google Chrome\";v=\"136\", \"Not.A/Brand\";v=\"99\"",
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Ch-Ua-Platform": "\"macOS\"",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-68452cc9-7287427f222e720c57971297"
}
}
>>> r = curl_cffi.get("https://httpbin.org/headers", impersonate="chrome", default_headers=False)
>>> print(r.text)
{
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate, br",
"Host": "httpbin.org",
"X-Amzn-Trace-Id": "Root=1-68452d20-2cf4cf00201987301c476c06"
}
}
```
--------------------------------
### Customize HTTP Header Order with extra_fp
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/impersonate/customize.md
Use the `extra_fp` dictionary to further refine TLS impersonation by specifying custom header ordering.
```python
extra_fp = {
"header_order": "User-Agent,Host,Connection"
}
```
--------------------------------
### Set Cookies with Curl-Cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/request_params.md
Shows how to set cookies for a request using the '+' separator. Multiple cookies can be set in a single command.
```bash
# Set a single cookie
curl-cffi get https://httpbin.org/cookies +session=abc123
```
```bash
# Set multiple cookies
curl-cffi get https://httpbin.org/cookies +session=abc123 +theme=dark
```
--------------------------------
### Configure Proxies in curl_cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/advanced.md
Use the proxy parameter for a single proxy or a dictionary for protocol-specific proxies.
```python
import curl_cffi
curl_cffi.get(url, proxy="http://user:pass@example.com:3128")
```
```python
import curl_cffi
proxies = {
"http": "http://localhost:3128",
"https": "http://localhost:3128"
}
curl_cffi.get(url, proxies=proxies)
```
--------------------------------
### Handling cookies in redirects
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/quick_start.md
Use sessions to ensure cookies are correctly tracked across redirects.
```python
import curl_cffi
r = curl_cffi.get("https://httpbin.org/redirect")
# ❌ Cookie from previous redirect may be lost.
do_something(r.cookies)
s = curl_cffi.Session()
r = s.get("https://httpbin.org/redirect")
# ✅ Use a session instead, to retrive all cookies in the session
do_something(s.cookies)
```
--------------------------------
### Update Fingerprints with curl-cffi
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/pro.md
Download the latest fingerprints for impersonate.pro. Successful updates will print the total number of fingerprints now stored in the local cache.
```bash
curl-cffi update
```
--------------------------------
### Manage Session State in Batch Execution
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/run.md
By default, requests in a batch share a session (cookies and connections persist). Use `--no-session` to execute each request independently.
```bash
# Default: shared session (cookies carry over between requests)
curl-cffi run requests.http
```
```bash
# Independent requests (no shared state)
curl-cffi run --no-session requests.http
```
--------------------------------
### AsyncCurl
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/api.md
A wrapper around the curl_multi handle to provide asyncio support using libcurl socket_action APIs.
```APIDOC
## class curl_cffi.AsyncCurl(cacert: str = '', loop=None)
### Description
Provides asyncio support for curl operations by managing multiple handles.
### Methods
- **add_handle(curl)**: Adds a curl handle to be managed.
- **remove_handle(curl)**: Cancels a future for a given handle.
- **setopt(option, value)**: Sets options for the multi handle.
- **close()**: Closes and cleans up resources.
```
--------------------------------
### Custom Output with --print Flag
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/cli/options.md
Employ the `--print` flag with a character string to specify fine-grained output control. 'H' for request headers, 'B' for request body, 'h' for response headers, and 'b' for response body.
```bash
# Custom: request headers + response headers
curl-cffi get -p Hh https://httpbin.org/get
```
--------------------------------
### Body Only Output
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Print only the response body, useful for piping or parsing.
```bash
curl-cffi get --body https://httpbin.org/get
```
--------------------------------
### HTTP/3 Request
Source: https://github.com/lexiforest/curl_cffi/blob/main/skills/imp-fetch/SKILL.md
Force the request to use the HTTP/3 protocol.
```bash
curl-cffi get --http3 https://fp.impersonate.pro/api/http3
```
--------------------------------
### Package with PyInstaller using --hidden-import
Source: https://github.com/lexiforest/curl_cffi/blob/main/docs/faq.md
When packaging with PyInstaller, use the `--hidden-import` option to include necessary modules like `_cffi_backend` and `--collect-all` for packages like `curl_cffi`.
```default
pyinstaller -F .\example.py --hidden-import=_cffi_backend --collect-all curl_cffi
```