### Basic urllib3 request example Source: https://urllib3.readthedocs.io/en/stable/index.html Shows how to make a simple GET request using urllib3.request and access the response status and data. ```python >>> import urllib3 >>> resp = urllib3.request("GET", "https://httpbin.org/robots.txt") >>> resp.status 200 >>> resp.data b"User-agent: *\nDisallow: /deny\n" ``` -------------------------------- ### Install urllib3 from source Source: https://urllib3.readthedocs.io/en/stable/index.html Commands to clone the urllib3 repository from GitHub and install it locally. ```bash $ git clone https://github.com/urllib3/urllib3.git $ cd urllib3 $ pip install . ``` -------------------------------- ### Basic urllib3 request in Pyodide Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/emscripten.html Shows how to make an HTTP request using urllib3 in a Pyodide environment. The response status, headers, and JSON body are printed. This example is from the 'Getting started' section. ```python import urllib3 resp = urllib3.request("GET", "https://httpbin.org/anything") print(resp.status) # 200 print(resp.headers) # HTTPHeaderDict(...) print(resp.json()) # {"headers": {"Accept": "*/*", ...}, ...} ``` -------------------------------- ### Using requests with urllib3 in Pyodide Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/emscripten.html Shows how to use the requests library, which is built on urllib3, in a Pyodide environment. The response status code and headers are printed. This example is from the 'Getting started' section. ```python import requests resp = requests.request("GET", "https://httpbin.org/anything") print(resp.status_code) # 200 print(resp.headers) ``` -------------------------------- ### Install nox for development environment Source: https://urllib3.readthedocs.io/en/stable/contributing.html Install nox with pip to set up the development environment. This is the only required tool for contributing. ```bash $ python -m pip install --user --upgrade nox ``` -------------------------------- ### Install urllib3 with SOCKS support Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html To use SOCKS proxies, install PySocks or install urllib3 with the socks extra. ```bash python -m pip install urllib3[socks] ``` -------------------------------- ### make_headers example Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Shows how to use urllib3.util.make_headers to generate HTTP headers. The example prints the resulting headers for keep_alive and user_agent, and for accept_encoding. ```python import urllib3 print(urllib3.util.make_headers(keep_alive=True, user_agent="Batman/1.0")) # {'connection': 'keep-alive', 'user-agent': 'Batman/1.0'} print(urllib3.util.make_headers(accept_encoding=True)) # {'accept-encoding': 'gzip,deflate'} ``` -------------------------------- ### Install urllib3 with pip Source: https://urllib3.readthedocs.io/en/stable/index.html Command to install urllib3 using pip. ```bash $ python -m pip install urllib3 ``` -------------------------------- ### Install pyOpenSSL dependencies Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/pyopenssl.html Install the required packages for the pyOpenSSL backend: pyOpenSSL, cryptography, and idna. Use this command before using the module. ```bash $ python -m pip install pyopenssl cryptography idna ``` -------------------------------- ### Install urllib3 with Brotli extra Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Install the Brotli package via the urllib3[brotli] extra to enable Brotli compression support. ```bash $ python -m pip install urllib3[brotli] ``` -------------------------------- ### Using urllib3.request() to send a GET request Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Shows the new top-level urllib3.request() function, which uses a global poolmanager instance. The example sends a GET request to https://example.com and prints the response status (200). ```python >>> import urllib3 >>> resp = urllib3.request("GET", "https://example.com") >>> resp.status 200 ``` -------------------------------- ### ProxyManager example: sharing pools for HTTP and HTTPS requests Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html This example creates a ProxyManager pointing to a local HTTPS proxy and makes requests to different hosts. It shows that plain HTTP requests share a single connection pool, while each HTTPS target gets its own pool, as indicated by the printed pool counts. ```python import urllib3 proxy = urllib3.ProxyManager("https://localhost:3128/") resp1 = proxy.request("GET", "http://google.com/") resp2 = proxy.request("GET", "http://httpbin.org/") # One pool was shared by both plain HTTP requests. print(len(proxy.pools)) # 1 resp3 = proxy.request("GET", "https://httpbin.org/") resp4 = proxy.request("GET", "https://twitter.com/") # A separate pool was added for each HTTPS target. print(len(proxy.pools)) # 3 ``` -------------------------------- ### Create a connection pool from a URL and make a GET request Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html Shows how to use connection_from_url() to create a ConnectionPool instance from a URL and then call request('GET', '/') on it. The URL must include the scheme; port is optional. Additional keyword arguments can be passed to the pool constructor. ```python >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') ``` -------------------------------- ### Timeout.start_connect() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Starts the timeout clock, used during a connect() attempt. Raises TimeoutStateError if the timer has already been started. ```APIDOC ## Timeout.start_connect() ### Description Starts the timeout clock, used during a connect() attempt. Raises TimeoutStateError if the timer has already been started. ### Method GET ### Endpoint urllib3.util.Timeout.start_connect ### Parameters None ### Response #### Success Response (200) - **return** (float) - The start time. #### Response Example ``` 0.0 ``` ### Errors - **urllib3.exceptions.TimeoutStateError** - If you attempt to start a timer that has been started already. ``` -------------------------------- ### Install certifi Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Installs the certifi package, which provides Mozilla's root certificate bundle for certificate verification. ```bash $ python -m pip install certifi ``` -------------------------------- ### Install urllib3 v2.x locally Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Upgrade urllib3 to version 2.x in your local environment to test your package against it. ```bash $ python -m pip install -U 'urllib3>=2' ``` -------------------------------- ### Run an HTTP proxy locally with python -m dummyserver.proxy Source: https://urllib3.readthedocs.io/en/stable/contributing.html Starts a local HTTP proxy server using the dummyserver module. Useful when developing proxy-related features. ```bash $ python -m dummyserver.proxy ``` -------------------------------- ### Pass query parameters via fields for GET/HEAD/DELETE Source: https://urllib3.readthedocs.io/en/stable/user-guide.html For `GET`, `HEAD`, and `DELETE` requests, pass query parameters as a dictionary in the `fields` argument to `urllib3.request()`. The example sends a GET request and prints the `args` from the response. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/get", fields={"arg": "value"} ) print(resp.json()["args"]) # {"arg": "value"} ``` -------------------------------- ### Make a GET request with PoolManager Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Sends a GET request to httpbin.org/robots.txt and prints the response data. ```python import urllib3 # Creating a PoolManager instance for sending requests. http = urllib3.PoolManager() # Sending a GET request and getting back response as HTTPResponse object. resp = http.request("GET", "https://httpbin.org/robots.txt") # Print the returned data. print(resp.data) # b"User-agent: *\nDisallow: /deny\n" ``` -------------------------------- ### Create multi-valued headers with HTTPHeaderDict Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Use `urllib3.HTTPHeaderDict` to add multiple values for the same header. The example adds two `Accept` headers and sends a GET request, resulting in a comma-separated header value. ```python import urllib3 # Create an HTTPHeaderDict and add headers headers = urllib3.HTTPHeaderDict() headers.add("Accept", "application/json") headers.add("Accept", "text/plain") # Make the request using the headers resp = urllib3.request( "GET", "https://httpbin.org/headers", headers=headers ) print(resp.json()["headers"]) # {"Accept": "application/json, text/plain", ...} ``` -------------------------------- ### Run an HTTPS proxy locally with python -m dummyserver.https_proxy Source: https://urllib3.readthedocs.io/en/stable/contributing.html Starts a local HTTPS proxy server using the dummyserver module. Useful when developing proxy-related features that require HTTPS. ```bash $ python -m dummyserver.https_proxy ``` -------------------------------- ### Read Set-Cookie header from response Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Cookies provided by the server are stored in the `Set-Cookie` header. The example sends a GET request with `redirect=False` and prints the `Set-Cookie` header. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/cookies/set/session/f3efe9db", redirect=False ) print(resp.headers["Set-Cookie"]) # session=f3efe9db; Path=/ ``` -------------------------------- ### Install urllib3 with Zstandard extra Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Install the urllib3[zstd] extra to enable Zstandard support on Python 3.13 and earlier. On Python 3.14+, Zstandard support is included in the standard library. Requires zstandard package v0.18.0 or later. ```bash # This is only necessary on Python 3.13 and earlier. # Otherwise zstandard support is included in the Python standard library. $ python -m pip install urllib3[zstd] ``` -------------------------------- ### Example output of getproxies() with problematic HTTPS proxy Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html This output shows a system-level proxy configuration where the HTTPS proxy uses https://, which is the problem. ```json { "http": "http://127.0.0.1:8888", "https": "https://127.0.0.1:8888" # <--- This setting is the problem! } ``` -------------------------------- ### Send request with custom headers using headers argument Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Specify headers as a dictionary in the `headers` argument to `urllib3.request()`. The example sends a GET request to httpbin.org/headers and prints the response headers. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/headers", headers={ "X-Something": "value" } ) print(resp.json()["headers"]) # {"X-Something": "value", ...} ``` -------------------------------- ### Example: parse_url and Url.url round-trip Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Shows how to parse a URL with urllib3.util.parse_url and convert it back to a string with the Url.url property. The output demonstrates that the round-trip preserves the URL, and that constructing a Url with explicit components yields the expected string. ```python import urllib3 U = urllib3.util.parse_url("https://google.com/mail/") print(U.url) # "https://google.com/mail/" print( urllib3.util.Url("https", "username:password", "host.com", 80, "/path", "query", "fragment" ).url ) # "https://username:password@host.com:80/path?query#fragment" ``` -------------------------------- ### Check and fix proxy environment variables in bash Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Inspect HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY environment variables. Ensure they start with http://, not https://. The example shows a problematic HTTPS_PROXY and how to fix it in the current session and persist it in the shell profile. ```bash # Check your existing environment variables in bash $ env | grep "_PROXY" HTTP_PROXY=http://127.0.0.1:8888 HTTPS_PROXY=https://127.0.0.1:8888 # <--- This setting is the problem! # Make the fix in your current session and test your script $ export HTTPS_PROXY="http://127.0.0.1:8888" $ python test-proxy.py # This should now pass. # Persist your change in your shell 'profile' (~/.bashrc, ~/.profile, ~/.bash_profile, etc) # You may need to logout and log back in to ensure this works across all programs. $ vim ~/.bashrc ``` -------------------------------- ### Example FutureWarning message Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Shows the format of a deprecation warning for the 'ssl_version' option, which is deprecated in favor of 'ssl_minimum_version'. ```text FutureWarning: 'ssl_version' option is deprecated and will be removed in urllib3 v3.0. Instead use 'ssl_minimum_version' ``` -------------------------------- ### PoolManager.connection_from_host() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a host and port. ```APIDOC ## PoolManager.connection_from_host() ### Description Gets a connection from a host and port. ### Method Method call ### Endpoint PoolManager.connection_from_host() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_host(host, port, scheme) ``` ### Response Returns a connection object. ``` -------------------------------- ### Create PoolManager and make requests Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Shows how to instantiate urllib3.PoolManager with num_pools=2, make three GET requests, and print the number of pools. The output is 2 because the pool manager caches up to num_pools pools before discarding the least recently used. ```python import urllib3 http = urllib3.PoolManager(num_pools=2) resp1 = http.request("GET", "https://google.com/") resp2 = http.request("GET", "https://google.com/mail") resp3 = http.request("GET", "https://yahoo.com/") print(len(http.pools)) # 2 ``` -------------------------------- ### ProxyManager.connection_from_host() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a host and port through the proxy. ```APIDOC ## ProxyManager.connection_from_host() ### Description Gets a connection from a host and port through the proxy. ### Method Method call ### Endpoint ProxyManager.connection_from_host() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = proxy_manager.connection_from_host(host, port, scheme) ``` ### Response Returns a connection object. ``` -------------------------------- ### Run specific test sessions with nox Source: https://urllib3.readthedocs.io/en/stable/contributing.html Runs the test-3.12 and test-pypy3.11 sessions. Nox creates virtualenvs if needed, installs dependencies, and runs commands in order. ```bash $ nox --reuse-existing-virtualenvs --sessions test-3.12 test-pypy3.11 [ Nox will create virtualenv if needed, install the specified dependencies, and run the commands in order.] ``` -------------------------------- ### Check dependencies with pipdeptree Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Install pipdeptree and list packages that require urllib3 to identify version constraints. The output shows botocore requiring urllib3>=1.25.4,<1.27 and requests requiring urllib3>=1.21.1,<3. ```bash # From inside your Python environment: $ python -m pip install pipdeptree # We only care about packages requiring urllib3 $ pipdeptree --reverse | grep "requires: urllib3" ├── botocore==1.38.36 [requires: urllib3>=1.25.4,<1.27] └── requests==2.32.4 [requires: urllib3>=1.21.1,<3] ``` -------------------------------- ### Request with Zstandard Accept-Encoding header Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Example of using zstd encoding via the Accept-Encoding header in a urllib3 request. Requires Zstandard support to be enabled. ```python import urllib3 urllib3.request( "GET", "https://www.facebook.com/", headers={"Accept-Encoding": "zstd"} ) ``` -------------------------------- ### connection_from_host(host, port=None, scheme='http', pool_kwargs=None) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Gets a ConnectionPool based on the host, port, and scheme. If port isn't given, it will be derived from the scheme using urllib3.connectionpool.port_by_scheme. If pool_kwargs is provided, it is merged with the instance's connection_pool_kw variable and used to create the new connection pool, if one is needed. ```APIDOC ## connection_from_host(host, port=None, scheme='http', pool_kwargs=None) ### Description Get a urllib3.connectionpool.ConnectionPool based on the host, port, and scheme. If `port` isn't given, it will be derived from the `scheme` using `urllib3.connectionpool.port_by_scheme`. If `pool_kwargs` is provided, it is merged with the instance's `connection_pool_kw` variable and used to create the new connection pool, if one is needed. ### Parameters - **host** (str | None) - Required - The host name. - **port** (int | None) - Optional - The port number. If not given, derived from scheme. - **scheme** (str | None) - Optional - The URL scheme (e.g., 'http', 'https'). Defaults to 'http'. - **pool_kwargs** (dict[str, Any] | None) - Optional - Additional keyword arguments for creating the connection pool. ### Return Type HTTPConnectionPool ``` -------------------------------- ### urllib3.request() - Query Parameters Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Explains how to pass query parameters using the `fields` argument for GET, HEAD, and DELETE requests, and manual URL encoding for POST and PUT requests. ```APIDOC ## urllib3.request() - Query Parameters ### Description For GET, HEAD, and DELETE requests, query parameters can be passed as a dictionary in the `fields` argument. For POST and PUT requests, parameters must be manually encoded into the URL. ### Method GET ### Endpoint https://httpbin.org/get ### Parameters #### Query Parameters - **fields** (dict) - Required - Query parameters to include in the request. ### Request Example ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/get", fields={"arg": "value"} ) print(resp.json()["args"]) # {"arg": "value"} ``` ### Response #### Success Response (200) - **args** (dict) - The query parameters sent in the request. #### Response Example ```json {"arg": "value"} ``` ### POST with Encoded Query Parameters ```python from urllib.parse import urlencode import urllib3 encoded_args = urlencode({"arg": "value"}) url = "https://httpbin.org/post?" + encoded_args resp = urllib3.request("POST", url) print(resp.json()["args"]) # {"arg": "value"} ``` ``` -------------------------------- ### connection_from_context(request_context) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Gets a ConnectionPool based on the request context. The request_context must at least contain the 'scheme' key and its value must be a key in key_fn_by_scheme instance variable. ```APIDOC ## connection_from_context(request_context) ### Description Get a urllib3.connectionpool.ConnectionPool based on the request context. `request_context` must at least contain the `scheme` key and its value must be a key in `key_fn_by_scheme` instance variable. ### Parameters - **request_context** (dict[str, Any]) - Required - The request context containing at least the 'scheme' key. ### Return Type HTTPConnectionPool ``` -------------------------------- ### Request with Brotli Accept-Encoding header Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Example of using Brotli encoding by setting the Accept-Encoding header to 'br' in a urllib3 request. ```python import urllib3 urllib3.request( "GET", "https://www.google.com/", headers={"Accept-Encoding": "br"} ) ``` -------------------------------- ### ProxyManager.connection_from_host Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Get a ConnectionPool based on host, port, and scheme. If port is not given, it is derived from the scheme. If pool_kwargs is provided, it is merged with the instance's connection_pool_kw variable. ```APIDOC ## ProxyManager.connection_from_host ### Description Get a `urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme. If `port` isn't given, it will be derived from the `scheme` using `urllib3.connectionpool.port_by_scheme`. If `pool_kwargs` is provided, it is merged with the instance's `connection_pool_kw` variable and used to create the new connection pool, if one is needed. ### Method connection_from_host ### Parameters #### Path Parameters - **host** (str | None) - Required - The hostname for the connection pool. - **port** (int | None) - Optional - The port number. If not provided, derived from scheme. - **scheme** (str | None) - Optional - The URL scheme (e.g., 'http', 'https'). Defaults to 'http'. - **pool_kwargs** (dict[str, Any] | None) - Optional - Additional keyword arguments to merge with connection_pool_kw. ### Return Type `urllib3.HTTPConnectionPool` ``` -------------------------------- ### Create PoolManager with certificate verification Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Creates a PoolManager that verifies certificates using certifi's CA bundle. This is the recommended setup for HTTPS requests. ```python import certifi import urllib3 http = urllib3.PoolManager( cert_reqs="CERT_REQUIRED", ca_certs=certifi.where() ) ``` -------------------------------- ### Create and customize SSL context for urllib3 PoolManager Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Use create_urllib3_context() to get a compatible SSLContext, load default certificates, and add the OP_ENABLE_MIDDLEBOX_COMPAT flag. This differs from passing an options argument because it adds to the default options instead of overwriting them. ```python import ssl from urllib3 import PoolManager from urllib3.util import create_urllib3_context ctx = create_urllib3_context() ctx.load_default_certs() ctx.options |= ssl.OP_ENABLE_MIDDLEBOX_COMPAT with PoolManager(ssl_context=ctx) as pool: pool.request("GET", "https://www.google.com/") ``` -------------------------------- ### Use io.BufferedReader for buffered reading Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Wrap the HTTPResponse in an io.BufferedReader to enable buffered reads. The example reads 4 bytes and then releases the connection. This is useful for efficient reading of large responses. ```python import io import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/bytes/1024", preload_content=False ) reader = io.BufferedReader(resp, 8) print(reader.read(4)) # b"\xbf\x9c\xd6" resp.release_conn() ``` -------------------------------- ### PoolManager.connection_from_context() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a context object. ```APIDOC ## PoolManager.connection_from_context() ### Description Gets a connection from a context object. ### Method Method call ### Endpoint PoolManager.connection_from_context() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_context(context) ``` ### Response Returns a connection object. ``` -------------------------------- ### PoolManager.connection_from_url() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a URL. ```APIDOC ## PoolManager.connection_from_url() ### Description Gets a connection from a URL. ### Method Method call ### Endpoint PoolManager.connection_from_url() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_url(url) ``` ### Response Returns a connection object. ``` -------------------------------- ### Use Retry instance for granular control (3 retries, 2 redirects) Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Use a urllib3.Retry instance to control retries and redirects. This example sets total retries to 3 and redirects to 2, resulting in a MaxRetryError. ```python urllib3.request( "GET", "https://httpbin.org/redirect/3", retries=urllib3.Retry(3, redirect=2) ) # MaxRetryError ``` -------------------------------- ### HTTPConnection.getresponse() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connection.html Get the response from the server. Returns an HTTPResponse instance if the connection is in the correct state, otherwise raises ResponseNotReady. ```APIDOC ## getresponse() ### Description Get the response from the server. If the HTTPConnection is in the correct state, returns an instance of HTTPResponse or of whatever object is returned by the response_class variable. If a request has not been sent or if a previous response has not been handled, ResponseNotReady is raised. If the HTTP response indicates that the connection should be closed, then it will be closed before the response is returned. When the connection is closed, the underlying socket is closed. ### Return Type HTTPResponse ``` -------------------------------- ### HTTPConnectionPool.urlopen(method, url, ...) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. More commonly, it's appropriate to use a convenience method such as request(). ```APIDOC ## urlopen(method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_TYPE_DEFAULT.token, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, preload_content=True, decode_content=True, **response_kw) ### Description Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. ### Parameters - **method** (str) - Required - HTTP request method (such as GET, POST, PUT, etc.) - **url** (str) - Required - The URL to perform the request on. - **body** (bytes | IO[Any] | Iterable[bytes | str] | str | None) - Optional - Data to send in the request body, either str, bytes, an iterable of str/bytes, or a file-like object. - **headers** (Mapping[str, str] | None) - Optional - Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. - **retries** (Retry, False, or an int) - Optional - Configure the number of retries to allow before raising a MaxRetryError exception. - **redirect** (bool) - Optional - Whether to follow redirects. - **assert_same_host** (bool) - Optional - Whether to assert that the host is the same as the pool's host. - **timeout** (float | None) - Optional - Timeout for the request. - **pool_timeout** (float | None) - Optional - Timeout for getting a connection from the pool. - **release_conn** (bool | None) - Optional - Whether to release the connection after the request. - **chunked** (bool) - Optional - Whether to use chunked transfer encoding. - **body_pos** (int | None) - Optional - Position in the body to start reading from. - **preload_content** (bool) - Optional - Whether to preload the response content. - **decode_content** (bool) - Optional - Whether to decode the response content. - **response_kw** - Optional - Additional keyword arguments for the response. ### Notes - More commonly, it's appropriate to use a convenience method such as request(). - release_conn will only behave as expected if preload_content=False because we want to make preload_content=False the default behaviour someday soon without breaking backwards compatibility. ### Example ```python pool.urlopen("GET", "/path", headers={"User-Agent": "my-app"}) ``` ``` -------------------------------- ### Access binary response content Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Shows that the data attribute of the response is always a byte string. The example prints the raw bytes returned by the server. ```python import urllib3 resp = urllib3.request("GET", "https://httpbin.org/bytes/8") print(resp.data) # b"\xaa\xa5H?\x95\xe9\x9b\x11" ``` -------------------------------- ### Build documentation locally with nox -rs docs Source: https://urllib3.readthedocs.io/en/stable/contributing.html Builds the documentation locally using the nox tool. Run this command from the project root. ```bash $ nox -rs docs ``` -------------------------------- ### Timeout.get_connect_duration() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Gets the time elapsed since the call to start_connect(). Raises TimeoutStateError if the timer hasn't been started. ```APIDOC ## Timeout.get_connect_duration() ### Description Gets the time elapsed since the call to start_connect(). ### Method get_connect_duration() ### Returns Elapsed time in seconds. ### Return Type float ### Raises - **urllib3.exceptions.TimeoutStateError** - if you attempt to get duration for a timer that hasn't been started. ``` -------------------------------- ### Disable retries and redirects with retries=False Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Setting retries=False disables all retry and redirect logic. The example shows a NewConnectionError for a non-existent domain and a 302 response for a redirect. ```python import urllib3 urllib3.request( "GET", "https://nxdomain.example.com", retries=False ) # NewConnectionError resp = urllib3.request( "GET", "https://httpbin.org/redirect/1", retries=False ) print(resp.status) # 302 ``` -------------------------------- ### Example of fields parameter for request_encode_body Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Shows the structure of the fields parameter for request_encode_body, supporting key/value strings and file tuples. A filetuple is (filename, data, MIME type) where MIME type is optional. Providing a filename is optional but recommended for browser-like behavior. ```python fields = { 'foo': 'bar', 'fakefile': ('foofile.txt', 'contents of foofile'), 'realfile': ('barfile.txt', open('realfile').read()), 'typedfile': ('bazfile.bin', open('bazfile').read(), 'image/jpeg'), 'nonamefile': 'contents of nonamefile field', } ``` -------------------------------- ### Use http:// scheme with urllib3.ProxyManager Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html When using urllib3 directly, ensure the proxy URL passed to ProxyManager starts with http:// instead of https:// to avoid ProxyError. ```python # Do this: http = urllib3.ProxyManager("http://...") # Not this: http = urllib3.ProxyManager("https://...") ``` -------------------------------- ### Manually encode query parameters for POST/PUT Source: https://urllib3.readthedocs.io/en/stable/user-guide.html For `POST` and `PUT` requests, you must manually encode query parameters in the URL using `urlencode`. The example encodes args and appends them to the URL before sending a POST request. ```python from urllib.parse import urlencode import urllib3 # Encode the args into url grammar. encoded_args = urlencode({"arg": "value"}) # Create a URL with args encoded. url = "https://httpbin.org/post?" + encoded_args resp = urllib3.request("POST", url) print(resp.json()["args"]) # {"arg": "value"} ``` -------------------------------- ### Sending JSON in a request and deserializing a JSON response Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Demonstrates the new json= parameter for requests, which automatically adds 'Content-Type: application/json' by default, and the HTTPResponse.json() method for deserializing response bodies. The example shows a POST request with a JSON body and a GET request whose response is parsed with resp.json(). ```python import urllib3 # Send a request with a JSON body. # This adds 'Content-Type: application/json' by default. resp = urllib3.request( "POST", "https://example.api.com", json={"key": "value"} ) # Receive a JSON body in the response. resp = urllib3.request("GET", "https://xkcd.com/2347/info.0.json") # There's always an XKCD... resp.json() { "num": 2347, "img": "https://imgs.xkcd.com/comics/dependency.png", "title": "Dependency", ... } ``` -------------------------------- ### HTTPConnection.__init__ Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connection.html Constructor for HTTPConnection. Accepts host, port, and various keyword arguments to configure the connection, such as timeout, source_address, blocksize, socket_options, proxy, and proxy_config. ```APIDOC ## HTTPConnection.__init__ ### Description Initializes a new HTTPConnection instance. Provides a backwards-compatibility layer between older and newer Pythons. ### Method Constructor ### Signature `HTTPConnection(host, port=None, *, timeout=_TYPE_DEFAULT.token, source_address=None, blocksize=16384, socket_options=[(6, 1, 1)], proxy=None, proxy_config=None)` ### Parameters - **host** (str) - Required - The host to connect to. - **port** (int | None) - Optional - The port to connect to. Defaults to None. - **timeout** (_TYPE_TIMEOUT) - Optional - Timeout for the connection. - **source_address** (tuple[str, int] | None) - Optional - Source address for the connection. - **blocksize** (int) - Optional - Block size for the connection. Defaults to 16384. - **socket_options** (list[tuple[int, int, int | bytes]] | None) - Optional - Socket options to set on the underlying socket. If not specified, defaults are loaded from `HTTPConnection.default_socket_options`. - **proxy** (Url | None) - Optional - Proxy URL. - **proxy_config** (ProxyConfig | None) - Optional - Proxy configuration. ### Notes - `socket_options` can be used to enable TCP Keep Alive by adding to the defaults, or disable defaults by passing an empty list. ### Example ```python HTTPConnection.default_socket_options + [ (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1), ] ``` ``` -------------------------------- ### connection_from_url(url, **kw) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html Given a url, return a ConnectionPool instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating a ConnectionPool instance. ```APIDOC ## connection_from_url(url, **kw) ### Description Given a url, return a ConnectionPool instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating a ConnectionPool instance. ### Method connection_from_url(url, **kw) ### Parameters - **url** (str) - Required - Absolute URL string that must include the scheme. Port is optional. - **\*\*kw** (Any) - Optional - Passes additional parameters to the constructor of the appropriate ConnectionPool. Useful for specifying things like timeout, maxsize, headers, etc. ### Return Type HTTPConnectionPool ### Example ``` >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') ``` ``` -------------------------------- ### Check system-level proxy settings with urllib.request.getproxies() Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html On Windows or macOS, proxy settings may be set at the system level. Run this command to see the current proxy configuration. ```python $ python -c 'import urllib.request; print(urllib.request.getproxies())' ``` -------------------------------- ### Update install_requires to support urllib3 1.26 and 2.x Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Set your package's dependency specifier to allow both urllib3 1.26.x and 2.x. Shown for setup.py (setuptools) and pyproject.toml (hatch). ```python # setup.py (setuptools) setup( ... install_requires=["urllib3>=1.26,<3"] ) ``` ```toml # pyproject.toml (hatch) [project] dependencies = [ "urllib3>=1.26,<3" ] ``` -------------------------------- ### PoolManager.connection_from_pool_key() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a pool key. ```APIDOC ## PoolManager.connection_from_pool_key() ### Description Gets a connection from a pool key. ### Method Method call ### Endpoint PoolManager.connection_from_pool_key() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_pool_key(pool_key) ``` ### Response Returns a connection object. ``` -------------------------------- ### urllib3.util.create_urllib3_context() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Creates an SSLContext with urllib3's default settings. ```APIDOC ## create_urllib3_context() ### Description Creates an SSLContext with urllib3's default settings. ### Parameters None ### Returns SSLContext - The created SSL context. ``` -------------------------------- ### Access response status, data, and headers Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Shows how to make a request and print the status, data, and headers of the HTTPResponse object. The output demonstrates the expected format for each attribute. ```python import urllib3 # Making the request (The request function returns HTTPResponse object) resp = urllib3.request("GET", "https://httpbin.org/ip") print(resp.status) # 200 print(resp.data) # b"{\n "origin": "104.232.115.37"\n}\n" print(resp.headers) # HTTPHeaderDict({"Content-Length": "32", ...}) ``` -------------------------------- ### HTTPConnection.connect Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connection.html Connects to the host and port specified in __init__. ```APIDOC ## HTTPConnection.connect ### Description Connects to the host and port specified in __init__. ### Method connect() ### Return Type None ``` -------------------------------- ### Specify client certificate with cert_file, cert_reqs, ca_certs Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Provide the full path to a client certificate and certificate bundle when creating a PoolManager. This is useful when both server and client need to verify each other's identity, typically with certificates issued from the same authority. ```python http = urllib3.PoolManager( cert_file="/path/to/your/client_cert.pem", cert_reqs="CERT_REQUIRED", ca_certs="/path/to/your/certificate_bundle" ) ``` -------------------------------- ### PoolManager.connection_from_url Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Similar to `urllib3.connectionpool.connection_from_url()`. If `pool_kwargs` is not provided and a new pool needs to be constructed, `self.connection_pool_kw` is used to initialize the ConnectionPool. If `pool_kwargs` is provided, it is used instead. Note that if a new pool does not need to be created, the provided `pool_kwargs` are not used. ```APIDOC ## PoolManager.connection_from_url ### Description Similar to `urllib3.connectionpool.connection_from_url()`. If `pool_kwargs` is not provided and a new pool needs to be constructed, `self.connection_pool_kw` is used to initialize the ConnectionPool. If `pool_kwargs` is provided, it is used instead. Note that if a new pool does not need to be created, the provided `pool_kwargs` are not used. ### Method connection_from_url ### Parameters - **url** (str) - Required - The URL to create a connection pool from. - **pool_kwargs** (dict[str, Any] | None) - Optional - Additional keyword arguments for the pool. If not provided, `self.connection_pool_kw` is used. ### Return Type HTTPConnectionPool ``` -------------------------------- ### urllib3.exceptions.HostChangedError Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.exceptions.html Raised when an existing pool gets a request for a foreign host. This exception indicates a host mismatch in connection pooling. ```APIDOC ## urllib3.exceptions.HostChangedError ### Description Raised when an existing pool gets a request for a foreign host. ### Constructor `urllib3.exceptions.HostChangedError(pool, url, retries=3)` ### Parameters - **pool** (ConnectionPool) - Required - The connection pool. - **url** (str) - Required - The URL of the request. - **retries** (Retry | int) - Optional - Number of retries, default is 3. ### Return Type None ``` -------------------------------- ### connection_from_pool_key(pool_key, request_context) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Gets a ConnectionPool based on the pool key and request context. This method is part of the PoolManager class. ```APIDOC ## connection_from_pool_key(pool_key, request_context) ### Description Get a urllib3.connectionpool.ConnectionPool based on the pool key and request context. ### Parameters - **pool_key** - Required - The pool key used to identify the connection pool. - **request_context** - Required - The request context. ### Return Type HTTPConnectionPool ``` -------------------------------- ### Activate pyOpenSSL certificate checking with inject_into_urllib3() Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/pyopenssl.html Call inject_into_urllib3() before making HTTP requests to enable certificate checking. Wrap in try/except ImportError to gracefully handle missing dependencies. Can be placed in a sitecustomize module or before application startup. ```python try: import urllib3.contrib.pyopenssl urllib3.contrib.pyopenssl.inject_into_urllib3() except ImportError: pass ``` -------------------------------- ### Timeout.read_timeout property Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Gets the value for the read timeout. This assumes some time has elapsed in the connection timeout and computes the read timeout appropriately. ```APIDOC ## Timeout.read_timeout property ### Description Gets the value for the read timeout. This assumes some time has elapsed in the connection timeout and computes the read timeout appropriately. ### Property read_timeout ### Returns The read timeout value. ``` -------------------------------- ### request_encode_url Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Makes a request using urlopen() with the fields encoded in the URL. Useful for request methods like GET, HEAD, DELETE, etc. ```APIDOC ## request_encode_url(method, url, fields=None, headers=None, **urlopen_kw) ### Description Make a request using `urlopen()` with the `fields` encoded in the url. This is useful for request methods like GET, HEAD, DELETE, etc. ### Method Python method ### Signature `request_encode_url(method, url, fields=None, headers=None, **urlopen_kw)` ### Parameters - **method** (str) - Required - HTTP request method (such as GET, POST, PUT, etc.) - **url** (str) - Required - The URL to perform the request on. - **fields** (Sequence[tuple[str, str | bytes | tuple[str, str | bytes] | tuple[str, str | bytes, str]]] | Mapping[str, str | bytes | tuple[str, str | bytes] | tuple[str, str | bytes, str]] | RequestField | None) - Optional - Data to encode and send in the request body. - **headers** (Mapping[str, str] | None) - Optional - Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. - **encode_multipart** (bool) - Optional - If True, encode the `fields` using the multipart/form-data MIME format. - **multipart_boundary** (str | None) - Optional - If not specified, then a random boundary will be generated using `urllib3.filepost.choose_boundary()`. - **urlopen_kw** (str) - Optional - Additional keyword arguments passed to `urlopen()`. ### Return Type `urllib3.response.BaseHTTPResponse` ### Usage Example ```python import urllib3 http = urllib3.PoolManager() response = http.request_encode_url('GET', 'https://httpbin.org/get', fields={'key': 'value'}) ``` ``` -------------------------------- ### Disable redirects but keep retries with redirect=False Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Specify redirect=False to disable redirects while keeping the retrying logic. The example shows a 302 response for a redirect. ```python resp = urllib3.request( "GET", "https://httpbin.org/redirect/1", redirect=False ) print(resp.status) # 302 ``` -------------------------------- ### HTTPSConnection.connect() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connection.html Connects to the host and port specified in the constructor. This method is called internally to establish the connection. ```APIDOC ## HTTPSConnection.connect() ### Description Connect to the host and port specified in __init__. ### Method connect() ### Parameters None ### Return Type None ``` -------------------------------- ### Catch urllib3 NewConnectionError Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Wraps lower-level exceptions. Attempts a GET request to a non-existent domain with retries disabled, catching urllib3.exceptions.NewConnectionError and printing 'Connection failed.' ```python import urllib3 try: urllib3.request("GET","https://nx.example.com", retries=False) except urllib3.exceptions.NewConnectionError: print("Connection failed.") # Connection failed. ``` -------------------------------- ### Timeout.connect_timeout property Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Gets the value to use when setting a connection timeout. This will be a positive float or integer, the value None (never timeout), or the default system timeout. ```APIDOC ## Timeout.connect_timeout property ### Description Gets the value to use when setting a connection timeout. This will be a positive float or integer, the value None (never timeout), or the default system timeout. ### Property connect_timeout ### Returns Connect timeout. ### Return Type int, float, Timeout.DEFAULT_TIMEOUT or None ``` -------------------------------- ### Parse URLs with urllib3.util.parse_url Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Shows how to use urllib3.util.parse_url to parse different URL formats, including full URLs, host:port, and path with query. The output comments show the resulting Url objects. ```python import urllib3 print( urllib3.util.parse_url('http://google.com/mail/')) # Url(scheme='http', host='google.com', port=None, path='/mail/', ...) print( urllib3.util.parse_url('google.com:80')) # Url(scheme=None, host='google.com', port=80, path=None, ...) print( urllib3.util.parse_url('/foo?bar')) # Url(scheme=None, host=None, port=None, path='/foo', query='bar', ...) ``` -------------------------------- ### urllib3.exceptions.NotOpenSSLWarning Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.exceptions.html Warning raised when using an unsupported SSL library. ```APIDOC ## urllib3.exceptions.NotOpenSSLWarning ### Description Warning raised when using an unsupported SSL library. ### Bases SecurityWarning ``` -------------------------------- ### Using ProxyManager to tunnel requests through an HTTP proxy Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Creates a ProxyManager pointing to an HTTP proxy and sends a GET request to https://google.com/. The usage is the same as PoolManager. ```python import urllib3 proxy = urllib3.ProxyManager("https://localhost:3128/") proxy.request("GET", "https://google.com/") ``` -------------------------------- ### Import urllib3 module Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Import the urllib3 module to use its functionality. ```python import urllib3 ```