### Basic urllib3 request example Source: https://urllib3.readthedocs.io/en/stable/index.html Shows how to make a simple GET request using urllib3.request and access the response status and data. ```python >>> import urllib3 >>> resp = urllib3.request("GET", "https://httpbin.org/robots.txt") >>> resp.status 200 >>> resp.data b"User-agent: *\nDisallow: /deny\n" ``` -------------------------------- ### Install urllib3 from source Source: https://urllib3.readthedocs.io/en/stable/index.html Commands to clone the urllib3 repository from GitHub and install it locally. ```bash $ git clone https://github.com/urllib3/urllib3.git $ cd urllib3 $ pip install . ``` -------------------------------- ### Basic urllib3 request in Pyodide Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/emscripten.html Shows how to make an HTTP request using urllib3 in a Pyodide environment. The response status, headers, and JSON body are printed. This example is from the 'Getting started' section. ```python import urllib3 resp = urllib3.request("GET", "https://httpbin.org/anything") print(resp.status) # 200 print(resp.headers) # HTTPHeaderDict(...) print(resp.json()) # {"headers": {"Accept": "*/*", ...}, ...} ``` -------------------------------- ### Using requests with urllib3 in Pyodide Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/emscripten.html Shows how to use the requests library, which is built on urllib3, in a Pyodide environment. The response status code and headers are printed. This example is from the 'Getting started' section. ```python import requests resp = requests.request("GET", "https://httpbin.org/anything") print(resp.status_code) # 200 print(resp.headers) ``` -------------------------------- ### Install nox for development environment Source: https://urllib3.readthedocs.io/en/stable/contributing.html Install nox with pip to set up the development environment. This is the only required tool for contributing. ```bash $ python -m pip install --user --upgrade nox ``` -------------------------------- ### Install urllib3 with SOCKS support Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html To use SOCKS proxies, install PySocks or install urllib3 with the socks extra. ```bash python -m pip install urllib3[socks] ``` -------------------------------- ### make_headers example Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Shows how to use urllib3.util.make_headers to generate HTTP headers. The example prints the resulting headers for keep_alive and user_agent, and for accept_encoding. ```python import urllib3 print(urllib3.util.make_headers(keep_alive=True, user_agent="Batman/1.0")) # {'connection': 'keep-alive', 'user-agent': 'Batman/1.0'} print(urllib3.util.make_headers(accept_encoding=True)) # {'accept-encoding': 'gzip,deflate'} ``` -------------------------------- ### Install urllib3 with pip Source: https://urllib3.readthedocs.io/en/stable/index.html Command to install urllib3 using pip. ```bash $ python -m pip install urllib3 ``` -------------------------------- ### Install pyOpenSSL dependencies Source: https://urllib3.readthedocs.io/en/stable/reference/contrib/pyopenssl.html Install the required packages for the pyOpenSSL backend: pyOpenSSL, cryptography, and idna. Use this command before using the module. ```bash $ python -m pip install pyopenssl cryptography idna ``` -------------------------------- ### Install urllib3 with Brotli extra Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Install the Brotli package via the urllib3[brotli] extra to enable Brotli compression support. ```bash $ python -m pip install urllib3[brotli] ``` -------------------------------- ### Using urllib3.request() to send a GET request Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Shows the new top-level urllib3.request() function, which uses a global poolmanager instance. The example sends a GET request to https://example.com and prints the response status (200). ```python >>> import urllib3 >>> resp = urllib3.request("GET", "https://example.com") >>> resp.status 200 ``` -------------------------------- ### ProxyManager example: sharing pools for HTTP and HTTPS requests Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html This example creates a ProxyManager pointing to a local HTTPS proxy and makes requests to different hosts. It shows that plain HTTP requests share a single connection pool, while each HTTPS target gets its own pool, as indicated by the printed pool counts. ```python import urllib3 proxy = urllib3.ProxyManager("https://localhost:3128/") resp1 = proxy.request("GET", "http://google.com/") resp2 = proxy.request("GET", "http://httpbin.org/") # One pool was shared by both plain HTTP requests. print(len(proxy.pools)) # 1 resp3 = proxy.request("GET", "https://httpbin.org/") resp4 = proxy.request("GET", "https://twitter.com/") # A separate pool was added for each HTTPS target. print(len(proxy.pools)) # 3 ``` -------------------------------- ### Create a connection pool from a URL and make a GET request Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html Shows how to use connection_from_url() to create a ConnectionPool instance from a URL and then call request('GET', '/') on it. The URL must include the scheme; port is optional. Additional keyword arguments can be passed to the pool constructor. ```python >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') ``` -------------------------------- ### Timeout.start_connect() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Starts the timeout clock, used during a connect() attempt. Raises TimeoutStateError if the timer has already been started. ```APIDOC ## Timeout.start_connect() ### Description Starts the timeout clock, used during a connect() attempt. Raises TimeoutStateError if the timer has already been started. ### Method GET ### Endpoint urllib3.util.Timeout.start_connect ### Parameters None ### Response #### Success Response (200) - **return** (float) - The start time. #### Response Example ``` 0.0 ``` ### Errors - **urllib3.exceptions.TimeoutStateError** - If you attempt to start a timer that has been started already. ``` -------------------------------- ### Install certifi Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Installs the certifi package, which provides Mozilla's root certificate bundle for certificate verification. ```bash $ python -m pip install certifi ``` -------------------------------- ### Install urllib3 v2.x locally Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Upgrade urllib3 to version 2.x in your local environment to test your package against it. ```bash $ python -m pip install -U 'urllib3>=2' ``` -------------------------------- ### Make a GET request with PoolManager Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Sends a GET request to httpbin.org/robots.txt and prints the response data. ```python import urllib3 # Creating a PoolManager instance for sending requests. http = urllib3.PoolManager() # Sending a GET request and getting back response as HTTPResponse object. resp = http.request("GET", "https://httpbin.org/robots.txt") # Print the returned data. print(resp.data) # b"User-agent: *\nDisallow: /deny\n" ``` -------------------------------- ### Install urllib3 with Zstandard extra Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Install the urllib3[zstd] extra to enable Zstandard support on Python 3.13 and earlier. On Python 3.14+, Zstandard support is included in the standard library. Requires zstandard package v0.18.0 or later. ```bash # This is only necessary on Python 3.13 and earlier. # Otherwise zstandard support is included in the Python standard library. $ python -m pip install urllib3[zstd] ``` -------------------------------- ### Example output of getproxies() with problematic HTTPS proxy Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html This output shows a system-level proxy configuration where the HTTPS proxy uses https://, which is the problem. ```json { "http": "http://127.0.0.1:8888", "https": "https://127.0.0.1:8888" # <--- This setting is the problem! } ``` -------------------------------- ### Example FutureWarning message Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Shows the format of a deprecation warning for the 'ssl_version' option, which is deprecated in favor of 'ssl_minimum_version'. ```text FutureWarning: 'ssl_version' option is deprecated and will be removed in urllib3 v3.0. Instead use 'ssl_minimum_version' ``` -------------------------------- ### Run specific test sessions with nox Source: https://urllib3.readthedocs.io/en/stable/contributing.html Runs the test-3.12 and test-pypy3.11 sessions. Nox creates virtualenvs if needed, installs dependencies, and runs commands in order. ```bash $ nox --reuse-existing-virtualenvs --sessions test-3.12 test-pypy3.11 [ Nox will create virtualenv if needed, install the specified dependencies, and run the commands in order.] ``` -------------------------------- ### Timeout.get_connect_duration() Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Gets the time elapsed since the call to start_connect(). Raises TimeoutStateError if the timer hasn't been started. ```APIDOC ## Timeout.get_connect_duration() ### Description Gets the time elapsed since the call to start_connect(). ### Method get_connect_duration() ### Returns Elapsed time in seconds. ### Return Type float ### Raises - **urllib3.exceptions.TimeoutStateError** - if you attempt to get duration for a timer that hasn't been started. ``` -------------------------------- ### Build documentation locally with nox -rs docs Source: https://urllib3.readthedocs.io/en/stable/contributing.html Builds the documentation locally using the nox tool. Run this command from the project root. ```bash $ nox -rs docs ``` -------------------------------- ### Pass query parameters via fields for GET/HEAD/DELETE Source: https://urllib3.readthedocs.io/en/stable/user-guide.html For `GET`, `HEAD`, and `DELETE` requests, pass query parameters as a dictionary in the `fields` argument to `urllib3.request()`. The example sends a GET request and prints the `args` from the response. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/get", fields={"arg": "value"} ) print(resp.json()["args"]) # {"arg": "value"} ``` -------------------------------- ### Run an HTTP proxy locally with python -m dummyserver.proxy Source: https://urllib3.readthedocs.io/en/stable/contributing.html Starts a local HTTP proxy server using the dummyserver module. Useful when developing proxy-related features. ```bash $ python -m dummyserver.proxy ``` -------------------------------- ### Read Set-Cookie header from response Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Cookies provided by the server are stored in the `Set-Cookie` header. The example sends a GET request with `redirect=False` and prints the `Set-Cookie` header. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/cookies/set/session/f3efe9db", redirect=False ) print(resp.headers["Set-Cookie"]) # session=f3efe9db; Path=/ ``` -------------------------------- ### Send request with custom headers using headers argument Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Specify headers as a dictionary in the `headers` argument to `urllib3.request()`. The example sends a GET request to httpbin.org/headers and prints the response headers. ```python import urllib3 resp = urllib3.request( "GET", "https://httpbin.org/headers", headers={ "X-Something": "value" } ) print(resp.json()["headers"]) # {"X-Something": "value", ...} ``` -------------------------------- ### Create multi-valued headers with HTTPHeaderDict Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Use `urllib3.HTTPHeaderDict` to add multiple values for the same header. The example adds two `Accept` headers and sends a GET request, resulting in a comma-separated header value. ```python import urllib3 # Create an HTTPHeaderDict and add headers headers = urllib3.HTTPHeaderDict() headers.add("Accept", "application/json") headers.add("Accept", "text/plain") # Make the request using the headers resp = urllib3.request( "GET", "https://httpbin.org/headers", headers=headers ) print(resp.json()["headers"]) # {"Accept": "application/json, text/plain", ...} ``` -------------------------------- ### Check and fix proxy environment variables in bash Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html Inspect HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY environment variables. Ensure they start with http://, not https://. The example shows a problematic HTTPS_PROXY and how to fix it in the current session and persist it in the shell profile. ```bash # Check your existing environment variables in bash $ env | grep "_PROXY" HTTP_PROXY=http://127.0.0.1:8888 HTTPS_PROXY=https://127.0.0.1:8888 # <--- This setting is the problem! # Make the fix in your current session and test your script $ export HTTPS_PROXY="http://127.0.0.1:8888" $ python test-proxy.py # This should now pass. # Persist your change in your shell 'profile' (~/.bashrc, ~/.profile, ~/.bash_profile, etc) # You may need to logout and log back in to ensure this works across all programs. $ vim ~/.bashrc ``` -------------------------------- ### Run an HTTPS proxy locally with python -m dummyserver.https_proxy Source: https://urllib3.readthedocs.io/en/stable/contributing.html Starts a local HTTPS proxy server using the dummyserver module. Useful when developing proxy-related features that require HTTPS. ```bash $ python -m dummyserver.https_proxy ``` -------------------------------- ### Sending JSON in a request and deserializing a JSON response Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Demonstrates the new json= parameter for requests, which automatically adds 'Content-Type: application/json' by default, and the HTTPResponse.json() method for deserializing response bodies. The example shows a POST request with a JSON body and a GET request whose response is parsed with resp.json(). ```python import urllib3 # Send a request with a JSON body. # This adds 'Content-Type: application/json' by default. resp = urllib3.request( "POST", "https://example.api.com", json={"key": "value"} ) # Receive a JSON body in the response. resp = urllib3.request("GET", "https://xkcd.com/2347/info.0.json") # There's always an XKCD... resp.json() { "num": 2347, "img": "https://imgs.xkcd.com/comics/dependency.png", "title": "Dependency", ... } ``` -------------------------------- ### Example: parse_url and Url.url round-trip Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html Shows how to parse a URL with urllib3.util.parse_url and convert it back to a string with the Url.url property. The output demonstrates that the round-trip preserves the URL, and that constructing a Url with explicit components yields the expected string. ```python import urllib3 U = urllib3.util.parse_url("https://google.com/mail/") print(U.url) # "https://google.com/mail/" print( urllib3.util.Url("https", "username:password", "host.com", 80, "/path", "query", "fragment" ).url ) # "https://username:password@host.com:80/path?query#fragment" ``` -------------------------------- ### HTTPConnection.__init__ Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connection.html Constructor for HTTPConnection. Accepts host, port, and various keyword arguments to configure the connection, such as timeout, source_address, blocksize, socket_options, proxy, and proxy_config. ```APIDOC ## HTTPConnection.__init__ ### Description Initializes a new HTTPConnection instance. Provides a backwards-compatibility layer between older and newer Pythons. ### Method Constructor ### Signature `HTTPConnection(host, port=None, *, timeout=_TYPE_DEFAULT.token, source_address=None, blocksize=16384, socket_options=[(6, 1, 1)], proxy=None, proxy_config=None)` ### Parameters - **host** (str) - Required - The host to connect to. - **port** (int | None) - Optional - The port to connect to. Defaults to None. - **timeout** (_TYPE_TIMEOUT) - Optional - Timeout for the connection. - **source_address** (tuple[str, int] | None) - Optional - Source address for the connection. - **blocksize** (int) - Optional - Block size for the connection. Defaults to 16384. - **socket_options** (list[tuple[int, int, int | bytes]] | None) - Optional - Socket options to set on the underlying socket. If not specified, defaults are loaded from `HTTPConnection.default_socket_options`. - **proxy** (Url | None) - Optional - Proxy URL. - **proxy_config** (ProxyConfig | None) - Optional - Proxy configuration. ### Notes - `socket_options` can be used to enable TCP Keep Alive by adding to the defaults, or disable defaults by passing an empty list. ### Example ```python HTTPConnection.default_socket_options + [ (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1), ] ``` ``` -------------------------------- ### connection_from_url(url, **kw) Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.connectionpool.html Given a url, return a ConnectionPool instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating a ConnectionPool instance. ```APIDOC ## connection_from_url(url, **kw) ### Description Given a url, return a ConnectionPool instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating a ConnectionPool instance. ### Method connection_from_url(url, **kw) ### Parameters - **url** (str) - Required - Absolute URL string that must include the scheme. Port is optional. - **\*\*kw** (Any) - Optional - Passes additional parameters to the constructor of the appropriate ConnectionPool. Useful for specifying things like timeout, maxsize, headers, etc. ### Return Type HTTPConnectionPool ### Example ``` >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') ``` ``` -------------------------------- ### Create PoolManager and make requests Source: https://urllib3.readthedocs.io/en/stable/reference/urllib3.poolmanager.html Shows how to instantiate urllib3.PoolManager with num_pools=2, make three GET requests, and print the number of pools. The output is 2 because the pool manager caches up to num_pools pools before discarding the least recently used. ```python import urllib3 http = urllib3.PoolManager(num_pools=2) resp1 = http.request("GET", "https://google.com/") resp2 = http.request("GET", "https://google.com/mail") resp3 = http.request("GET", "https://yahoo.com/") print(len(http.pools)) # 2 ``` -------------------------------- ### Check system-level proxy settings with urllib.request.getproxies() Source: https://urllib3.readthedocs.io/en/stable/advanced-usage.html On Windows or macOS, proxy settings may be set at the system level. Run this command to see the current proxy configuration. ```python $ python -c 'import urllib.request; print(urllib.request.getproxies())' ``` -------------------------------- ### PoolManager.connection_from_url() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a URL. ```APIDOC ## PoolManager.connection_from_url() ### Description Gets a connection from a URL. ### Method Method call ### Endpoint PoolManager.connection_from_url() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_url(url) ``` ### Response Returns a connection object. ``` -------------------------------- ### Update install_requires to support urllib3 1.26 and 2.x Source: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html Set your package's dependency specifier to allow both urllib3 1.26.x and 2.x. Shown for setup.py (setuptools) and pyproject.toml (hatch). ```python # setup.py (setuptools) setup( ... install_requires=["urllib3>=1.26,<3"] ) ``` ```toml # pyproject.toml (hatch) [project] dependencies = [ "urllib3>=1.26,<3" ] ``` -------------------------------- ### PoolManager.connection_from_pool_key() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a pool key. ```APIDOC ## PoolManager.connection_from_pool_key() ### Description Gets a connection from a pool key. ### Method Method call ### Endpoint PoolManager.connection_from_pool_key() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_pool_key(pool_key) ``` ### Response Returns a connection object. ``` -------------------------------- ### PoolManager.connection_from_host() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a host and port. ```APIDOC ## PoolManager.connection_from_host() ### Description Gets a connection from a host and port. ### Method Method call ### Endpoint PoolManager.connection_from_host() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_host(host, port, scheme) ``` ### Response Returns a connection object. ``` -------------------------------- ### urllib3.util.create_urllib3_context() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Creates an SSLContext with urllib3's default settings. ```APIDOC ## create_urllib3_context() ### Description Creates an SSLContext with urllib3's default settings. ### Parameters None ### Returns SSLContext - The created SSL context. ``` -------------------------------- ### PoolManager.connection_from_context() Source: https://urllib3.readthedocs.io/en/stable/reference/index.html Gets a connection from a context object. ```APIDOC ## PoolManager.connection_from_context() ### Description Gets a connection from a context object. ### Method Method call ### Endpoint PoolManager.connection_from_context() ### Parameters Refer to the detailed documentation for parameter definitions. ### Request Example ```python conn = manager.connection_from_context(context) ``` ### Response Returns a connection object. ``` -------------------------------- ### Access response status, data, and headers Source: https://urllib3.readthedocs.io/en/stable/user-guide.html Shows how to make a request and print the status, data, and headers of the HTTPResponse object. The output demonstrates the expected format for each attribute. ```python import urllib3 # Making the request (The request function returns HTTPResponse object) resp = urllib3.request("GET", "https://httpbin.org/ip") print(resp.status) # 200 print(resp.data) # b"{\n "origin": "104.232.115.37"\n}\n" print(resp.headers) # HTTPHeaderDict({"Content-Length": "32", ...}) ```