### Setup and Run Tests with httpbin Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Install httpbin and gunicorn, start the httpbin server, and then run Hackney tests. ```bash pip3 install httpbin gunicorn gunicorn -b 127.0.0.1:8000 httpbin:app & rebar3 eunit ``` -------------------------------- ### Quick Start WebSocket Connection Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md A basic example demonstrating how to connect, send a text message, receive a response, and close a WebSocket connection. ```erlang {ok, Conn} = hackney:ws_connect(<<"wss://echo.websocket.org">>), ok = hackney:ws_send(Conn, {text, <<"Hello!">>}), {ok, {text, <<"Hello!">>}} = hackney:ws_recv(Conn), hackney:ws_close(Conn). ``` -------------------------------- ### Install Middleware Per-Request Source: https://github.com/benoitc/hackney/blob/master/guides/middleware.md Demonstrates how to install a list of middleware for a specific HTTP request. ```erlang hackney:request(get, URL, [], <<>>, [{middleware, [Mw1, Mw2]}]). ``` -------------------------------- ### Install Erlang on Ubuntu/Debian Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Install Erlang and build essentials on Ubuntu or Debian-based systems using apt-get. ```bash sudo apt-get install erlang build-essential ``` -------------------------------- ### Start Hackney and Make a Simple GET Request Source: https://github.com/benoitc/hackney/blob/master/README.md Ensures Hackney application is started and performs a basic GET request. The response body is returned directly. ```erlang %% Start hackney application:ensure_all_started(hackney). %% Simple GET - the body is returned directly {ok, 200, _Headers, Body} = hackney:get(<<"https://httpbin.org/get">>). ``` -------------------------------- ### Install Erlang and rebar3 on FreeBSD Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Install specific Erlang runtime versions and rebar3 on FreeBSD using pkg. ```bash pkg install erlang-runtime28 rebar3 ``` -------------------------------- ### Install Middleware Globally Source: https://github.com/benoitc/hackney/blob/master/guides/middleware.md Shows how to set a global fallback list of middleware for all Hackney requests. ```erlang application:set_env(hackney, middleware, [Mw1, Mw2]). ``` -------------------------------- ### Install Erlang on macOS Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Use Homebrew to install Erlang on macOS. Ensure you have the latest version. ```bash brew install erlang ``` -------------------------------- ### Complete Manual Connection Management Example Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Demonstrates reusing a single connection for multiple API calls, including optional protocol checking and proper connection cleanup. ```erlang %% Reuse a connection for multiple API calls {ok, Conn} = hackney:connect(hackney_ssl, "api.example.com", 443, []), %% Check protocol (optional) case hackney_conn:get_protocol(Conn) of http2 -> io:format("Using HTTP/2 multiplexing~n"); http1 -> io:format("Using HTTP/1.1 keep-alive~n") end, %% Make requests {ok, 200, _, Token} = hackney:send_request(Conn, {post, <<"/auth">>, [], Credentials}), {ok, 200, _, Users} = hackney:send_request(Conn, {get, <<"/users">>, AuthHeaders, <<>>}), {ok, 200, _, Data} = hackney:send_request(Conn, {get, <<"/data">>, AuthHeaders, <<>>}), %% Clean up hackney:close(Conn). ``` -------------------------------- ### Connection Pooling Configuration Source: https://github.com/benoitc/hackney/blob/master/GETTING_STARTED.md Demonstrates how to start a named connection pool with a maximum number of connections and how to use it for requests. Also shows how to explicitly disable pooling. ```erlang %% Create a pool hackney_pool:start_pool(api_pool, [{max_connections, 50}]). %% Use the pool hackney:get(URL, [], <<>>, [{pool, api_pool}]). %% Disable pooling hackney:get(URL, [], <<>>, [{pool, false}]). ``` -------------------------------- ### Start Hackney OTP Application Source: https://github.com/benoitc/hackney/blob/master/GETTING_STARTED.md Ensure the hackney OTP application is started before making any HTTP requests. ```erlang application:ensure_all_started(hackney). ``` -------------------------------- ### WebSocket Quick Start for Comparison Source: https://github.com/benoitc/hackney/blob/master/guides/webtransport_guide.md Connect to a WebSocket server, send a message, receive a response, and close the connection. This is shown for comparison with the WebTransport API. ```erlang {ok, Conn} = hackney:ws_connect(<<"wss://example.com/socket">>), ok = hackney:ws_send(Conn, {binary, <<"Hello!">>}), {ok, {binary, <<"Hello!">>}} = hackney:ws_recv(Conn), hackney:ws_close(Conn). ``` -------------------------------- ### Erlang HTTP Request: 1.x vs 2.x/3.x Simple Request Body Handling Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md A simplified example of migrating a basic HTTP GET request. It highlights that in 2.x/3.x, the response body is directly included in the result tuple, eliminating the need for a separate `hackney:body/1` call. ```erlang %% 1.x {ok, 200, Headers, Ref} = hackney:get(URL), {ok, Body} = hackney:body(Ref). %% 2.x/3.x - simpler! {ok, 200, Headers, Body} = hackney:get(URL). ``` -------------------------------- ### Elixir HTTP/2 Request Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md An example in Elixir demonstrating how to make an HTTP/2 request and check the protocol used via response headers. ```elixir # Start hackney Application.ensure_all_started(:hackney) # HTTP/2 request - body is returned directly {:ok, status, headers, body} = :hackney.get("https://nghttp2.org/") # Check protocol via header case case headers do [{"date", _} | _] -> IO.puts("HTTP/2") [{"Date", _} | _] -> IO.puts("HTTP/1.1") end. ``` -------------------------------- ### Named Connection Pool Configuration and Usage Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Starts a named connection pool with custom settings like max connections and timeout, then uses it for an HTTP GET request. ```erlang hackney_pool:start_pool(my_api, [ {max_connections, 100}, {timeout, 150000} ]), hackney:get(URL, [], <<>>, [{pool, my_api}]). ``` -------------------------------- ### Conventional Commits Type Examples Source: https://github.com/benoitc/hackney/blob/master/CONTRIBUTING.md Examples of different commit types used in PR titles. ```markdown feat: add HTTP/3 connection pooling ``` ```markdown fix: handle timeout in async responses ``` ```markdown docs: update WebSocket guide ``` ```markdown refactor: simplify connection state machine ``` ```markdown perf: reduce memory allocation in header parsing ``` ```markdown security: update SSL certificate bundle ``` -------------------------------- ### WebTransport Quick Start Source: https://github.com/benoitc/hackney/blob/master/guides/webtransport_guide.md Connect to a WebTransport server, send a message, receive a response, and close the connection. This demonstrates the basic API mirroring WebSockets. ```erlang {ok, Conn} = hackney:wt_connect(<<"https://example.com/wt">>), ok = hackney:wt_send(Conn, {binary, <<"Hello!">>}), {ok, {binary, <<"Hello!"_>>}} = hackney:wt_recv(Conn), hackney:wt_close(Conn). ``` -------------------------------- ### Configure HTTP Proxy with Hackney Source: https://github.com/benoitc/hackney/blob/master/README.md Set up an HTTP proxy for requests. This example shows how to specify the proxy URL directly and also mentions that environment variables are automatically used. ```erlang %% HTTP proxy hackney:get(URL, [], <<>>, [{proxy, <<"http://proxy:8080">>}]). %% With authentication hackney:get(URL, [], <<>>, [{proxy, <<"http://user:pass@proxy:8080">>}]). %% Environment variables work automatically %% HTTP_PROXY, HTTPS_PROXY, NO_PROXY ``` -------------------------------- ### Interactive Debugging in Docker Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Start an interactive bash shell within the 'hackney-test' Docker container for debugging. ```bash docker run --rm -it hackney-test bash ``` -------------------------------- ### Example Chat Client Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md A simple Erlang module implementing a chat client that connects to a WebSocket URL, sends messages, and receives/prints incoming messages in active mode. ```erlang -module(chat). -export([start/1, send/2]). start(URL) -> {ok, Conn} = hackney:ws_connect(URL, [{active, true}]), spawn(fun() -> loop(Conn) end), Conn. send(Conn, Msg) -> hackney:ws_send(Conn, {text, Msg}). loop(Conn) -> receive {hackney_ws, Conn, {text, Text}} -> io:format("~s~n", [Text]), loop(Conn); {hackney_ws, Conn, closed} -> ok end. ``` -------------------------------- ### Simple GET Request Source: https://github.com/benoitc/hackney/blob/master/GETTING_STARTED.md Make a simple GET request and receive the response body directly in the tuple. The status code, headers, and body are returned upon success. ```erlang {ok, 200, Headers, Body} = hackney:get(<<"https://httpbin.org/get">>). ``` -------------------------------- ### HTTP/2 Error Handling Example Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md Shows how to handle various HTTP/2 specific errors like GOAWAY, stream resets, and connection closures when making requests. ```erlang case hackney:get(URL) of {ok, Status, Headers, Body} -> ok; {error, {goaway, ErrorCode}} -> %% Peer sent GOAWAY io:format("HTTP/2 GOAWAY: ~p~n", [ErrorCode]); {error, {stream_error, ErrorCode}} -> %% Peer sent RST_STREAM for this request io:format("HTTP/2 stream reset: ~p~n", [ErrorCode]); {error, {closed, _Reason}} -> %% Connection closed ok; {error, Reason} -> io:format("Error: ~p~n", [Reason]) end. ``` -------------------------------- ### Hackney Owner Monitoring Example Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Demonstrates how a connection process monitors its owner and terminates if the owner crashes, preventing socket leaks. ```erlang %% When connection is checked out MonitorRef = monitor(process, Owner), %% If owner dies {'DOWN', MonitorRef, process, Owner, _} -> terminate ``` -------------------------------- ### Handle Async Responses Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Initiates an asynchronous GET request and receives response messages in stages: status, headers, and the body (or done). ```erlang {ok, Ref} = hackney:get(URL, [], <<>>, [async]), receive {hackney_response, Ref, {status, Status, _}} -> ok end, receive {hackney_response, Ref, {headers, Headers}} -> ok end, receive {hackney_response, Ref, done} -> ok; {hackney_response, Ref, Bin} -> ok end. ``` -------------------------------- ### Timeout Capping Example (1.x vs 2.x) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Illustrates the timeout capping behavior in Hackney. In 1.x, any timeout value could be set, while in 2.x, the keepalive timeout is capped at 2000ms, regardless of the value provided. ```erlang %% 1.x - any timeout value hackney_pool:start_pool(p, [{timeout, 300000}]). %% 5 minutes %% 2.x - capped at 2000ms hackney_pool:start_pool(p, [{timeout, 300000}]). %% Becomes 2000ms ``` -------------------------------- ### WebTransport Echo Client Source: https://github.com/benoitc/hackney/blob/master/guides/webtransport_guide.md An example of a WebTransport client that connects to a URL, sends a message, receives a reply, and then closes the connection. ```erlang -module(wt_echo). -export([run/1]). run(URL) -> {ok, Conn} = hackney:wt_connect(URL), ok = hackney:wt_send(Conn, {binary, <<"hello">>}), {ok, {binary, Reply}} = hackney:wt_recv(Conn, 5000), io:format("echo: ~s~n", [Reply]), hackney:wt_close(Conn). ``` -------------------------------- ### Default Connection Pool Usage Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Makes an HTTP GET request using the default connection pool. This is the simplest way to make a request. ```erlang hackney:get(URL). %% Uses default pool ``` -------------------------------- ### Send Multiple Requests on a Manual Connection Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Sends a sequence of HTTP requests (GET, POST) using a pre-established manual connection. This allows for request multiplexing on HTTP/2 or keep-alive on HTTP/1.1. ```erlang %% Send multiple requests on the same connection {ok, 200, Headers1, Body1} = hackney:send_request(ConnPid, {get, <<"/api/users">>, [], <<>>}). {ok, 201, Headers2, Body2} = hackney:send_request(ConnPid, {post, <<"/api/users">>, [{<< ``` ```erlang content-type">>, << ``` ```erlang application/json">>}]), <<"{\"name\": \"Alice\"}">>}). {ok, 200, Headers3, Body3} = hackney:send_request(ConnPid, {get, <<"/api/users/1">>, [], <<>>}). ``` -------------------------------- ### Async Once Response Handling Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Performs an asynchronous GET request expecting a single response message. Use `stream_next` to request subsequent messages if available. ```erlang {ok, Ref} = hackney:get(URL, [], <<>>, [{async, once}]), receive {hackney_response, Ref, Msg} -> ok end, hackney:stream_next(Ref). %% Request next message ``` -------------------------------- ### Send Streaming Request Body (Basic) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Use this snippet to send a request body in chunks when the size is known. It involves starting the request with `stream`, sending chunks with `send_body`, finishing with `finish_send_body`, and then handling the response. ```erlang %% 1. Start request with body = stream {ok, ConnPid} = hackney:request(post, URL, Headers, stream, []), %% 2. Send body chunks (can be called multiple times) ok = hackney:send_body(ConnPid, <<"first chunk">>), ok = hackney:send_body(ConnPid, <<"second chunk">>), %% 3. Signal end of body ok = hackney:finish_send_body(ConnPid), %% 4. Get response headers {ok, Status, RespHeaders, ConnPid} = hackney:start_response(ConnPid), %% 5. Read response body {ok, RespBody} = hackney:body(ConnPid), %% 6. Close connection when done hackney:close(ConnPid). ``` -------------------------------- ### Event-Driven Architecture for QUIC Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Illustrates the event loop for managing QUIC connections in an Erlang owner process using the quic library. It shows the flow of events from connection creation to data processing and timer-based retries. ```Erlang ┌─────────────────────────────────────────────────────────────────┐ │ Owner Process (Erlang) │ │ │ │ 1. connect() → Creates QUIC connection via quic library │ │ │ │ 2. Receives {select, Resource, Ref, ready_input} │ │ └── Socket has data ready │ │ │ │ 3. Calls hackney_h3:process(ConnRef) │ │ └── Receives UDP packets │ │ └── Processes QUIC frames │ │ └── Triggers events (headers, data, etc.) │ │ └── Returns next timeout in ms │ │ │ │ 4. Receives {h3, ConnRef, Event} │ │ └── {connected, Info} │ │ └── {stream_headers, StreamId, Headers, Fin} │ │ └── {stream_data, StreamId, Data, Fin} │ │ └── {closed, Reason} │ │ │ │ 5. Schedules timer: erlang:send_after(TimeoutMs, self(), ...) │ │ └── Calls process() again when timer fires │ └─────────────────────────────────────────────────────────────────┘ ``` -------------------------------- ### Build Docker Image for Testing Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Build a Docker image named 'hackney-test' using the provided Dockerfile.test. ```bash docker build -f Dockerfile.test -t hackney-test . ``` -------------------------------- ### Get Negotiated Protocol from Connection Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Retrieves the communication protocol (HTTP/1, HTTP/2, or HTTP/3) negotiated for an active connection. ```erlang %% See which protocol was negotiated Protocol = hackney_conn:get_protocol(ConnPid). %% http1 | http2 | http3 ``` -------------------------------- ### Connect to a Host for Manual Connection Management Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Establishes a manual connection to a host and port, returning a connection PID for subsequent requests. Supports both SSL and plain connections. ```erlang %% Connect to a host (returns a connection PID) {ok, ConnPid} = hackney:connect(hackney_ssl, "example.com", 443, []). %% Or from a URL {ok, ConnPid} = hackney:connect(<<"https://example.com">>). ``` -------------------------------- ### Getting Negotiated Protocol from SSL Socket Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Retrieves the negotiated protocol from an SSL socket after the handshake, returning http2 or http1. ```erlang %% After connection get_negotiated_protocol(SslSocket) -> case ssl:negotiated_protocol(SslSocket) of {ok, << ``` -------------------------------- ### Simple WebSocket Connection Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md Establishes a basic WebSocket connection to a specified URL. ```erlang {ok, Conn} = hackney:ws_connect(<<"wss://example.com/socket">>). ``` -------------------------------- ### Benefits of Pure Erlang Implementation Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Compares the advantages of a pure Erlang implementation of QUIC over a NIF-based approach, highlighting portability, build complexity, debugging, crash isolation, and hot code loading. ```Markdown | Aspect | Pure Erlang | NIF-based | |--------|-------------|-----------| | Portability | All platforms | Requires C compiler | | Build complexity | rebar3 compile | CMake + dependencies | | Debugging | Erlang tools | GDB/LLDB | | Crash isolation | Process crash | Possible VM crash | | Hot code loading | Supported | Limited | ``` -------------------------------- ### QUIC Connection Lifecycle Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Visualizes the interaction between the owner process and the quic library during the lifecycle of a QUIC connection, from initiation to closure. ```Erlang Owner Process quic library │ │ │ hackney_h3:connect(...) │ ├────────────────────────────────────►│ Create UDP socket │ │ Generate TLS keys │ │ Send Initial packet │◄────────────────────────────────────┤ {ok, ConnRef} │ │ │ {select, _, _, ready_input} │ │◄────────────────────────────────────┤ UDP packet received │ │ │ hackney_h3:process(ConnRef) │ ├────────────────────────────────────►│ Process QUIC packets │ │ Continue handshake │ {h3, ConnRef, {connected, Info}} │ │◄────────────────────────────────────┤ │ │ │ ... (request/response cycle) ... │ │ │ │ hackney_h3:close(ConnRef, ...) │ ├────────────────────────────────────►│ Send CONNECTION_CLOSE │ {h3, ConnRef, {closed, normal}} │ │◄────────────────────────────────────┤ │ │ ``` -------------------------------- ### GET Request to Read Full Response Body Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Retrieves the entire response body directly. This is the default behavior for handling responses. ```erlang %% Body is returned directly {ok, 200, Headers, Body} = hackney:get(URL). ``` -------------------------------- ### Concurrent Requests with Multiplexing Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md Demonstrates how to take advantage of HTTP/2 multiplexing by making multiple requests concurrently using spawned processes. ```erlang Parent = self(), URLs = [<<"https://api.example.com/1">>, <<"https://api.example.com/2">>], Pids = [spawn_link(fun() -> Result = hackney:get(URL), Parent ! {self(), Result} end) || URL <- URLs], Results = [receive {Pid, R} -> R end || Pid <- Pids]. ``` -------------------------------- ### Get Load Regulation Stats Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Query the current number of concurrent connections to a specific host and port. This is managed by the load regulation system. ```erlang hackney_load_regulation:current(Host, Port). %% Returns: integer() - current concurrent connections to host ``` -------------------------------- ### Application Environment Configuration (1.x vs 2.x) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Compares the application environment configuration for Hackney between 1.x and 2.x. Version 2.x introduces new options like `prewarm_count` and caps the `timeout` value. ```erlang %% 1.x {hackney, [ {max_connections, 50}, {timeout, 150000} %% Could be any value ]}. %% 2.x {hackney, [ {max_connections, 50}, {timeout, 2000}, %% Capped at 2000ms {prewarm_count, 4} %% New option ]}. ``` -------------------------------- ### Pool Configuration (1.x vs 2.x) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Illustrates how to configure connection pools in Hackney. Version 1.x uses a global pool limit, while 2.x offers per-pool and per-host limits with additional options like `prewarm_count` and `checkout_timeout`. ```erlang %% 1.x - global pool limit hackney_pool:start_pool(mypool, [{max_connections, 50}]), hackney:get(URL, [], <<>>, [{pool, mypool}]). %% 2.x - same API works, but behavior differs: %% - max_connections is now per pool, not global limit %% - Add max_per_host for per-host limiting hackney_pool:start_pool(mypool, [ {max_connections, 100}, %% Pool capacity {prewarm_count, 4}, %% Warm connections per host {timeout, 2000} %% Keepalive timeout (max 2s) ]), hackney:get(URL, [], <<>>, [ {pool, mypool}, {max_per_host, 50}, %% Per-host limit {checkout_timeout, 5000} %% Wait time for slot ]). ``` -------------------------------- ### Get Per-Host Pool Statistics Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Fetch statistics for a specific host within a connection pool. This includes active, in-use, and free connections. ```erlang hackney_pool:host_stats(PoolName, Host, Port). %% Returns: %% [{active, N}, %% Currently in use (from load_regulation) %% {in_use, N}, %% Checked out from pool %% {free, N}] %% Available in pool ``` -------------------------------- ### Get Pool Statistics Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Retrieve statistics for a specified connection pool. Note that queue_count is always 0 as load regulation handles queuing. ```erlang hackney_pool:get_stats(PoolName). %% Returns: %% [{name, PoolName}, %% {max, MaxConnections}, %% {in_use_count, InUse}, %% {free_count, Free}, %% {queue_count, 0}] %% Always 0, load regulation handles queuing ``` -------------------------------- ### Initiating an HTTP/2 Request in hackney_conn Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Handles initiating an HTTP/2 request by getting a stream ID, tracking the caller, and sending HEADERS frames. ```erlang %% In hackney_conn.erl do_h2_request(From, Method, Path, Headers, Body, Data) -> %% 1. Get next stream ID from h2_machine {ok, StreamId, H2Machine1} = hackney_cow_http2_machine:init_stream(...), %% 2. Track caller for this stream Streams = maps:put(StreamId, {From, waiting_response}, Data#conn_data.h2_streams), %% 3. Send HEADERS frame (and DATA if body present) HeadersFrame = hackney_cow_http2:headers(StreamId, ...), Transport:send(Socket, HeadersFrame), %% 4. Return updated state (caller will receive reply when response arrives) {keep_state, Data#conn_data{h2_streams = Streams}}. ``` -------------------------------- ### Connect to an HTTP/3 Server Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Establishes a connection to an HTTP/3 server. The calling process becomes the connection owner and receives events. ```erlang {ok, ConnRef} = hackney_h3:connect(<<"cloudflare.com">>, 443, #{}, self()). receive {h3, ConnRef, {connected, _Info}} -> ok after 5000 -> error(connect_timeout) end. ``` -------------------------------- ### Verifying Connection Reuse Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md This snippet demonstrates how to verify that multiple connection attempts to the same host and port result in the same connection PID, indicating successful connection reuse. ```erlang {ok, Conn1} = hackney:connect(hackney_ssl, "nghttp2.org", 443, []). {ok, Conn2} = hackney:connect(hackney_ssl, "nghttp2.org", 443, []). {ok, Conn3} = hackney:connect(hackney_ssl, "nghttp2.org", 443, []). Conn1 =:= Conn2. %% true - same PID Conn2 =:= Conn3. %% true - same PID ``` -------------------------------- ### Connection Multiplexing with HTTP/3 Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Illustrates how multiple HTTP/3 requests can be multiplexed over a single QUIC connection, similar to HTTP/2. ```erlang %% All requests share ONE QUIC connection {ok, _, _, _} = hackney:get(<<"https://cloudflare.com/">>, [], <<>>, [{protocols, [http3]}]). {ok, _, _, _} = hackney:get(<<"https://cloudflare.com/cdn-cgi/trace">>, [], <<>>, [{protocols, [http3]}]). ``` -------------------------------- ### Async Response Handling Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md This snippet shows how to handle asynchronous responses. It uses `hackney:get` with the `async` option and a `receive` block to process the response. ```erlang {ok, Ref} = hackney:get(URL, [], <<>>, [async]), receive {hackney_response, Ref, {status, Status, _}} -> ok end. ``` -------------------------------- ### Support proxy environment settings Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.19.0 adds support for proxy environment settings, enhancing flexibility in configuring network connections. ```erlang feature: add support for proxy environment setting ``` -------------------------------- ### Connection Reuse for Performance Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md Illustrates the performance benefit of reusing HTTP/2 connections by using the default pool, compared to opening a new connection for each request. ```erlang %% Good: connections are reused [hackney:get(URL, [], <<>>, [{pool, default}]) || _ <- lists:seq(1, 100)]. %% Bad: new connection each time [hackney:get(URL, [], <<>>, [{pool, false}]) || _ <- lists:seq(1, 100)]. ``` -------------------------------- ### Hackney Request Telemetry Middleware Source: https://github.com/benoitc/hackney/blob/master/guides/middleware.md This middleware captures request start, stop, and exception events using the telemetry library. It requires the 'telemetry' library to be available. ```erlang Telemetry = fun(Req, Next) -> T0 = erlang:monotonic_time(), telemetry:execute([hackney, request, start], #{system_time => erlang:system_time()}, #{method => maps:get(method, Req)}), try Resp = Next(Req), telemetry:execute([hackney, request, stop], #{duration => erlang:monotonic_time() - T0}, #{result => Resp}), Resp catch Class:Reason:Stack -> telemetry:execute([hackney, request, exception], #{duration => erlang:monotonic_time() - T0}, #{kind => Class, reason => Reason, stacktrace => Stack}), erlang:raise(Class, Reason, Stack) end end. ``` -------------------------------- ### Build hackney Project Source: https://github.com/benoitc/hackney/blob/master/CONTRIBUTING.md Compile the hackney project using rebar3. ```shell rebar3 compile ``` -------------------------------- ### Configure Redirect Following Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Enables automatic following of HTTP redirects up to a specified maximum number of redirects. ```erlang {ok, 200, Headers, Body} = hackney:get(URL, [], <<>>, [ {follow_redirect, true}, {max_redirect, 5} ]). ``` -------------------------------- ### GET Request for Streaming Response Body (Async Mode) Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Handles response bodies incrementally in asynchronous mode, allowing processing of chunks as they arrive. This is memory-efficient for large responses. ```erlang {ok, Ref} = hackney:get(URL, [], <<>>, [async]), stream_loop(Ref). stream_loop(Ref) -> receive {hackney_response, Ref, {status, Status, _}} -> io:format("Status: ~p~n", [Status]), stream_loop(Ref); {hackney_response, Ref, {headers, Headers}} -> io:format("Headers: ~p~n", [Headers]), stream_loop(Ref); {hackney_response, Ref, done} -> ok; {hackney_response, Ref, Chunk} when is_binary(Chunk) -> process_chunk(Chunk), stream_loop(Ref) end. ``` -------------------------------- ### Monitoring Pool Statistics (1.x vs 2.x) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Demonstrates how to monitor pool statistics. Both versions use `hackney_pool:get_stats/1`, but 2.x also adds `hackney_pool:host_stats/3` for per-host statistics. ```erlang %% 1.x hackney_pool:get_stats(mypool). %% 2.x - same, plus per-host stats hackney_pool:get_stats(mypool). hackney_pool:host_stats(mypool, "api.example.com", 443). %% Returns: [{active, N}, {in_use, N}, {free, N}] ``` -------------------------------- ### GET Request with Automatic Response Decompression Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Automatically decompresses gzip and deflate encoded responses. It adds the 'Accept-Encoding' header and handles decompression based on the 'Content-Encoding' response header. ```erlang {ok, 200, Headers, Body} = hackney:get(URL, [], <<>>, [ {auto_decompress, true} ]). ``` -------------------------------- ### HTTP/2 Async Mode in Hackney 3.x Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Illustrates the consistent use of the async API for HTTP/1.1, HTTP/2, and HTTP/3 requests in Hackney 3.x. ```erlang %% Works for HTTP/1.1, HTTP/2, and HTTP/3 {ok, Ref} = hackney:get(<< ``` ```erlang https://nghttp2.org/>>, [], <<>>, [async]), receive {hackney_response, Ref, {status, Status, _}} -> io:format("Status: ~p~n", [Status]) end, receive {hackney_response, Ref, {headers, Headers}} -> io:format("Headers: ~p~n", [Headers]) end, receive {hackney_response, Ref, done} -> ok; {hackney_response, Ref, Chunk} -> io:format("Chunk: ~p~n", [Chunk]) end. ``` -------------------------------- ### Get Hackney Pool Statistics Source: https://github.com/benoitc/hackney/blob/master/guides/middleware.md This snippet shows how to retrieve statistics for a Hackney connection pool. Use this to monitor pool state like in-use count and free count. ```erlang Stats = hackney_pool:get_stats(default), %% #{name, max, in_use_count, free_count, queue_count} ``` -------------------------------- ### Automatic Connection Reuse for Multiplexing Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md Hackney automatically reuses HTTP/2 connections for multiple requests, enabling multiplexing. This example shows three requests sharing a single TCP connection. ```erlang %% All three requests share ONE TCP connection {ok, _, _, _} = hackney:get(<<"https://nghttp2.org/">>). {ok, _, _, _} = hackney:get(<<"https://nghttp2.org/blog/">>). {ok, _, _, _} = hackney:get(<<"https://nghttp2.org/documentation/">>). ``` -------------------------------- ### Run All Tests in Docker Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Execute all tests within the Docker environment by running the 'hackney-test' container. ```bash docker run --rm hackney-test ``` -------------------------------- ### Enable HTTP/3 with Fallback Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Configure Hackney to attempt HTTP/3 first, falling back to HTTP/2 and then HTTP/1.1 if necessary. ```erlang %% Try HTTP/3 first, fall back to HTTP/2, then HTTP/1.1 hackney:get(URL, [], <<>>, [{protocols, [http3, http2, http1]}]). ``` -------------------------------- ### Detect Negotiated Protocol on Connection Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Connect to a host and then retrieve the negotiated protocol (http3, http2, or http1) for that connection. ```erlang {ok, Conn} = hackney:connect(hackney_ssl, "cloudflare.com", 443, [{protocols, [http3]}]), Protocol = hackney_conn:get_protocol(Conn). %% http3 | http2 | http1 hackney:close(Conn). ``` -------------------------------- ### Run Specific Tests in Docker Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Run specific test modules within the Docker container by executing a bash command. ```bash docker run --rm hackney-test bash -c "rebar3 eunit --module=hackney_h3_low_level_tests" ``` -------------------------------- ### Configure Connection and Receive Timeouts Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Set custom timeouts for establishing a connection and receiving data. The connect_timeout is in milliseconds, and recv_timeout is also in milliseconds. ```erlang hackney:get(URL, [], <<>>, [ {connect_timeout, 5000}, {recv_timeout, 30000} ]). ``` -------------------------------- ### WebSocket Connection with Options Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md Connects to a WebSocket with custom options including connection timeout, receive timeout, authorization headers, and supported protocols. ```erlang {ok, Conn} = hackney:ws_connect(<<"wss://example.com/socket">>, [ {connect_timeout, 5000}, {recv_timeout, 30000}, {headers, [{<<"authorization">>, <<"Bearer token">>}]}, {protocols, [<<"graphql-ws">>}]} ]). ``` -------------------------------- ### Add application variable support for insecure_basic_auth Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.24.0 adds support for the `insecure_basic_auth` application variable, allowing configuration via `application:set_env/2`. ```erlang security: add application variable support for insecure_basic_auth ``` -------------------------------- ### High-Concurrency Pool Configuration (1.x vs 2.x) Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Shows how to manage high concurrency in Hackney pools. Version 1.x uses a single global limit, whereas 2.x provides better isolation by allowing per-host limits alongside the total pool limit. ```erlang %% 1.x - 1000 connections shared across all hosts hackney_pool:start_pool(bigpool, [{max_connections, 1000}]). %% 2.x - 100 connections per host (better isolation) hackney_pool:start_pool(bigpool, [{max_connections, 1000}]), hackney:get(URL, [], <<>>, [ {pool, bigpool}, {max_per_host, 100} %% Each host gets up to 100 ]). ``` -------------------------------- ### Erlang Hackney Pool: 1.x Global vs 2.x Per-Host Limits Source: https://github.com/benoitc/hackney/blob/master/guides/MIGRATION.md Demonstrates how to configure connection pools. Version 1.x used a single global pool limit, while 2.x introduced per-host limits, configurable via `max_per_host` option during requests. ```erlang %% 1.x - global pool limit hackney_pool:start_pool(mypool, [{max_connections, 100}]). %% 100 total %% 2.x - per-host limit (100 connections per host) hackney_pool:start_pool(mypool, [{max_connections, 100}]). %% Plus request option: hackney:get(URL, [], <<>>, [{pool, mypool}, {max_per_host, 100}]). ``` -------------------------------- ### Upload Files and Form Data with Multipart Source: https://github.com/benoitc/hackney/blob/master/README.md Construct and send a multipart request containing form fields, a file, and an image with custom content type. ```erlang Multipart = {multipart, [ {<<"field">>, <<"value">>}, {file, <<"/path/to/file.txt">>}, {file, <<"/path/to/image.png">>, <<"image.png">>, [{<<"content-type">>, <<"image/png">>}]} ]}, hackney:post(URL, [], Multipart). ``` -------------------------------- ### Clone Hackney Repository Source: https://github.com/benoitc/hackney/blob/master/DEVELOPMENT.md Clone the Hackney repository and navigate into the project directory. ```bash git clone https://github.com/benoitc/hackney.git cd hackney ``` -------------------------------- ### Use latest SSL certificate bundle Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.21.0 ensures the use of the latest SSL certificate bundle for enhanced security. ```erlang use latest SSL certificate bundle ``` -------------------------------- ### Specify Protocol Preference Order Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md You can specify a preferred order for protocols, such as preferring HTTP/1.1 and falling back to HTTP/2. ```erlang %% Prefer HTTP/1.1, fall back to HTTP/2 hackney:get(URL, [], <<>>, [{protocols, [http1, http2]}]). ``` -------------------------------- ### Configure Custom CA Certificate for SSL/TLS Source: https://github.com/benoitc/hackney/blob/master/guides/http_guide.md Provides a custom CA certificate file for verifying the server's SSL/TLS certificate. ```erlang hackney:get(URL, [], <<>>, [ {ssl_options, [{cacertfile, "/path/to/ca.crt"}]} ]). ``` -------------------------------- ### Handle Different WebSocket Frame Types Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md A case statement to handle various incoming WebSocket frame types, including text, binary, ping, pong, and connection closure. ```erlang case hackney:ws_recv(Conn) of {ok, {text, Text}} -> handle_text(Text); {ok, {binary, Data}} -> handle_binary(Data); {ok, ping} -> ok; %% Auto-responded {ok, pong} -> ok; {error, {closed, Code, Reason}} -> handle_close(Code) end. ``` -------------------------------- ### WebTransport Connection with Options Source: https://github.com/benoitc/hackney/blob/master/guides/webtransport_guide.md Connect to a WebTransport server with custom options, including transport protocol, timeouts, and headers. The `transport` option can be `h3` (QUIC) or `h2`. ```erlang {ok, Conn} = hackney:wt_connect(<<"https://example.com/wt">>, [ {transport, h3}, {connect_timeout, 5000}, {recv_timeout, 30000}, {headers, [{<<"authorization">>, <<"Bearer token">>}]} ]). ``` -------------------------------- ### Run Dialyzer on hackney Source: https://github.com/benoitc/hackney/blob/master/CONTRIBUTING.md Perform static analysis on the hackney project using Dialyzer. ```shell rebar3 dialyzer ``` -------------------------------- ### Configure Redirect Following with Hackney Source: https://github.com/benoitc/hackney/blob/master/README.md Enable automatic following of HTTP redirects and set a maximum number of redirects to prevent infinite loops. ```erlang %% Follow redirects automatically hackney:get(URL, [], <<>>, [{follow_redirect, true}, {max_redirect, 5}]). ``` -------------------------------- ### Configure insecure_basic_auth globally Source: https://github.com/benoitc/hackney/blob/master/NEWS.md To maintain backward compatibility with pre-1.24.0 behavior, `insecure_basic_auth` now defaults to `true`. Set this globally to `false` if strict HTTPS-only basic auth is required. ```erlang application:set_env(hackney, insecure_basic_auth, false). ``` -------------------------------- ### Handle empty trailers and fix connection pool race condition Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.17.0 includes fixes for handling empty trailers, addressing a race condition in the connection pool, and resolving a memory leak within the pool. ```erlang handle empty trailers ``` ```erlang fix race condition in connection pool ``` ```erlang fix memory leak in connection pool ``` -------------------------------- ### Basic WebTransport Connection Source: https://github.com/benoitc/hackney/blob/master/guides/webtransport_guide.md Establish a basic WebTransport connection using the 'https://' scheme. WebTransport requires TLS. ```erlang {ok, Conn} = hackney:wt_connect(<<"https://example.com/wt">>). ``` -------------------------------- ### Fix build on macOS with OTP >= 20.1 Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.17.0 resolves build issues on macOS when using OTP versions 20.1 and later. ```erlang fix build on macosx with OTP >= 20.1 ``` -------------------------------- ### Restoring Cross-Host Redirect Behavior Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Demonstrates how to re-enable the forwarding of authorization headers and credentials when following redirects to a different host, using the `location_trusted` option. This is not recommended due to potential credential leakage. ```erlang hackney:get(URL, [], <<>>, [{location_trusted, true}]). ``` -------------------------------- ### Manual Alt-Svc Cache Management Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Provides functions to manually check, cache, clear, or clear all HTTP/3 Alt-Svc entries for hosts. ```erlang %% Check if HTTP/3 is cached for a host hackney_altsvc:lookup(<<"example.com">>, 443). %% {ok, h3, 443} | none %% Manually cache HTTP/3 endpoint hackney_altsvc:cache(<<"example.com">>, 443, 443, 86400). %% Clear cached entry hackney_altsvc:clear(<<"example.com">>, 443). %% Clear all cached entries hackney_altsvc:clear_all(). ``` -------------------------------- ### Handle WebSocket Connection Errors Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md Demonstrates error handling for WebSocket connection attempts, covering specific HTTP errors, timeouts, and general network errors. ```erlang case hackney:ws_connect(URL) of {ok, Conn} -> use(Conn); {error, {http_error, 401}} -> unauthorized; {error, timeout} -> timeout; {error, Reason} -> {error, Reason} end. ``` -------------------------------- ### HTTP/3 Headers Format Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Illustrates the header format used in HTTP/3, which is identical to HTTP/2, utilizing lowercase header names. ```erlang %% HTTP/3 headers (same as HTTP/2) [{<<":status">>, << ``` -------------------------------- ### Configure insecure_basic_auth per-request Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Alternatively, to enforce strict HTTPS-only basic auth for a specific request, include `{insecure_basic_auth, false}` in the request options. ```erlang {insecure_basic_auth, false} ``` -------------------------------- ### Bidirectional Streaming with h2_* API Source: https://github.com/benoitc/hackney/blob/master/guides/http2_guide.md Demonstrates sending and receiving interleaved data frames for full-duplex communication, similar to gRPC bidi RPCs. Requires HTTPS and uses a dedicated connection per h2_open. ```erlang {ok, S} = hackney:h2_open(<<"https://host/pkg.Service/BidiMethod">>, [{<<"content-type">>, <<"application/grpc">>}, {<<"te">>, <<"trailers">>}], [{ssl_options, [...]}]), {ok, {response, 200, _Headers}} = hackney:h2_recv(S), ok = hackney:h2_send(S, Frame1), {ok, {data, Reply1}} = hackney:h2_recv(S), ok = hackney:h2_send(S, Frame2), %% keep sending while receiving {ok, {data, Reply2}} = hackney:h2_recv(S), ok = hackney:h2_send(S, <<>>, fin), %% half-close the request {ok, {trailers, Trailers}} = hackney:h2_recv(S), {ok, done} = hackney:h2_recv(S), ok = hackney:h2_close(S). ``` -------------------------------- ### Hackney Process Supervision Tree Source: https://github.com/benoitc/hackney/blob/master/guides/design.md Illustrates the hierarchical structure of Hackney's supervision tree, including managers, connection supervisors, and pool supervisors. ```erlang hackney_sup ├── hackney_manager (connection registry) ├── hackney_conn_sup (connection supervisor) │ └── hackney_conn [1..N] (connection processes for HTTP/1.1, HTTP/2) ├── hackney_pools_sup (pool supervisor) │ └── hackney_pool [1..N] (pool processes) └── hackney_altsvc (Alt-Svc cache for HTTP/3 discovery) QUIC connections (HTTP/3): └── hackney_h3.erl (HTTP/3 high-level + adapter over quic_h3) ``` -------------------------------- ### Update GitHub Actions and dependencies Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.24.0 updates GitHub Actions to use `ubuntu-22.04` and bumps `certifi`/`mimerl` dependencies. ```erlang improvement: update GitHub Actions to ubuntu-22.04 and bump certifi/mimerl dependencies ``` -------------------------------- ### Handle HTTP and HTTPS proxies separately Source: https://github.com/benoitc/hackney/blob/master/NEWS.md Version 1.21.0 ensures that HTTP and HTTPS proxies are handled distinctly, preventing potential configuration conflicts. ```erlang fix: handle http & https proxies separately ``` -------------------------------- ### Automatic HTTP/2 Negotiation with Hackney Source: https://github.com/benoitc/hackney/blob/master/README.md This snippet shows how HTTP/2 is automatically used when the server supports it via ALPN negotiation. ```erlang %% Automatic HTTP/2 via ALPN negotiation {ok, 200, Headers, Body} = hackney:get(<<"https://nghttp2.org/">>). ``` -------------------------------- ### Receive Messages in Active Mode Source: https://github.com/benoitc/hackney/blob/master/guides/websocket_guide.md Demonstrates how to receive WebSocket messages and connection status updates when the client is in active mode using Erlang's 'receive' construct. ```erlang receive {hackney_ws, Conn, {text, Text}} -> handle(Text); {hackney_ws, Conn, closed} -> done; {hackney_ws_error, Conn, Reason} -> error end. ``` -------------------------------- ### Automatic Alt-Svc Discovery for HTTP/3 Source: https://github.com/benoitc/hackney/blob/master/guides/http3_guide.md Demonstrates how Hackney automatically uses HTTP/3 on subsequent requests after discovering it via the Alt-Svc header. The first request might use HTTP/2 or HTTP/1.1. ```erlang %% First request uses HTTP/2 or HTTP/1.1 %% Server returns Alt-Svc: h3=":443"; ma=86400 {ok, _, Headers1, _} = hackney:get(URL, [], <<>>, [{protocols, [http3, http2, http1]}]). %% Alt-Svc is now cached, second request uses HTTP/3 {ok, _, Headers2, _} = hackney:get(URL, [], <<>>, [{protocols, [http3, http2, http1]}]). ```