# HPX - High Performance HTTP Client

## Introduction

HPX is a high-performance HTTP client library for Rust, forked from wreq/reqwest, specifically optimized for the network layer of cryptocurrency exchange high-frequency trading (HFT) applications. The library provides an ergonomic, all-in-one HTTP client with advanced browser emulation capabilities, including TLS fingerprinting (JA3/JA4) and HTTP/2 settings replication. Built on Tokio's async runtime, HPX offers robust connection pooling, Tower middleware integration, and support for both HTTP/1 and HTTP/2 protocols with flexible TLS backends (BoringSSL or Rustls).

HPX excels in scenarios requiring precise browser fingerprint emulation, making it ideal for web scraping, API testing, penetration testing, and applications that need to bypass fingerprint-based detection systems. The library supports over 90 pre-configured browser profiles spanning Chrome (v100-143), Firefox (v109-146), Safari (v15.3-26.2 including iOS and iPad variants), Edge (v101-142), Opera (v116-119), and OkHttp (v3.9-5) clients. It provides comprehensive features including cookie stores, redirect policies, rotating proxies, WebSocket upgrades, and multiple compression algorithms, all accessible through a fluent builder API similar to the popular reqwest crate.

## API Reference

### Simple GET Request

Basic HTTP GET request with response text extraction.

```rust
use hpx;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let body = hpx::get("https://www.rust-lang.org")
        .send()
        .await?
        .text()
        .await?;

    println!("body = {:?}", body);
    Ok(())
}
```

### JSON POST Request with Typed Responses

Send JSON data and deserialize the response into a strongly-typed struct.

```rust
use hpx;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
struct Post {
    id: Option<i32>,
    title: String,
    body: String,
    #[serde(rename = "userId")]
    user_id: i32,
}

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let new_post = Post {
        id: None,
        title: "hpx.rs".into(),
        body: "https://docs.rs/hpx".into(),
        user_id: 1,
    };

    let response: Post = hpx::post("https://jsonplaceholder.typicode.com/posts")
        .json(&new_post)
        .send()
        .await?
        .json()
        .await?;

    println!("{response:#?}");
    // Output:
    // Post {
    //     id: Some(101),
    //     title: "hpx.rs",
    //     body: "https://docs.rs/hpx",
    //     user_id: 1
    // }
    Ok(())
}
```

### Form Data POST Request

Submit URL-encoded form data to an endpoint.

```rust
use hpx;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let response = hpx::post("http://www.example.com/form")
        .form(&[("username", "alice"), ("password", "secret123")])
        .send()
        .await?;

    println!("Response status: {}", response.status());
    println!("Response body: {}", response.text().await?);
    Ok(())
}
```

### Client Builder with Configuration

Create a reusable client with custom settings including timeouts, redirects, and cookie store.

```rust
use hpx::{Client, redirect};
use std::time::Duration;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let client = Client::builder()
        .user_agent("MyApp/1.0")
        .timeout(Duration::from_secs(10))
        .connect_timeout(Duration::from_secs(5))
        .cookie_store(true)
        .redirect(redirect::Policy::limited(10))
        .pool_max_idle_per_host(10)
        .tcp_nodelay(true)
        .build()?;

    let response = client
        .get("https://api.example.com/data")
        .header("X-API-Key", "your-api-key")
        .send()
        .await?;

    println!("Status: {}", response.status());
    println!("Headers: {:#?}", response.headers());
    println!("Body: {}", response.text().await?);

    Ok(())
}
```

### Browser Emulation - Quick Mode

Use pre-configured browser profiles for simple fingerprint emulation.

```rust
use hpx::Client;
use hpx_util::Emulation;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Build a client that emulates Safari 26 browser
    let client = Client::builder()
        .emulation(Emulation::Safari26)
        .cert_verification(false)
        .build()?;

    let text = client
        .get("https://tls.browserleaks.com/json")
        .send()
        .await?
        .text()
        .await?;

    println!("{}", text);
    Ok(())
}
```

### Browser Emulation - Per-Request

Apply browser emulation to individual requests using the `hpx_util` crate.

```rust
use hpx;
use hpx_util::Emulation;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let resp = hpx::get("https://tls.peet.ws/api/all")
        .emulation(Emulation::Firefox136)
        .send()
        .await?;

    println!("{}", resp.text().await?);
    Ok(())
}
```

### Browser Emulation - Advanced Custom Configuration

Build a fully customized browser emulation profile with precise TLS, HTTP/2, and header settings.

```rust
use hpx::{
    Emulation, Client,
    header::{self, HeaderMap, HeaderValue, OrigHeaderMap},
    http2::{Http2Options, PseudoId, PseudoOrder},
    tls::{AlpnProtocol, TlsOptions, TlsVersion},
};

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Configure TLS parameters to match target browser
    let tls = TlsOptions::builder()
        .enable_ocsp_stapling(true)
        .curves_list("X25519:P-256:P-384")
        .cipher_list(
            "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:\
             TLS_CHACHA20_POLY1305_SHA256:\
             TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:\
             TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
        )
        .sigalgs_list(
            "ecdsa_secp256r1_sha256:rsa_pss_rsae_sha256:\
             rsa_pkcs1_sha256:ecdsa_secp384r1_sha384:\
             rsa_pss_rsae_sha384:rsa_pkcs1_sha384"
        )
        .alpn_protocols([AlpnProtocol::HTTP2, AlpnProtocol::HTTP1])
        .min_tls_version(TlsVersion::TLS_1_2)
        .max_tls_version(TlsVersion::TLS_1_3)
        .build();

    // Configure HTTP/2 settings to match target browser
    let http2 = Http2Options::builder()
        .initial_stream_id(3)
        .initial_window_size(16777216)
        .initial_connection_window_size(16711681 + 65535)
        .headers_pseudo_order(
            PseudoOrder::builder()
                .extend([
                    PseudoId::Method,
                    PseudoId::Path,
                    PseudoId::Authority,
                    PseudoId::Scheme,
                ])
                .build(),
        )
        .build();

    // Set default headers with standard values
    let headers = {
        let mut headers = HeaderMap::new();
        headers.insert(
            header::USER_AGENT,
            HeaderValue::from_static("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36")
        );
        headers.insert(header::ACCEPT_LANGUAGE, HeaderValue::from_static("en-US,en;q=0.9"));
        headers.insert(header::ACCEPT_ENCODING, HeaderValue::from_static("br, gzip, deflate"));
        headers.insert(header::ACCEPT, HeaderValue::from_static("application/json"));
        headers.insert(header::CACHE_CONTROL, HeaderValue::from_static("no-store"));
        headers
    };

    // Preserve original header casing (important for fingerprinting)
    let orig_headers = {
        let mut orig_headers = OrigHeaderMap::new();
        orig_headers.insert("cookie");
        orig_headers.insert("content-length");
        orig_headers.insert("User-Agent");
        orig_headers.insert("Accept-Language");
        orig_headers.insert("Accept-Encoding");
        orig_headers
    };

    // Combine all settings into an emulation profile
    let emulation = Emulation::builder()
        .tls_options(tls)
        .http2_options(http2)
        .headers(headers)
        .orig_headers(orig_headers)
        .build();

    let resp = hpx::get("https://tls.peet.ws/api/all")
        .emulation(emulation)
        .send()
        .await?;

    println!("{}", resp.text().await?);
    Ok(())
}
```

### Request with Authentication

Add authentication headers including basic auth and bearer tokens.

```rust
use hpx::Client;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let client = Client::new();

    // Basic authentication
    let resp1 = client
        .get("https://api.example.com/protected")
        .basic_auth("username", Some("password"))
        .send()
        .await?;
    println!("Basic auth response: {}", resp1.status());

    // Bearer token authentication
    let resp2 = client
        .get("https://api.example.com/oauth-protected")
        .bearer_auth("eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...")
        .send()
        .await?;
    println!("Bearer auth response: {}", resp2.status());

    // Custom authorization header
    let resp3 = client
        .get("https://api.example.com/custom")
        .header("Authorization", "CustomScheme token123")
        .send()
        .await?;
    println!("Custom auth response: {}", resp3.status());

    Ok(())
}
```

### Proxy Configuration

Route requests through HTTP, HTTPS, or SOCKS proxies.

```rust
use hpx::{Client, Proxy};

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Per-request proxy
    let resp = hpx::get("https://api.ip.sb/ip")
        .proxy(Proxy::all("socks5h://localhost:6153")?)
        .send()
        .await?;
    println!("IP via SOCKS5: {}", resp.text().await?);

    // Client-level proxy configuration
    let client = Client::builder()
        .proxy(Proxy::http("http://proxy.example.com:8080")?)
        .proxy(Proxy::https("https://proxy.example.com:8443")?)
        .build()?;

    let response = client
        .get("https://httpbin.org/ip")
        .send()
        .await?;
    println!("IP via HTTP proxy: {}", response.text().await?);

    // Proxy with authentication
    let authenticated_proxy = Proxy::all("http://username:password@proxy.example.com:8080")?;
    let resp = hpx::get("https://api.example.com/data")
        .proxy(authenticated_proxy)
        .send()
        .await?;

    Ok(())
}
```

### WebSocket Connection with Message Exchange

Establish WebSocket connections and exchange messages using the split interface.

```rust
use futures_util::{SinkExt, StreamExt, TryStreamExt};
use hpx::{header, ws::message::Message};

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Initiate WebSocket handshake
    let resp = hpx::websocket("wss://echo.websocket.org")
        .header(header::USER_AGENT, "MyApp/1.0")
        .send()
        .await?;

    // Verify HTTP version and upgrade to WebSocket
    assert_eq!(resp.version(), http::Version::HTTP_11);
    let websocket = resp.into_websocket().await?;

    // Check negotiated subprotocol
    if let Some(protocol) = websocket.protocol() {
        println!("WebSocket subprotocol: {:?}", protocol);
    }

    // Split into sender and receiver
    let (mut tx, mut rx) = websocket.split();

    // Spawn task to send messages
    tokio::spawn(async move {
        for i in 1..11 {
            if let Err(err) = tx.send(Message::text(format!("Hello, World! {i}"))).await {
                eprintln!("Failed to send message: {err}");
            }
        }
    });

    // Receive messages
    while let Some(message) = rx.try_next().await? {
        match message {
            Message::Text(text) => println!("Received: {text}"),
            Message::Binary(data) => println!("Received binary: {} bytes", data.len()),
            Message::Ping(_) => println!("Received ping"),
            Message::Pong(_) => println!("Received pong"),
            Message::Close(frame) => {
                println!("Connection closed: {:?}", frame);
                break;
            }
        }
    }

    Ok(())
}
```

### Response Handling with Streaming

Handle responses with various consumption methods including streaming.

```rust
use hpx::Client;
use std::fs::File;
use std::io::Write;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let client = Client::new();
    let response = client
        .get("https://httpbin.org/json")
        .send()
        .await?;

    // Inspect response metadata
    println!("Status: {}", response.status());
    println!("Version: {:?}", response.version());
    println!("Headers: {:#?}", response.headers());
    println!("Content-Length: {:?}", response.content_length());

    // Get the full response as text
    let text = response.text().await?;
    println!("Body as text: {}", text);

    // For binary data or streaming to file
    let mut response = client
        .get("https://example.com/largefile.zip")
        .send()
        .await?;

    let mut file = File::create("download.zip")?;
    while let Some(chunk) = response.chunk().await? {
        file.write_all(&chunk)?;
    }

    println!("File downloaded successfully");
    Ok(())
}
```

### Error Handling and Retry Logic

Implement comprehensive error handling with custom retry policies.

```rust
use hpx::{Client, retry};
use std::time::Duration;

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Client with retry policy
    let client = Client::builder()
        .retry(
            retry::Policy::default()
                .with_max_retries(3)
                .with_min_retry_interval(Duration::from_millis(100))
                .with_max_retry_interval(Duration::from_secs(10))
        )
        .timeout(Duration::from_secs(30))
        .build()?;

    // Handle response with error checking
    match client.get("https://api.example.com/data").send().await {
        Ok(response) => {
            if response.status().is_success() {
                let data = response.text().await?;
                println!("Success: {}", data);
            } else if response.status().is_client_error() {
                eprintln!("Client error {}: {}", response.status(), response.text().await?);
            } else if response.status().is_server_error() {
                eprintln!("Server error {}: retrying...", response.status());
            }
        }
        Err(e) => {
            if e.is_timeout() {
                eprintln!("Request timeout");
            } else if e.is_connect() {
                eprintln!("Connection error: {}", e);
            } else if e.is_request() {
                eprintln!("Request error: {}", e);
            } else {
                eprintln!("Other error: {}", e);
            }
        }
    }

    Ok(())
}
```

### Advanced Request Configuration

Combine multiple configuration options for complex request scenarios.

```rust
use hpx::{Client, redirect, Version};
use std::time::Duration;
use std::net::{IpAddr, Ipv4Addr};

#[tokio::main]
async fn main() -> hpx::Result<()> {
    let client = Client::builder()
        .cookie_store(true)
        .gzip(true)
        .brotli(true)
        .deflate(true)
        .build()?;

    let response = client
        .post("https://api.example.com/submit")
        .version(Version::HTTP_2)
        .timeout(Duration::from_secs(10))
        .read_timeout(Duration::from_secs(60))
        .local_address(IpAddr::V4(Ipv4Addr::new(192, 168, 1, 100)))
        .header("X-Request-ID", "req-12345")
        .header("X-Correlation-ID", "corr-67890")
        .query(&[("filter", "active"), ("limit", "100")])
        .json(&serde_json::json!({
            "action": "create",
            "data": {
                "name": "Example",
                "value": 42
            }
        }))
        .send()
        .await?;

    println!("Status: {}", response.status());
    println!("Final URL: {}", response.uri());
    println!("Remote address: {:?}", response.remote_addr());

    let json: serde_json::Value = response.json().await?;
    println!("Response: {:#?}", json);

    Ok(())
}
```

### Certificate Store Configuration

Configure custom certificate stores for TLS validation.

```rust
use hpx::{Client, tls::CertStore};

#[tokio::main]
async fn main() -> hpx::Result<()> {
    // Load custom CA certificates
    let cert_pem = std::fs::read("custom-ca.pem")?;
    let cert_store = CertStore::from_pem(&cert_pem)?;

    let client = Client::builder()
        .cert_store(cert_store)
        .cert_verification(true)
        .tls_sni(true)
        .build()?;

    let response = client
        .get("https://internal-api.company.com")
        .send()
        .await?;

    println!("Response: {}", response.text().await?);

    // Disable certificate verification (use only for testing)
    let insecure_client = Client::builder()
        .cert_verification(false)
        .build()?;

    Ok(())
}
```

## Summary

HPX provides a comprehensive solution for HTTP client needs in high-performance Rust applications, with particular strengths in scenarios requiring browser fingerprint emulation and low-latency networking. The library's main use cases include cryptocurrency exchange API integration, web scraping with anti-detection requirements, automated testing of web applications with browser-specific behavior, and penetration testing where precise TLS and HTTP/2 fingerprinting is crucial. The extensive collection of over 90 pre-configured emulation profiles covers all major browsers (Chrome, Firefox, Safari, Edge, Opera) across desktop and mobile platforms, plus OkHttp for Android app emulation, eliminating the need for manual fingerprint configuration in most cases.

Integration patterns follow familiar Rust async/await conventions with Tokio runtime compatibility. The fluent builder API enables both quick one-off requests through top-level functions (`hpx::get`, `hpx::post`) and sophisticated client configurations through `Client::builder()`. HPX seamlessly integrates with the Tower ecosystem for middleware composition, supporting custom layers for logging, metrics, rate limiting, and circuit breaking. The library's architecture promotes code reuse through shareable `Client` instances (which are `Clone` and thread-safe via `Arc` internally), while maintaining zero-copy efficiency with `bytes::Bytes` for body handling. WebSocket support provides full-duplex communication with automatic handshake negotiation, and the flexible TLS backend selection (BoringSSL for maximum compatibility, Rustls for pure-Rust security) ensures deployment flexibility across different environments and compliance requirements.