HPX (longcipher/hpx)

HPX

https://github.com/longcipher/hpx
Admin
HPX is a high-performance HTTP client for crypto exchange HFT applications with browser emulation,...

Tokens:73,260
Snippets:240
Trust Score:7.5
Update:1 month ago
Show doc for...
Context Summary (auto-generated)
Raw
# hpx

hpx is a high-performance HTTP client library for Rust, designed specifically for the network layer of cryptocurrency exchange HFT (high-frequency trading) applications. It's a fork of wreq/reqwest with a primary focus on performance optimization. The library provides an ergonomic all-in-one HTTP client with browser emulation capabilities including TLS fingerprinting (JA3/JA4) and HTTP/2 fingerprint simulation.

The hpx ecosystem consists of five crates: `hpx` (core HTTP client), `hpx-emulation` (browser emulation profiles), `hpx-transport` (exchange SDK toolkit), `hpx-yawc` (RFC 6455 WebSocket with compression), and `hpx-fastwebsockets` (minimal WebSocket implementation). Key features include connection pooling, Tower middleware support, automatic compression (gzip, brotli, zstd, deflate), cookie management, redirect policies, rotating proxies, and both BoringSSL and Rustls TLS backends.

## Making a GET Request

The simplest way to make HTTP requests using the shortcut functions that create a new client on each call. For production use with multiple requests, reuse a `Client` instance to take advantage of connection pooling.

```rust
use hpx::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Simple GET request with shortcut function
    let body = hpx::get("https://www.rust-lang.org")
        .send()
        .await?
        .text()
        .await?;

    println!("Response body: {}", body);
    Ok(())
}
```

## Making POST Requests with JSON

Send JSON data using the `json` feature. The `json()` method automatically serializes your struct and sets the `Content-Type: application/json` header.

```rust
use serde::{Deserialize, Serialize};
use hpx::Result;

#[derive(Debug, Serialize, Deserialize)]
struct Post {
    id: Option<i32>,
    title: String,
    body: String,
    #[serde(rename = "userId")]
    user_id: i32,
}

#[tokio::main]
async fn main() -> Result<()> {
    let new_post = Post {
        id: None,
        title: "hpx.rs".into(),
        body: "https://docs.rs/hpx".into(),
        user_id: 1,
    };

    // POST with JSON body, response deserialized to typed struct
    let created_post: Post = hpx::post("https://jsonplaceholder.typicode.com/posts")
        .json(&new_post)
        .send()
        .await?
        .json()
        .await?;

    println!("Created post with id: {:?}", created_post.id);
    // Output: Created post with id: Some(101)
    Ok(())
}
```

## Creating a Reusable Client

For applications making multiple requests, create a `Client` instance once and reuse it. This enables connection pooling, shared configuration, and better performance.

```rust
use hpx::{Client, Result};
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<()> {
    // Build a client with custom configuration
    let client = Client::builder()
        .timeout(Duration::from_secs(30))
        .user_agent("my-app/1.0")
        .build()?;

    // Make multiple requests reusing the same client
    let resp1 = client.get("https://api.example.com/users").send().await?;
    let resp2 = client.get("https://api.example.com/orders").send().await?;

    println!("Status 1: {}, Status 2: {}", resp1.status(), resp2.status());
    Ok(())
}
```

## Browser Emulation

Emulate browser TLS/HTTP2 fingerprints using the `hpx-emulation` crate. This is essential for accessing sites that block non-browser clients based on TLS fingerprinting.

```rust
use hpx::{Client, Result};
use hpx_emulation::Emulation;

#[tokio::main]
async fn main() -> Result<()> {
    // Build client emulating Safari 26
    let client = Client::builder()
        .emulation(Emulation::Safari26)
        .cert_verification(false)
        .build()?;

    let text = client
        .get("https://tls.browserleaks.com/")
        .send()
        .await?
        .text()
        .await?;

    println!("{}", text);
    Ok(())
}
```

## Per-Request Browser Emulation

Apply browser emulation to individual requests without configuring the entire client. Supports Chrome (100-143), Edge (101-142), Firefox (109-146), Safari (15.3-26.2), and Opera (116-119).

```rust
use hpx::Result;
use hpx_emulation::Emulation;

#[tokio::main]
async fn main() -> Result<()> {
    // Emulate Firefox 136 for this specific request
    let resp = hpx::get("https://tls.peet.ws/api/all")
        .emulation(Emulation::Firefox136)
        .send()
        .await?;

    println!("TLS fingerprint: {}", resp.text().await?);
    Ok(())
}
```

## WebSocket Connections (HTTP/1.1)

Establish WebSocket connections with the `ws` feature enabled. Uses futures-based stream/sink API for sending and receiving messages.

```rust
use futures_util::TryStreamExt;
use hpx::{header, ws::message::Message, Result};

#[tokio::main]
async fn main() -> Result<()> {
    let resp = hpx::websocket("wss://echo.websocket.org")
        .header(header::USER_AGENT, "my-app")
        .send()
        .await?;

    assert_eq!(resp.version(), http::Version::HTTP_11);

    let websocket = resp.into_websocket().await?;
    if let Some(protocol) = websocket.protocol() {
        println!("WebSocket subprotocol: {:?}", protocol);
    }

    let (mut tx, mut rx) = websocket.split();

    // Send messages in a separate task
    tokio::spawn(async move {
        for i in 1..11 {
            if let Err(err) = tx.send(Message::text(format!("Hello, World! {i}"))).await {
                eprintln!("Failed to send message: {err}");
            }
        }
    });

    // Receive messages
    while let Some(message) = rx.try_next().await? {
        if let Message::Text(text) = message {
            println!("Received: {text}");
        }
    }

    Ok(())
}
```

## Cookie Management

Enable automatic cookie storage and sending with the `cookies` feature. Cookies persist across requests within the same client session.

```rust
use hpx::{Client, Result, cookie::Jar};

#[tokio::main]
async fn main() -> Result<()> {
    let jar = Jar::default();

    let client = Client::builder()
        .cookie_provider(jar.clone())
        .build()?;

    // This request sets a cookie
    let _resp = client
        .get("https://httpbin.org/cookies/set/session/123")
        .send()
        .await?;

    // Cookies are automatically sent with subsequent requests
    println!("Stored cookies: {:?}", jar.cookies("https://httpbin.org"));
    Ok(())
}
```

## Form Data Submission

Send URL-encoded form data with the `form` feature. Automatically sets `Content-Type: application/x-www-form-urlencoded`.

```rust
use std::collections::HashMap;
use hpx::{Client, Result};

#[tokio::main]
async fn main() -> Result<()> {
    let mut params = HashMap::new();
    params.insert("username", "admin");
    params.insert("password", "secret123");

    let client = Client::new();
    let res = client
        .post("http://httpbin.org/post")
        .form(&params)
        .send()
        .await?;

    println!("Response status: {}", res.status());
    Ok(())
}
```

## Authentication Methods

Multiple authentication methods are supported: basic auth, bearer tokens, and custom authorization headers.

```rust
use hpx::{Client, Result};

#[tokio::main]
async fn main() -> Result<()> {
    let client = Client::new();

    // Basic authentication
    let resp = client
        .delete("http://httpbin.org/delete")
        .basic_auth("admin", Some("good_password"))
        .send()
        .await?;

    // Bearer token authentication
    let resp = client
        .get("http://httpbin.org/get")
        .bearer_auth("your_jwt_token_here")
        .send()
        .await?;

    // Custom authorization header
    let resp = client
        .get("http://httpbin.org/get")
        .auth("CustomToken abc123")
        .send()
        .await?;

    Ok(())
}
```

## Proxy Configuration

Configure HTTP, HTTPS, and SOCKS proxies. The `socks` feature enables SOCKS4/SOCKS5 proxy support.

```rust
use hpx::{Client, Proxy, Result};

#[tokio::main]
async fn main() -> Result<()> {
    // HTTP proxy
    let proxy = Proxy::http("http://proxy.example.com:8080")?;

    // SOCKS5 proxy (requires `socks` feature)
    let socks_proxy = Proxy::all("socks5://127.0.0.1:1080")?;

    let client = Client::builder()
        .proxy(socks_proxy)
        .build()?;

    let resp = client.get("https://api.example.com").send().await?;
    println!("Status: {}", resp.status());
    Ok(())
}
```

## Rotating Proxy Pool

Use multiple proxies with rotation strategies for load distribution or failover scenarios.

```rust
use hpx::{Client, Proxy, ProxyPool, ProxyPoolStrategy, Result};

#[tokio::main]
async fn main() -> Result<()> {
    let proxies = vec![
        Proxy::all("http://proxy1.example.com:8080")?,
        Proxy::all("http://proxy2.example.com:8080")?,
        Proxy::all("http://proxy3.example.com:8080")?,
    ];

    // Round-robin rotation
    let pool = ProxyPool::with_strategy(proxies, ProxyPoolStrategy::RoundRobin)?;

    // Or sticky failover (use first until it fails)
    // let pool = ProxyPool::with_strategy(proxies, ProxyPoolStrategy::StickyFailover)?;

    let client = Client::builder()
        .proxy_pool(pool)
        .build()?;

    let resp = client.get("https://api.example.com").send().await?;
    Ok(())
}
```

## Exchange REST Client (hpx-transport)

The `hpx-transport` crate provides a high-level REST client for cryptocurrency exchange APIs with built-in authentication support.

```rust
use hpx_transport::{
    auth::ApiKeyAuth,
    exchange::{ExchangeClient, RestClient, RestConfig},
    TypedResponse,
};
use serde::Deserialize;
use std::time::Duration;

#[derive(Debug, Deserialize)]
struct AccountInfo {
    balance: f64,
    currency: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = RestConfig::new("https://api.exchange.com")
        .timeout(Duration::from_secs(30))
        .user_agent("my-trading-bot/1.0");

    let auth = ApiKeyAuth::header("X-API-Key", "your-api-key-here");
    let client = RestClient::new(config, auth)?;

    // GET request with typed response
    let response: TypedResponse<AccountInfo> = client.get("/v1/account").await?;
    println!("Balance: {} {}", response.data.balance, response.data.currency);
    println!("Latency: {:?}", response.latency);

    Ok(())
}
```

## HMAC Authentication for Exchanges

Use HMAC-SHA256/SHA512 signing for exchanges like Binance, OKX that require request signatures.

```rust
use hpx_transport::{
    auth::{HmacAuth, HmacAlgorithm},
    exchange::{ExchangeClient, RestClient, RestConfig},
};
use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct OrderParams {
    symbol: String,
    side: String,
    quantity: f64,
}

#[derive(Deserialize)]
struct OrderResponse {
    order_id: String,
    status: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Binance-style HMAC authentication
    let auth = HmacAuth::new("your-api-key", "your-secret-key")
        .algorithm(HmacAlgorithm::Sha256)
        .api_key_header("X-MBX-APIKEY")
        .signature_param("signature")
        .timestamp_param("timestamp")
        .timestamp_in_query(true);

    let config = RestConfig::new("https://api.binance.com");
    let client = RestClient::new(config, auth)?;

    let order = OrderParams {
        symbol: "BTCUSDT".to_string(),
        side: "BUY".to_string(),
        quantity: 0.001,
    };

    let response: hpx_transport::TypedResponse<OrderResponse> =
        client.post("/api/v3/order", &order).await?;

    println!("Order ID: {}", response.data.order_id);
    Ok(())
}
```

## WebSocket Connection with Protocol Handler (hpx-transport)

Establish WebSocket connections with request-response correlation and subscription management for exchange APIs.

```rust
use hpx_transport::websocket::{
    Connection, WsConfig, Event,
    handlers::GenericJsonHandler,
};
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = WsConfig::new("wss://api.exchange.com/ws")
        .ping_interval(Duration::from_secs(30))
        .request_timeout(Duration::from_secs(10));

    let handler = GenericJsonHandler::new();
    let connection = Connection::connect_stream(config, handler).await?;
    let (handle, mut stream) = connection.split();

    // Subscribe to a topic
    let mut guard = handle.subscribe("orderbook.BTC").await?;

    // Receive subscription updates
    tokio::spawn(async move {
        while let Some(msg) = guard.recv().await {
            println!("Orderbook update: {:?}", msg);
        }
    });

    // Process stream events
    while let Some(event) = stream.next().await {
        match event {
            Event::Message(msg) => println!("Message: {:?}", msg.kind),
            Event::Connected(epoch) => println!("Connected, epoch: {:?}", epoch),
            Event::Disconnected(reason) => println!("Disconnected: {:?}", reason),
        }
    }

    Ok(())
}
```

## Rate Limiting

Token bucket rate limiter for managing API request rates to avoid hitting exchange rate limits.

```rust
use hpx_transport::rate_limit::RateLimiter;
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limiter = RateLimiter::new();

    // 10 requests capacity, refills at 1 token per second
    limiter.add_limit("orders", 10, 1.0)?;
    // 100 requests capacity, refills at 10 tokens per second
    limiter.add_limit("public", 100, 10.0)?;

    // Non-blocking check
    if limiter.try_acquire("orders") {
        println!("Request allowed, making order...");
    } else {
        let wait = limiter.time_until_available("orders");
        println!("Rate limited, wait: {:?}", wait);
    }

    // Blocking acquire (waits until token available)
    limiter.acquire("orders").await;
    println!("Token acquired after waiting");

    // Check available tokens
    if let Some(tokens) = limiter.available_tokens("orders") {
        println!("Available tokens: {:.1}", tokens);
    }

    Ok(())
}
```

## hpx-yawc WebSocket Client

Direct WebSocket connections using the low-level `hpx-yawc` crate with RFC 6455 compliance and permessage-deflate compression.

```rust
use futures::{SinkExt, StreamExt};
use hpx_yawc::{WebSocket, frame::OpCode};

#[tokio::main]
async fn main() -> hpx_yawc::Result<()> {
    let mut ws = WebSocket::connect("wss://echo.websocket.org".parse()?).await?;

    // Send a text message
    let frame = hpx_yawc::frame::Frame::text("Hello, WebSocket!");
    ws.send(frame).await?;

    // Echo server: receive and echo back messages
    while let Some(frame) = ws.next().await {
        match frame.opcode() {
            OpCode::Text | OpCode::Binary => {
                println!("Received: {:?}", frame);
                ws.send(frame).await?;  // Echo back
            }
            OpCode::Ping => {
                // Pong is sent automatically, but ping frame is still returned
                println!("Received ping");
            }
            OpCode::Close => {
                println!("Connection closed");
                break;
            }
            _ => {}
        }
    }

    Ok(())
}
```

## Timeout Configuration

Fine-grained timeout control for different phases of the request lifecycle.

```rust
use hpx::{Client, Result};
use std::time::Duration;

#[tokio::main]
async fn main() -> Result<()> {
    let client = Client::builder()
        .timeout(Duration::from_secs(30))  // Total request timeout
        .connect_timeout(Duration::from_secs(5))  // TCP + TLS handshake
        .build()?;

    // Per-request timeout overrides
    let resp = client
        .get("https://api.example.com/slow-endpoint")
        .timeout(Duration::from_secs(60))
        .read_timeout(Duration::from_secs(10))
        .timeout_connect(Some(Duration::from_secs(3)))
        .send()
        .await?;

    Ok(())
}
```

## Multipart Form Upload

Upload files and form data using multipart/form-data encoding with the `multipart` feature.

```rust
use hpx::{Client, multipart, Result};

#[tokio::main]
async fn main() -> Result<()> {
    let form = multipart::Form::new()
        .text("username", "john_doe")
        .text("description", "Profile picture upload")
        .file("avatar", "/path/to/image.png")?;

    let client = Client::new();
    let response = client
        .post("https://api.example.com/upload")
        .multipart(form)
        .send()
        .await?;

    println!("Upload status: {}", response.status());
    Ok(())
}
```

## Custom Emulation Options

Configure advanced browser emulation settings including operating system, HTTP/2 settings, and header behavior.

```rust
use hpx::{Client, Result};
use hpx_emulation::{Emulation, EmulationOption, EmulationOS};

#[tokio::main]
async fn main() -> Result<()> {
    // Full control over emulation settings
    let emulation_opts = EmulationOption::builder()
        .emulation(Emulation::Chrome143)
        .emulation_os(EmulationOS::MacOS)
        .skip_http2(false)
        .skip_headers(false)
        .build();

    let client = Client::builder()
        .emulation(emulation_opts)
        .build()?;

    let resp = client
        .get("https://tls.peet.ws/api/all")
        .send()
        .await?;

    println!("Fingerprint: {}", resp.text().await?);
    Ok(())
}
```

hpx is designed for high-performance trading applications where network latency and reliability are critical. The library uses lock-free data structures (`scc::HashMap`, `arc-swap`) for hot-path operations, SIMD-accelerated JSON parsing with the `simd-json` feature, and async DNS resolution with `hickory-dns` to avoid blocking system calls. The architecture supports Tower middleware for adding cross-cutting concerns like retry logic, rate limiting, and request/response hooks.

For cryptocurrency exchange integrations, the `hpx-transport` crate provides a complete SDK with authentication strategies (API key, HMAC signing, Bearer tokens), WebSocket connection management with auto-reconnection, subscription handling, and OpenTelemetry metrics integration. The WebSocket implementation uses a single-owner connection loop with lock-free stores for pending requests and subscriptions, enabling high-throughput market data streaming.