Try Live
Add Docs
Rankings
Pricing
Docs
Install
Install
Docs
Pricing
More...
More...
Try Live
Rankings
Enterprise
Create API Key
Add Docs
HPX
https://github.com/longcipher/hpx
Admin
HPX is a high-performance HTTP client for crypto exchange HFT applications with browser emulation,
...
Tokens:
67,055
Snippets:
197
Trust Score:
7.5
Update:
1 week ago
Context
Skills
Chat
Benchmark
76.5
Suggestions
Latest
Show doc for...
Code
Info
Show Results
Context Summary (auto-generated)
Raw
Copy
Link
# hpx hpx is a high-performance HTTP client library for Rust, designed specifically for the network layer of cryptocurrency exchange HFT (high-frequency trading) applications. It's a fork of wreq/reqwest with a primary focus on performance optimization. The library provides an ergonomic all-in-one HTTP client with browser emulation capabilities including TLS fingerprinting (JA3/JA4) and HTTP/2 fingerprint simulation. The hpx ecosystem consists of five crates: `hpx` (core HTTP client), `hpx-emulation` (browser emulation profiles), `hpx-transport` (exchange SDK toolkit), `hpx-yawc` (RFC 6455 WebSocket with compression), and `hpx-fastwebsockets` (minimal WebSocket implementation). Key features include connection pooling, Tower middleware support, automatic compression (gzip, brotli, zstd, deflate), cookie management, redirect policies, rotating proxies, and both BoringSSL and Rustls TLS backends. ## Making a GET Request The simplest way to make HTTP requests using the shortcut functions that create a new client on each call. For production use with multiple requests, reuse a `Client` instance to take advantage of connection pooling. ```rust use hpx::Result; #[tokio::main] async fn main() -> Result<()> { // Simple GET request with shortcut function let body = hpx::get("https://www.rust-lang.org") .send() .await? .text() .await?; println!("Response body: {}", body); Ok(()) } ``` ## Making POST Requests with JSON Send JSON data using the `json` feature. The `json()` method automatically serializes your struct and sets the `Content-Type: application/json` header. ```rust use serde::{Deserialize, Serialize}; use hpx::Result; #[derive(Debug, Serialize, Deserialize)] struct Post { id: Option<i32>, title: String, body: String, #[serde(rename = "userId")] user_id: i32, } #[tokio::main] async fn main() -> Result<()> { let new_post = Post { id: None, title: "hpx.rs".into(), body: "https://docs.rs/hpx".into(), user_id: 1, }; // POST with JSON body, response deserialized to typed struct let created_post: Post = hpx::post("https://jsonplaceholder.typicode.com/posts") .json(&new_post) .send() .await? .json() .await?; println!("Created post with id: {:?}", created_post.id); // Output: Created post with id: Some(101) Ok(()) } ``` ## Creating a Reusable Client For applications making multiple requests, create a `Client` instance once and reuse it. This enables connection pooling, shared configuration, and better performance. ```rust use hpx::{Client, Result}; use std::time::Duration; #[tokio::main] async fn main() -> Result<()> { // Build a client with custom configuration let client = Client::builder() .timeout(Duration::from_secs(30)) .user_agent("my-app/1.0") .build()?; // Make multiple requests reusing the same client let resp1 = client.get("https://api.example.com/users").send().await?; let resp2 = client.get("https://api.example.com/orders").send().await?; println!("Status 1: {}, Status 2: {}", resp1.status(), resp2.status()); Ok(()) } ``` ## Browser Emulation Emulate browser TLS/HTTP2 fingerprints using the `hpx-emulation` crate. This is essential for accessing sites that block non-browser clients based on TLS fingerprinting. ```rust use hpx::{Client, Result}; use hpx_emulation::Emulation; #[tokio::main] async fn main() -> Result<()> { // Build client emulating Safari 26 let client = Client::builder() .emulation(Emulation::Safari26) .cert_verification(false) .build()?; let text = client .get("https://tls.browserleaks.com/") .send() .await? .text() .await?; println!("{}", text); Ok(()) } ``` ## Per-Request Browser Emulation Apply browser emulation to individual requests without configuring the entire client. Supports Chrome (100-143), Edge (101-142), Firefox (109-146), Safari (15.3-26.2), and Opera (116-119). ```rust use hpx::Result; use hpx_emulation::Emulation; #[tokio::main] async fn main() -> Result<()> { // Emulate Firefox 136 for this specific request let resp = hpx::get("https://tls.peet.ws/api/all") .emulation(Emulation::Firefox136) .send() .await?; println!("TLS fingerprint: {}", resp.text().await?); Ok(()) } ``` ## WebSocket Connections (HTTP/1.1) Establish WebSocket connections with the `ws` feature enabled. Uses futures-based stream/sink API for sending and receiving messages. ```rust use futures_util::TryStreamExt; use hpx::{header, ws::message::Message, Result}; #[tokio::main] async fn main() -> Result<()> { let resp = hpx::websocket("wss://echo.websocket.org") .header(header::USER_AGENT, "my-app") .send() .await?; assert_eq!(resp.version(), http::Version::HTTP_11); let websocket = resp.into_websocket().await?; if let Some(protocol) = websocket.protocol() { println!("WebSocket subprotocol: {:?}", protocol); } let (mut tx, mut rx) = websocket.split(); // Send messages in a separate task tokio::spawn(async move { for i in 1..11 { if let Err(err) = tx.send(Message::text(format!("Hello, World! {i}"))).await { eprintln!("Failed to send message: {err}"); } } }); // Receive messages while let Some(message) = rx.try_next().await? { if let Message::Text(text) = message { println!("Received: {text}"); } } Ok(()) } ``` ## Cookie Management Enable automatic cookie storage and sending with the `cookies` feature. Cookies persist across requests within the same client session. ```rust use hpx::{Client, Result, cookie::Jar}; #[tokio::main] async fn main() -> Result<()> { let jar = Jar::default(); let client = Client::builder() .cookie_provider(jar.clone()) .build()?; // This request sets a cookie let _resp = client .get("https://httpbin.org/cookies/set/session/123") .send() .await?; // Cookies are automatically sent with subsequent requests println!("Stored cookies: {:?}", jar.cookies("https://httpbin.org")); Ok(()) } ``` ## Form Data Submission Send URL-encoded form data with the `form` feature. Automatically sets `Content-Type: application/x-www-form-urlencoded`. ```rust use std::collections::HashMap; use hpx::{Client, Result}; #[tokio::main] async fn main() -> Result<()> { let mut params = HashMap::new(); params.insert("username", "admin"); params.insert("password", "secret123"); let client = Client::new(); let res = client .post("http://httpbin.org/post") .form(¶ms) .send() .await?; println!("Response status: {}", res.status()); Ok(()) } ``` ## Authentication Methods Multiple authentication methods are supported: basic auth, bearer tokens, and custom authorization headers. ```rust use hpx::{Client, Result}; #[tokio::main] async fn main() -> Result<()> { let client = Client::new(); // Basic authentication let resp = client .delete("http://httpbin.org/delete") .basic_auth("admin", Some("good_password")) .send() .await?; // Bearer token authentication let resp = client .get("http://httpbin.org/get") .bearer_auth("your_jwt_token_here") .send() .await?; // Custom authorization header let resp = client .get("http://httpbin.org/get") .auth("CustomToken abc123") .send() .await?; Ok(()) } ``` ## Proxy Configuration Configure HTTP, HTTPS, and SOCKS proxies. The `socks` feature enables SOCKS4/SOCKS5 proxy support. ```rust use hpx::{Client, Proxy, Result}; #[tokio::main] async fn main() -> Result<()> { // HTTP proxy let proxy = Proxy::http("http://proxy.example.com:8080")?; // SOCKS5 proxy (requires `socks` feature) let socks_proxy = Proxy::all("socks5://127.0.0.1:1080")?; let client = Client::builder() .proxy(socks_proxy) .build()?; let resp = client.get("https://api.example.com").send().await?; println!("Status: {}", resp.status()); Ok(()) } ``` ## Rotating Proxy Pool Use multiple proxies with rotation strategies for load distribution or failover scenarios. ```rust use hpx::{Client, Proxy, ProxyPool, ProxyPoolStrategy, Result}; #[tokio::main] async fn main() -> Result<()> { let proxies = vec![ Proxy::all("http://proxy1.example.com:8080")?, Proxy::all("http://proxy2.example.com:8080")?, Proxy::all("http://proxy3.example.com:8080")?, ]; // Round-robin rotation let pool = ProxyPool::with_strategy(proxies, ProxyPoolStrategy::RoundRobin)?; // Or sticky failover (use first until it fails) // let pool = ProxyPool::with_strategy(proxies, ProxyPoolStrategy::StickyFailover)?; let client = Client::builder() .proxy_pool(pool) .build()?; let resp = client.get("https://api.example.com").send().await?; Ok(()) } ``` ## Exchange REST Client (hpx-transport) The `hpx-transport` crate provides a high-level REST client for cryptocurrency exchange APIs with built-in authentication support. ```rust use hpx_transport::{ auth::ApiKeyAuth, exchange::{ExchangeClient, RestClient, RestConfig}, TypedResponse, }; use serde::Deserialize; use std::time::Duration; #[derive(Debug, Deserialize)] struct AccountInfo { balance: f64, currency: String, } #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let config = RestConfig::new("https://api.exchange.com") .timeout(Duration::from_secs(30)) .user_agent("my-trading-bot/1.0"); let auth = ApiKeyAuth::header("X-API-Key", "your-api-key-here"); let client = RestClient::new(config, auth)?; // GET request with typed response let response: TypedResponse<AccountInfo> = client.get("/v1/account").await?; println!("Balance: {} {}", response.data.balance, response.data.currency); println!("Latency: {:?}", response.latency); Ok(()) } ``` ## HMAC Authentication for Exchanges Use HMAC-SHA256/SHA512 signing for exchanges like Binance, OKX that require request signatures. ```rust use hpx_transport::{ auth::{HmacAuth, HmacAlgorithm}, exchange::{ExchangeClient, RestClient, RestConfig}, }; use serde::{Deserialize, Serialize}; #[derive(Serialize)] struct OrderParams { symbol: String, side: String, quantity: f64, } #[derive(Deserialize)] struct OrderResponse { order_id: String, status: String, } #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { // Binance-style HMAC authentication let auth = HmacAuth::new("your-api-key", "your-secret-key") .algorithm(HmacAlgorithm::Sha256) .api_key_header("X-MBX-APIKEY") .signature_param("signature") .timestamp_param("timestamp") .timestamp_in_query(true); let config = RestConfig::new("https://api.binance.com"); let client = RestClient::new(config, auth)?; let order = OrderParams { symbol: "BTCUSDT".to_string(), side: "BUY".to_string(), quantity: 0.001, }; let response: hpx_transport::TypedResponse<OrderResponse> = client.post("/api/v3/order", &order).await?; println!("Order ID: {}", response.data.order_id); Ok(()) } ``` ## WebSocket Connection with Protocol Handler (hpx-transport) Establish WebSocket connections with request-response correlation and subscription management for exchange APIs. ```rust use hpx_transport::websocket::{ Connection, WsConfig, Event, handlers::GenericJsonHandler, }; use std::time::Duration; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let config = WsConfig::new("wss://api.exchange.com/ws") .ping_interval(Duration::from_secs(30)) .request_timeout(Duration::from_secs(10)); let handler = GenericJsonHandler::new(); let connection = Connection::connect_stream(config, handler).await?; let (handle, mut stream) = connection.split(); // Subscribe to a topic let mut guard = handle.subscribe("orderbook.BTC").await?; // Receive subscription updates tokio::spawn(async move { while let Some(msg) = guard.recv().await { println!("Orderbook update: {:?}", msg); } }); // Process stream events while let Some(event) = stream.next().await { match event { Event::Message(msg) => println!("Message: {:?}", msg.kind), Event::Connected(epoch) => println!("Connected, epoch: {:?}", epoch), Event::Disconnected(reason) => println!("Disconnected: {:?}", reason), } } Ok(()) } ``` ## Rate Limiting Token bucket rate limiter for managing API request rates to avoid hitting exchange rate limits. ```rust use hpx_transport::rate_limit::RateLimiter; use std::time::Duration; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let limiter = RateLimiter::new(); // 10 requests capacity, refills at 1 token per second limiter.add_limit("orders", 10, 1.0)?; // 100 requests capacity, refills at 10 tokens per second limiter.add_limit("public", 100, 10.0)?; // Non-blocking check if limiter.try_acquire("orders") { println!("Request allowed, making order..."); } else { let wait = limiter.time_until_available("orders"); println!("Rate limited, wait: {:?}", wait); } // Blocking acquire (waits until token available) limiter.acquire("orders").await; println!("Token acquired after waiting"); // Check available tokens if let Some(tokens) = limiter.available_tokens("orders") { println!("Available tokens: {:.1}", tokens); } Ok(()) } ``` ## hpx-yawc WebSocket Client Direct WebSocket connections using the low-level `hpx-yawc` crate with RFC 6455 compliance and permessage-deflate compression. ```rust use futures::{SinkExt, StreamExt}; use hpx_yawc::{WebSocket, frame::OpCode}; #[tokio::main] async fn main() -> hpx_yawc::Result<()> { let mut ws = WebSocket::connect("wss://echo.websocket.org".parse()?).await?; // Send a text message let frame = hpx_yawc::frame::Frame::text("Hello, WebSocket!"); ws.send(frame).await?; // Echo server: receive and echo back messages while let Some(frame) = ws.next().await { match frame.opcode() { OpCode::Text | OpCode::Binary => { println!("Received: {:?}", frame); ws.send(frame).await?; // Echo back } OpCode::Ping => { // Pong is sent automatically, but ping frame is still returned println!("Received ping"); } OpCode::Close => { println!("Connection closed"); break; } _ => {} } } Ok(()) } ``` ## Timeout Configuration Fine-grained timeout control for different phases of the request lifecycle. ```rust use hpx::{Client, Result}; use std::time::Duration; #[tokio::main] async fn main() -> Result<()> { let client = Client::builder() .timeout(Duration::from_secs(30)) // Total request timeout .connect_timeout(Duration::from_secs(5)) // TCP + TLS handshake .build()?; // Per-request timeout overrides let resp = client .get("https://api.example.com/slow-endpoint") .timeout(Duration::from_secs(60)) .read_timeout(Duration::from_secs(10)) .timeout_connect(Some(Duration::from_secs(3))) .send() .await?; Ok(()) } ``` ## Multipart Form Upload Upload files and form data using multipart/form-data encoding with the `multipart` feature. ```rust use hpx::{Client, multipart, Result}; #[tokio::main] async fn main() -> Result<()> { let form = multipart::Form::new() .text("username", "john_doe") .text("description", "Profile picture upload") .file("avatar", "/path/to/image.png")?; let client = Client::new(); let response = client .post("https://api.example.com/upload") .multipart(form) .send() .await?; println!("Upload status: {}", response.status()); Ok(()) } ``` ## Custom Emulation Options Configure advanced browser emulation settings including operating system, HTTP/2 settings, and header behavior. ```rust use hpx::{Client, Result}; use hpx_emulation::{Emulation, EmulationOption, EmulationOS}; #[tokio::main] async fn main() -> Result<()> { // Full control over emulation settings let emulation_opts = EmulationOption::builder() .emulation(Emulation::Chrome143) .emulation_os(EmulationOS::MacOS) .skip_http2(false) .skip_headers(false) .build(); let client = Client::builder() .emulation(emulation_opts) .build()?; let resp = client .get("https://tls.peet.ws/api/all") .send() .await?; println!("Fingerprint: {}", resp.text().await?); Ok(()) } ``` hpx is designed for high-performance trading applications where network latency and reliability are critical. The library uses lock-free data structures (`scc::HashMap`, `arc-swap`) for hot-path operations, SIMD-accelerated JSON parsing with the `simd-json` feature, and async DNS resolution with `hickory-dns` to avoid blocking system calls. The architecture supports Tower middleware for adding cross-cutting concerns like retry logic, rate limiting, and request/response hooks. For cryptocurrency exchange integrations, the `hpx-transport` crate provides a complete SDK with authentication strategies (API key, HMAC signing, Bearer tokens), WebSocket connection management with auto-reconnection, subscription handling, and OpenTelemetry metrics integration. The WebSocket implementation uses a single-owner connection loop with lock-free stores for pending requests and subscriptions, enabling high-throughput market data streaming.