Stealthscraper Rs

https://github.com/ypno/stealthscraper-rs
Admin
Stealthscraper Rs is a stealthy web scraping library for Rust providing TLS JA4 fingerprint...
Tokens:57,739
Snippets:409
Trust Score:6.5
License:MIT
Update:1 week ago
Show doc for...
Tokens:
Raw
### Run fingerprint capture examples

Source: https://github.com/ypno/stealthscraper-rs/blob/main/README.md

Run these cargo commands to capture TLS, HTTP/2, and client-hint fingerprints. Each prints values ready to paste into src/emulation.rs or src/client_hints.rs. Point a browser at the listener each prints, including one on another machine; capture_hints also prints a console snippet for machines without a Rust toolchain.

```bash
cargo run --example capture_fingerprint          # TLS ClientHello → JA4
cargo run --example capture_h2                   # HTTP/2 SETTINGS, window, pseudo order
cargo run --example capture_h2 -- --h1           # header names and order, as plaintext
cargo run --example capture_hints --features browser   # User-Agent Client Hints
cargo run --example emulation_roundtrip          # prove an entry reproduces its browser
```

--------------------------------

### Add stealthscraper-rs dependency to Cargo.toml

Source: https://github.com/ypno/stealthscraper-rs/blob/main/README.md

Declares the `stealthscraper-rs` dependency in Cargo.toml. The `browser` feature must be enabled to use the headless-browser API (`CloudScraper`) in the examples.

```toml
[dependencies]
stealthscraper-rs = { version = "0.1", features = ["browser"] }
```

--------------------------------

### Build CloudScraper with geo proxies, rotation, and state store

Source: https://github.com/ypno/stealthscraper-rs/blob/main/README.md

Configures a CloudScraper with geo-tagged proxies, a round-robin rotation strategy, a maximum of 3 challenge attempts, an in-memory state store for per-domain outcomes and cooldowns, and a log event sink. Requires the `persistence` feature for durable state using `RedbStateStore::open("state.redb")?`.

```rust
use stealthscraper_rs::{CloudScraper, CountryCode, RotationStrategy, InMemoryStateStore, LogEventSink};
use std::sync::Arc;

let scraper = CloudScraper::builder()
    // Geo-tagged residential proxies — locale is derived per exit country.
    .with_geo_proxies([
        ("http://user:pass@de.proxy:8080".to_string(), CountryCode::new("DE").unwrap()),
        ("http://user:pass@fr.proxy:8080".to_string(), CountryCode::new("FR").unwrap()),
    ])
    .proxy_strategy(RotationStrategy::RoundRobin)
    .with_max_challenge_attempts(3)
    // Remember per-domain outcomes & rate-limit cooldowns (in-memory here).
    .with_state_store(Arc::new(InMemoryStateStore::new()))
    // Stream scrape events (challenge detected, proxy rotated, …) to the `log` crate.
    .with_event_sink(Arc::new(LogEventSink))
    .build()
    .await?;
```

--------------------------------

### No-browser default build: impersonation_client with BrowserProfile::random()

Source: https://github.com/ypno/stealthscraper-rs/blob/main/README.md

Use when the target endpoint gates on the TLS/HTTP-2 fingerprint alone and serves no JavaScript challenge. The returned wreq::ClientBuilder can still be extended with a proxy, cookie jar, or redirect policy; wreq is re-exported as stealthscraper_rs::wreq to keep the same major version as the measured emulation.

```rust
use stealthscraper_rs::{BrowserProfile, impersonation_client};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Carries the measured TLS + HTTP/2 emulation, and the User-Agent,
    // Sec-CH-UA* and Accept-Language of the same profile.
    let client = impersonation_client(&BrowserProfile::random()).build()?;

    let body = client
        .get("https://target-website.com")
        .send()
        .await?
        .text()
        .await?;
    println!("{} bytes", body.len());
    Ok(())
}

```

--------------------------------

### Rotate browser profile with rotate_profile

Source: https://github.com/ypno/stealthscraper-rs/blob/main/README.md

Rotates to a fresh fingerprint when the browser identity is burned, not just the IP. This relaunches Chrome but keeps the MITM port and egress proxy. The method consumes `self` and returns a fresh scraper.

```rust
let scraper = scraper.rotate_profile().await?; // consumes self, returns a fresh scraper
```

--------------------------------

### Cargo.toml dependency edit: tokio-rustls with ring provider and rcgen

Source: https://github.com/ypno/stealthscraper-rs/blob/main/docs/implementation-plan-lean-stealth.md

Replace the tokio-rustls dependency to disable default features and enable ring, logging, and tls12. The rcgen line keeps its default features; the aws_lc_rs feature was additive and is dropped. Verified with cargo check on all three feature combos and cargo test --features persistence (65 passed).

```toml
tokio-rustls = { version = "0.26.4", default-features = false, features = ["ring", "logging", "tls12"] }
rcgen        = "0.14.8"   # default = ["crypto","pem","ring"]; the aws_lc_rs feature was additive
```

--------------------------------

### StealthIdentity struct definition

Source: https://github.com/ypno/stealthscraper-rs/blob/main/docs/implementation-plan-lean-stealth.md

Portable, serializable identity shared by both transports so switching between Chrome and HTTP is lossless. Holds the browser profile, emulation kind, locale, cookies, client hints, sticky egress proxy, and cf_clearance expiry that drives re-escalation.

```rust
StealthIdentity {
    profile: BrowserProfile,            // UA, platform, HW, WebGL, viewport
    emulation: BrowserKind,             // JA4/H2 source (§2)
    locale: Option<Locale>,             // proxy-led (existing)
    cookies: Vec<Cookie>,               // domain/path/expiry/secure/httponly/samesite
    client_hints: ClientHints,          // Sec-CH-UA{,-platform,-mobile,-arch,...}
    egress: Option<String>,             // sticky upstream proxy (same exit IP!)
    cf_clearance_expiry: Option<u64>,   // drives re-escalation
}
```
Tokens:
Raw