### Example: Building a Policy with Blocked CIDR Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Demonstrates how to use PolicyBuilder to create a custom policy that blocks a specific IP range. Requires importing PolicyBuilder and Policy. ```rust use url_jail::{PolicyBuilder, Policy}; let policy = PolicyBuilder::new(Policy::AllowPrivate) .block_cidr("10.0.0.0/8") .build(); ``` -------------------------------- ### Example Usage of ValidateOptions Source: https://docs.rs/url_jail/latest/url_jail/struct.ValidateOptions.html Demonstrates how to create and use `ValidateOptions` with the `validate_with_options` function to customize URL validation, setting a specific DNS timeout. ```rust use url_jail::{validate_with_options, Policy, ValidateOptions}; use std::time::Duration; let opts = ValidateOptions { dns_timeout: Duration::from_secs(5), }; let result = validate_with_options("https://example.com/", Policy::PublicOnly, opts).await?; ``` -------------------------------- ### Create New PolicyBuilder Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Initializes a new PolicyBuilder with a specified base policy. This is the starting point for defining custom network policies. ```rust pub fn new(base: Policy) -> Self ``` -------------------------------- ### Getting the URL Path Source: https://docs.rs/url_jail/latest/url_jail/struct.SafeUrl.html Retrieves the path component of the URL. Returns '/' if no path is specified. ```rust pub fn path(&self) -> &str ``` -------------------------------- ### Validate a URL with PublicOnly Policy Source: https://docs.rs/url_jail/latest/url_jail/fn.validate.html This example demonstrates how to use the `validate` function to check if a URL is safe to connect to when only public IP addresses are allowed. It requires importing the `validate` function and the `Policy` enum from the `url_jail` crate. The result, if successful, contains the host and IP address. ```rust use url_jail::{validate, Policy}; let result = validate("https://example.com/api", Policy::PublicOnly).await?; println!("Safe to connect to {} ({})", result.host, result.ip); ``` -------------------------------- ### Synchronous URL Validation Example Source: https://docs.rs/url_jail/latest/url_jail/fn.validate_sync.html Demonstrates how to use the `validate_sync` function to validate a URL against a public-only policy. This function blocks the current thread during DNS resolution. It returns a Result containing either the validated URL information or an error. ```rust use url_jail::{validate_sync, Policy}; let result = validate_sync("https://example.com/api", Policy::PublicOnly)?; println!("Safe to connect to {} ({})", result.host, result.ip); ``` -------------------------------- ### Get URL Path Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Retrieves the path component of the SafeUrl. Returns '/' if no path is specified. ```rust pub fn path(&self) -> &str { self.inner.path() } ``` -------------------------------- ### Custom Policy with Block and Allow CIDRs Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Example of creating a custom policy using PolicyBuilder. It blocks a specific CIDR range but allows a sub-range within it, demonstrating precedence rules. The policy is immutable after building. ```rust use url_jail::{PolicyBuilder, Policy}; let policy = PolicyBuilder::new(Policy::AllowPrivate) .block_cidr("10.0.0.0/8") .allow_cidr("10.1.0.0/16") // This takes precedence .build(); // 10.1.x.x is allowed (matches allow rule) assert!(policy.is_ip_allowed("10.1.2.3".parse().unwrap()).is_ok()); // 10.2.x.x is blocked (matches block rule, no allow rule) assert!(policy.is_ip_allowed("10.2.0.1".parse().unwrap()).is_err()); ``` -------------------------------- ### Match hostname against pattern with wildcard Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Checks if a given hostname matches a pattern that supports the '*' wildcard. Handles patterns starting with '*.' for subdomains. ```rust fn matches_hostname_pattern(host: &str, pattern: &str) -> bool { if pattern.starts_with("*.") { let suffix = &pattern[1..]; // ".example.com" host.ends_with(suffix) || host == &pattern[2..] } else { host == pattern } } ``` -------------------------------- ### Test Block IPv6 Link-Local Address Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Verifies that IPv6 link-local addresses (starting with `fe80::`) are blocked. ```rust #[tokio::test] async fn test_block_ipv6_link_local() { let result = validate("http://[fe80::1]/", Policy::PublicOnly).await; assert!(result.is_err()); } ``` -------------------------------- ### Getting the Full Normalized URL String Source: https://docs.rs/url_jail/latest/url_jail/struct.SafeUrl.html Converts the SafeUrl back into its full normalized string representation, including scheme, host, port (if non-default), path, query, and fragment. ```rust pub fn as_str(&self) -> &str ``` -------------------------------- ### Parse URL with Hostname Starting with Hyphen Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Checks parsing of a URL with a hostname that starts with a hyphen. This is an invalid DNS name but might be accepted by URL parsers without panicking. ```rust let result = SafeUrl::parse("http://-invalid.example.com/"); // Either accept or reject is fine, just don't panic let _ = result; ``` -------------------------------- ### Creating and Using a CustomPolicy Source: https://docs.rs/url_jail/latest/url_jail/struct.CustomPolicy.html Demonstrates how to build a CustomPolicy with specific allow and block rules for IP addresses. Allow rules take precedence over block rules. ```rust use url_jail::{PolicyBuilder, Policy}; let policy = PolicyBuilder::new(Policy::AllowPrivate) .block_cidr("10.0.0.0/8") .allow_cidr("10.1.0.0/16") // This takes precedence .build(); // 10.1.x.x is allowed (matches allow rule) assert!(policy.is_ip_allowed("10.1.2.3".parse().unwrap()).is_ok()); // 10.2.x.x is blocked (matches block rule, no allow rule) assert!(policy.is_ip_allowed("10.2.0.1".parse().unwrap()).is_err()); ``` -------------------------------- ### Wildcard Hostname Matches Exact Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Confirms that a wildcard pattern like '*.example.com' also blocks the exact hostname 'example.com'. ```Rust #[test] fn test_block_host_wildcard_matches_exact() { // *.example.com should also match example.com itself let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("*.example.com") .build(); assert!(policy.is_hostname_allowed("sub.example.com").is_err()); assert!(policy.is_hostname_allowed("example.com").is_err()); } ``` -------------------------------- ### Getting the Port Number Source: https://docs.rs/url_jail/latest/url_jail/struct.SafeUrl.html Retrieves the port number from a SafeUrl. Defaults to 80 for http and 443 for https if not explicitly specified. ```rust pub fn port(&self) -> u16 ``` -------------------------------- ### Handle DNS Rebinding Hostnames (nip.io style) Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Demonstrates parsing URLs with hostnames that resolve to IPs, like nip.io. Parsing succeeds, but DNS validation is required to catch these. ```rust let url = SafeUrl::parse("http://127.0.0.1.nip.io/").unwrap(); assert_eq!(url.host(), "127.0.0.1.nip.io"); ``` -------------------------------- ### PolicyBuilder::new Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Initializes a new PolicyBuilder with a specified base policy. ```APIDOC ## PolicyBuilder::new ### Description Creates a new builder with the given base policy. ### Method ```rust pub fn new(base: Policy) -> Self ``` ### Parameters * **base** (Policy) - The base policy to initialize the builder with. ``` -------------------------------- ### Create a new PolicyBuilder Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Initializes a new PolicyBuilder with a specified base policy. Uses Default::default() for other fields. ```rust pub fn new(base: Policy) -> Self { Self { base, ..Default::default() } } ``` -------------------------------- ### Get Port Number Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Retrieves the port number for a SafeUrl. It returns the explicitly specified port or defaults to 80 for http and 443 for https. ```rust pub fn port(&self) -> u16 { self.inner.port_or_known_default().unwrap_or(80) } ``` -------------------------------- ### Handle DNS Rebinding Hostnames (xip.io style) Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Similar to nip.io, this snippet shows parsing URLs with xip.io style hostnames. Parsing should succeed, relying on subsequent DNS validation. ```rust let url = SafeUrl::parse("http://192.168.1.1.xip.io/").unwrap(); assert_eq!(url.host(), "192.168.1.1.xip.io"); ``` -------------------------------- ### Getting the Normalized Hostname Source: https://docs.rs/url_jail/latest/url_jail/struct.SafeUrl.html Retrieves the normalized hostname from a SafeUrl. The hostname is lowercased and has any trailing dot removed. For IPv6 addresses, brackets are included. ```rust pub fn host(&self) -> &str ``` -------------------------------- ### Allow Wildcard Hostname Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Shows how to allow hostnames matching a wildcard pattern, while still blocking others. ```Rust #[test] fn test_allow_host_wildcard() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("*.example.com") .allow_host("*.trusted.example.com") .build(); assert!(policy .is_hostname_allowed("api.trusted.example.com") .is_ok()); assert!(policy.is_hostname_allowed("other.example.com").is_err()); } ``` -------------------------------- ### Parse URLs with Hostnames Containing Numbers Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that URLs with hostnames that include numbers, including those starting with numbers or having numbers in the TLD, are parsed correctly. ```rust let url = SafeUrl::parse("http://host123.example.com/").unwrap(); assert_eq!(url.host(), "host123.example.com"); ``` ```rust let url = SafeUrl::parse("http://123host.example.com/").unwrap(); assert_eq!(url.host(), "123host.example.com"); ``` ```rust let url = SafeUrl::parse("http://example.co2/").unwrap(); assert_eq!(url.host(), "example.co2"); ``` ```rust let url = SafeUrl::parse("http://0177.example.com/").unwrap(); assert_eq!(url.host(), "0177.example.com"); ``` -------------------------------- ### Test Wildcard Matches Base Hostname Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Verifies that a wildcard pattern like '*.example.com' also matches the base domain 'example.com'. ```rust #[test] fn test_matches_hostname_pattern_wildcard_matches_base() { // *.example.com should also match example.com assert!(matches_hostname_pattern("example.com", "*.example.com")); } ``` -------------------------------- ### Parse URL with Fragment and Access Host Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Demonstrates parsing a URL that includes a fragment identifier and verifying its host. ```rust let url = SafeUrl::parse("https://example.com/path#fragment").unwrap(); assert_eq!(url.host(), "example.com"); ``` -------------------------------- ### Custom URL Validation with PolicyBuilder Source: https://docs.rs/url_jail/latest/url_jail/fn.validate_custom.html Demonstrates how to create a custom policy that blocks a specific CIDR range and then use it to validate a URL. Requires importing `validate_custom`, `PolicyBuilder`, and `Policy` from the url_jail crate. ```rust use url_jail::{validate_custom, PolicyBuilder, Policy}; let policy = PolicyBuilder::new(Policy::AllowPrivate) .block_cidr("10.0.0.0/8") .build(); let result = validate_custom("https://example.com/", &policy).await?; ``` -------------------------------- ### Block Hostname Pattern Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates blocking hostnames that match a wildcard pattern, ensuring subdomains are also blocked. ```Rust #[test] fn test_block_host_pattern() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("*.internal.example.com") .build(); assert!(policy .is_hostname_allowed("api.internal.example.com") .is_err()); assert!(policy.is_hostname_allowed("api.example.com").is_ok()); } ``` -------------------------------- ### Verify DNS Resolution Path Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Checks that the DNS resolution and verification code path is executed for a valid domain like example.com. This test implicitly relies on other tests for actual multi-IP blocking logic. ```Rust #[tokio::test] async fn test_multi_ip_all_checked() { // We can't easily control DNS results, but we can verify the code path // by checking that resolve_and_verify_dns exists and is used // The actual multi-IP check is implicitly tested by the blocklist tests let result = validate("https://example.com/", Policy::PublicOnly).await; assert!(result.is_ok()); } ``` -------------------------------- ### Block Multiple Hostnames Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Shows how to apply multiple distinct hostname block rules to a policy. ```Rust #[test] fn test_block_multiple_hosts() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("*.internal.com") .block_host("*.private.org") .build(); assert!(policy.is_hostname_allowed("api.internal.com").is_err()); assert!(policy.is_hostname_allowed("db.private.org").is_err()); assert!(policy.is_hostname_allowed("public.example.com").is_ok()); } ``` -------------------------------- ### Allow Specific Hostname Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates allowing a specific hostname, overriding any general blocking rules. ```Rust #[test] fn test_allow_host_pattern() { let policy = PolicyBuilder::new(Policy::PublicOnly) .allow_host("trusted.internal") .build(); assert!(policy.is_hostname_allowed("trusted.internal").is_ok()); } ``` -------------------------------- ### Test Policy Builder Chaining Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates constructing a complex policy by chaining builder methods for blocking/allowing CIDRs and hostnames. ```rust #[test] fn test_builder_chaining() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_cidr("10.0.0.0/8") .allow_cidr("10.1.0.0/16") .block_host("*.internal.com") .allow_host("trusted.internal.com") .build(); // IP checks assert!(policy.is_ip_allowed("10.1.2.3".parse().unwrap()).is_ok()); assert!(policy.is_ip_allowed("10.2.0.1".parse().unwrap()).is_err()); // Hostname checks assert!(policy.is_hostname_allowed("trusted.internal.com").is_ok()); assert!(policy.is_hostname_allowed("other.internal.com").is_err()); } ``` -------------------------------- ### DANGEROUS: Allow Localhost Access Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates a dangerous use of allow_* methods, which can override base policy blocks and defeat SSRF protection. Use with extreme caution and only for audited use cases. ```rust use url_jail::{PolicyBuilder, Policy}; // DANGEROUS: This allows localhost access! let bad_policy = PolicyBuilder::new(Policy::PublicOnly) .allow_cidr("127.0.0.0/8") // Defeats SSRF protection! .build(); ``` -------------------------------- ### Allow CIDR Precedence Over Block Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Illustrates that a more specific 'allow' rule takes precedence over a broader 'block' rule for IP addresses. ```Rust #[test] fn test_allow_takes_precedence_over_block() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_cidr("10.0.0.0/8") .allow_cidr("10.1.0.0/16") .build(); // 10.1.x.x is allowed despite 10.0.0.0/8 being blocked assert!(policy.is_ip_allowed("10.1.2.3".parse().unwrap()).is_ok()); // Other 10.x.x.x still blocked assert!(policy.is_ip_allowed("10.2.0.1".parse().unwrap()).is_err()); } ``` -------------------------------- ### Allow Specific Private IP Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates allowing a specific private IP address that would otherwise be blocked by a default policy. ```Rust #[test] fn test_allow_private_ip() { let policy = PolicyBuilder::new(Policy::PublicOnly) .allow_cidr("192.168.1.50/32") .build(); assert!(policy .is_ip_allowed("192.168.1.50".parse().unwrap()) .is_ok()); // Other private IPs still blocked by base policy assert!(policy .is_ip_allowed("192.168.2.1".parse().unwrap()) .is_err()); } ``` -------------------------------- ### Customizing URL Validation Options Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Demonstrates how to set custom DNS resolution timeouts for URL validation using ValidateOptions. ```rust use url_jail::{validate_with_options, Policy, ValidateOptions}; use std::time::Duration; # async fn example() -> Result<(), url_jail::Error> { let opts = ValidateOptions { dns_timeout: Duration::from_secs(5), }; let result = validate_with_options("https://example.com/", Policy::PublicOnly, opts).await?; # Ok(()) # } ``` -------------------------------- ### Handle IPv6 with Zone ID Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Demonstrates parsing an IPv6 address that includes a zone ID (e.g., fe80::1%eth0). The URL crate might parse this, and the host should contain the zone ID. Rejection is also acceptable. ```rust let result = SafeUrl::parse("http://[fe80::1%25eth0]/"); match result { Ok(url) => { assert!(url.host().contains("fe80")); } Err(_) => {} // Rejection is fine } ``` -------------------------------- ### Handle IPv6 with Embedded IPv4 Loopback Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Shows parsing of an IPv6 address in the format ::127.0.0.1. The parsed host should indicate the presence of 127.0.0.1 or ::. ```rust let result = SafeUrl::parse("http://[::127.0.0.1]/"); match result { Ok(url) => { let host = url.host(); assert!(host.contains("127.0.0.1") || host.contains("::")); } Err(_) => {} // Rejection is fine } ``` -------------------------------- ### Custom URL Policy Builder Source: https://docs.rs/url_jail/latest/index.html Demonstrates creating a custom URL validation policy using `PolicyBuilder`. Allows for fine-grained control by blocking and allowing specific CIDR ranges or hosts. Builds upon `Policy::AllowPrivate`. ```rust use url_jail::{PolicyBuilder, Policy}; let policy = PolicyBuilder::new(Policy::AllowPrivate) .block_cidr("10.0.0.0/8") // Block specific range .allow_cidr("10.1.0.0/16") // But allow a subnet .block_host("*.internal.example.com") .build(); ``` -------------------------------- ### Generic Implementations for Validated Source: https://docs.rs/url_jail/latest/url_jail/struct.Validated.html Shows various blanket implementations for traits like Any, Borrow, CloneToUninit, From, Instrument, Into, Pointable, ToOwned, TryFrom, TryInto, VZip, and WithSubscriber. ```rust fn type_id(&self) -> TypeId ``` ```rust fn borrow(&self) -> &T ``` ```rust fn borrow_mut(&mut self) -> &mut T ``` ```rust unsafe fn clone_to_uninit(&self, dest: *mut u8) ``` ```rust fn from(t: T) -> T ``` ```rust fn instrument(self, span: Span) -> Instrumented ``` ```rust fn in_current_span(self) -> Instrumented ``` ```rust fn into(self) -> U ``` ```rust const ALIGN: usize ``` ```rust type Init = T ``` ```rust unsafe fn init(init: ::Init) -> usize ``` ```rust unsafe fn deref<'a>(ptr: usize) -> &'a T ``` ```rust unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T ``` ```rust unsafe fn drop(ptr: usize) ``` ```rust type Owned = T ``` ```rust fn to_owned(&self) -> T ``` ```rust fn clone_into(&self, target: &mut T) ``` ```rust type Error = Infallible ``` ```rust fn try_from(value: U) -> Result>::Error> ``` ```rust type Error = >::Error ``` ```rust fn try_into(self) -> Result>::Error> ``` ```rust fn vzip(self) -> V ``` ```rust fn with_subscriber(self, subscriber: S) -> WithDispatch ``` ```rust fn with_current_subscriber(self) -> WithDispatch ``` -------------------------------- ### Parse URL with Empty Query and Access Host Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing a URL with an empty query string and confirms the host is correctly extracted. ```rust let url = SafeUrl::parse("https://example.com/path?").unwrap(); assert_eq!(url.host(), "example.com"); ``` -------------------------------- ### Allow Standard IPv4 Address with Port Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL with a standard IPv4 address and a custom port is parsed successfully. ```rust let url = SafeUrl::parse("http://192.168.1.1:8080/").unwrap(); assert_eq!(url.host(), "192.168.1.1"); assert_eq!(url.port(), 8080); ``` -------------------------------- ### Reject Userinfo with IP Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests the rejection of URLs containing userinfo (e.g., 'safe.com@') before an IP address, which is a security risk. ```rust #[test] fn test_redteam_at_symbol_userinfo_with_ip() { // Attempt: user thinks they're going to safe.com, but @ makes 127.0.0.1 the real host let result = SafeUrl::parse("http://safe.com@127.0.0.1/"); // This MUST be rejected - we block userinfo assert!(result.is_err(), "Userinfo with @ must be rejected"); } ``` -------------------------------- ### Allow IPv6 Address with Port Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL with an IPv6 address and a custom port is parsed successfully. ```rust let url = SafeUrl::parse("http://[::1]:8080/").unwrap(); assert_eq!(url.host(), "[::1]"); assert_eq!(url.port(), 8080); ``` -------------------------------- ### Block Exact Hostname Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Shows how to block a specific hostname exactly, without affecting subdomains or other hostnames. ```Rust #[test] fn test_block_host_exact() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("blocked.example.com") .build(); assert!(policy.is_hostname_allowed("blocked.example.com").is_err()); assert!(policy.is_hostname_allowed("other.example.com").is_ok()); // Subdomain should NOT match exact hostname block assert!(policy .is_hostname_allowed("sub.blocked.example.com") .is_ok()); } ``` -------------------------------- ### Test Red Team Allow Private Blocks Link Local Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Verifies that the AllowPrivate policy still blocks link-local IP addresses. ```rust #[tokio::test] async fn test_redteam_allow_private_blocks_link_local() { let result = validate("http://169.254.1.1/", Policy::AllowPrivate).await; assert!(result.is_err(), "AllowPrivate must still block link-local"); } ``` -------------------------------- ### Parse URL with Empty Fragment and Access Host Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Verifies that a URL with an empty fragment identifier is parsed correctly, and its host is accessible. ```rust let url = SafeUrl::parse("https://example.com/path#").unwrap(); assert_eq!(url.host(), "example.com"); ``` -------------------------------- ### Allow Hostname Precedence Over Block Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Illustrates that a specific 'allow' rule for a hostname takes precedence over a broader 'block' rule. ```Rust #[test] fn test_allow_host_takes_precedence_over_block() { let policy = PolicyBuilder::new(Policy::PublicOnly) .block_host("*.example.com") .allow_host("trusted.example.com") .build(); // trusted.example.com is allowed despite *.example.com being blocked assert!(policy.is_hostname_allowed("trusted.example.com").is_ok()); // Other subdomains still blocked assert!(policy.is_hostname_allowed("blocked.example.com").is_err()); } ``` -------------------------------- ### Test Custom Validation with Options Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Tests the `validate_custom_with_options` function, demonstrating how to pass custom validation options like DNS timeouts. ```rust #[tokio::test] async fn test_validate_custom_with_options() { let policy = PolicyBuilder::new(Policy::PublicOnly).build(); let opts = ValidateOptions { dns_timeout: Duration::from_secs(10), }; let result = validate_custom_with_options("https://example.com/", &policy, opts).await; assert!(result.is_ok()); } ``` -------------------------------- ### Test URL Parsing Without Scheme Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Validates that URLs lacking a scheme (e.g., 'example.com' or '//example.com') are rejected. ```rust assert!(SafeUrl::parse("example.com").is_err()); assert!(SafeUrl::parse("//example.com").is_err()); ``` -------------------------------- ### PolicyBuilder::build Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Constructs and returns the final custom policy based on the builder's configuration. ```APIDOC ## PolicyBuilder::build ### Description Builds the custom policy. ### Method ```rust pub fn build(self) -> CustomPolicy ``` ### Returns * **CustomPolicy** - The constructed custom policy. ``` -------------------------------- ### Handle IPv4-Mapped IPv6 for Metadata Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing an IPv4-mapped IPv6 address, often used for metadata endpoints. Parsing should succeed, and IP validation should catch it. ```rust let url = SafeUrl::parse("http://[::ffff:169.254.169.254]/").unwrap(); assert!(url.host().contains("169.254") || url.host().contains("ffff")); ``` -------------------------------- ### Fetch with Redirect Validation Source: https://docs.rs/url_jail/latest/index.html Uses the `fetch` function from `url_jail` to validate each redirect in a chain. This provides the safest approach for handling redirects. Requires the `fetch` feature to be enabled. ```rust use url_jail::{fetch, Policy}; let result = fetch("https://example.com/", Policy::PublicOnly).await?; println!("Final response: {:?}", result.response.status()); println!("Redirect chain: {} hops", result.chain.len()); ``` -------------------------------- ### DNS Resolution and Policy Verification Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Resolves a hostname to an IP address with a timeout and verifies ALL resolved IPs against a given policy. Fails if any IP is blocked. ```rust /// Resolve a hostname to IP addresses with timeout, checking ALL against policy. /// Returns the first allowed IP, or an error if any IP is blocked. async fn resolve_and_verify_dns( host: &str, timeout: Duration, policy: Policy, ) -> Result { let host_str = host.trim_start_matches('[').trim_end_matches(']'); if let Ok(ip) = host_str.parse::() { // Literal IP - check directly if let Some(reason) = is_ip_blocked(ip, policy) { return Err(Error::ssrf_blocked(host, ip, reason)); } return Ok(ip); } let resolve_future = async { let resolver = TokioResolver::builder_tokio() .map_err(|e| Error::dns_error(host, e.to_string()))?; let response = resolver .lookup_ip(host) .await .map_err(|e| Error::dns_error(host, e.to_string()))?; let ips: Vec = response.iter().collect(); if ips.is_empty() { return Err(Error::dns_error(host, "no IP addresses found")); } // Check ALL resolved IPs - if ANY is blocked, fail // This prevents attackers from hiding a blocked IP among allowed ones for ip in &ips { if let Some(reason) = is_ip_blocked(*ip, policy) { return Err(Error::ssrf_blocked(host, *ip, reason)); } } // All IPs are safe, return the first one Ok(ips[0]) }; tokio::time::timeout(timeout, resolve_future) .await .map_err(|_| Error::Timeout { message: format!("DNS resolution for {} timed out after {:?}", host, timeout), })?; } ``` -------------------------------- ### Block Hostname or Pattern with PolicyBuilder Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Adds a rule to block a specific hostname or a pattern. Wildcards like '*.internal.example.com' are supported. This method can be chained. ```rust pub fn block_host(self, pattern: &str) -> Self ``` -------------------------------- ### Reject Userinfo with Metadata IP Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests the rejection of URLs attempting to access metadata services (e.g., 169.254.169.254) via userinfo confusion. ```rust #[test] fn test_redteam_at_symbol_userinfo_with_metadata() { // Attempt to access metadata via userinfo confusion let result = SafeUrl::parse("http://google.com@169.254.169.254/"); assert!(result.is_err(), "Userinfo must be rejected"); } ``` -------------------------------- ### Test Hostname Blocklist Source: https://docs.rs/url_jail/latest/src/url_jail/blocklist.rs.html Verifies that known internal hostnames are blocked, while external hostnames are not. ```Rust assert!(is_hostname_blocked("metadata.google.internal").is_some()); assert!(is_hostname_blocked("METADATA.GOOGLE.INTERNAL").is_some()); assert!(is_hostname_blocked("metadata.azure.internal").is_some()); assert!(is_hostname_blocked("example.com").is_none()); ``` -------------------------------- ### Allow Loopback IP Override Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Shows how to override the default blocking of loopback addresses by explicitly allowing a CIDR range. ```Rust #[test] fn test_allow_loopback_override() { let policy = PolicyBuilder::new(Policy::PublicOnly) .allow_cidr("127.0.0.0/8") .build(); // Loopback is normally always blocked, but we explicitly allowed it assert!(policy.is_ip_allowed("127.0.0.1".parse().unwrap()).is_ok()); assert!(policy.is_ip_allowed("127.1.2.3".parse().unwrap()).is_ok()); } ``` -------------------------------- ### Parse and Access URL Host Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Demonstrates parsing a URL string into a SafeUrl and accessing its host component. Ensures the URL contains a specific domain. ```rust let safe_url = SafeUrl::parse("http://example.com/").unwrap(); let inner_url = safe_url.into_url(); assert_eq!(inner_url.host_str(), Some("example.com")); ``` -------------------------------- ### Block Metadata Server Access with Path Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Tests the blocking of access attempts to specific paths on the instance metadata server. ```Rust #[tokio::test] async fn test_redteam_ssrf_metadata_with_path() { let result = validate( "http://169.254.169.254/latest/meta-data/iam/security-credentials/", Policy::PublicOnly, ) .await; assert!(result.is_err()); } ``` -------------------------------- ### Parse URL with Query and Fragment Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Parses a URL that includes query parameters and a fragment identifier. Verifies host and path extraction. ```rust let url = SafeUrl::parse("https://example.com/path?query=1#fragment").unwrap(); assert_eq!(url.host(), "example.com"); assert_eq!(url.path(), "/path"); ``` -------------------------------- ### Test Full-Width IP Address Normalization Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests how the parser handles full-width numerals in an IP address. The URL crate's normalization to ASCII should be caught by the IP blocklist. ```rust fn test_redteam_full_width_ip() { // Full-width numerals: 127 (U+FF11 U+FF12 U+FF17) // URL crate converts these to ASCII - this is GOOD because our IP blocklist catches it let result = SafeUrl::parse("http://\u{FF11}\u{FF12}\u{FF17}.0.0.1/"); match result { Ok(url) => { // Normalized to 127.0.0.1, which our IP blocklist will catch // Either the URL crate normalizes it (127.0.0.1) or keeps it as-is // Either way, IP validation will handle it let _ = url.host(); } Err(_) => {} // Rejection is also fine } } ``` -------------------------------- ### Handle Port with Leading Zeros Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing of URLs with ports that have leading zeros, ensuring they are correctly interpreted as decimal. Rejection is also acceptable. ```Rust let result = SafeUrl::parse("http://example.com:0080/"); match result { Ok(url) => assert_eq!(url.port(), 80), Err(_) => {} // Rejection is also acceptable } ``` -------------------------------- ### Reject URLs with Userinfo Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that URLs containing user information (username and password) in the host are rejected. ```rust assert!(SafeUrl::parse("http://admin:password@127.0.0.1/").is_err()); ``` -------------------------------- ### Allow IPv6 Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL with a standard IPv6 loopback address is parsed successfully. ```rust let url = SafeUrl::parse("http://[::1]/").unwrap(); assert_eq!(url.host(), "[::1]"); ``` -------------------------------- ### Allow Full IPv6 Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL with a full IPv6 address is parsed successfully. ```rust let url = SafeUrl::parse("http://[2001:db8::1]/").unwrap(); assert_eq!(url.host(), "[2001:db8::1]"); ``` -------------------------------- ### Reject Userinfo with IP Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that URLs containing userinfo along with an IP address are rejected. ```rust assert!(SafeUrl::parse("http://user:pass@127.0.0.1").is_err()); ``` -------------------------------- ### Handle Scheme Without Slashes Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing URLs where the scheme is not followed by double slashes (e.g., http:127.0.0.1/). The URL crate typically parses this as a path, not a host. ```rust let result = SafeUrl::parse("http:127.0.0.1/"); match result { Ok(url) => { let _host = url.host(); } Err(_) => {} // Rejection is fine } ``` -------------------------------- ### Test AllowPrivate Policy: Still Blocks Loopback Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Confirms that even with the `AllowPrivate` policy, loopback addresses (127.0.0.1) are still blocked for security. ```rust #[tokio::test] async fn test_allow_private_still_blocks_loopback() { let result = validate("http://127.0.0.1/", Policy::AllowPrivate).await; assert!(result.is_err()); } ``` -------------------------------- ### Test IPv4-Compatible IPv6 Metadata Address Source: https://docs.rs/url_jail/latest/src/url_jail/blocklist.rs.html Verifies that the metadata address represented in IPv4-compatible IPv6 format (::169.254.169.254) is blocked. This ensures that metadata services are not accessible via this format. ```Rust fn test_ipv4_compatible_ipv6_metadata() { // ::169.254.169.254 - metadata via IPv4-compatible let compat: IpAddr = "::169.254.169.254".parse().unwrap(); assert!(is_ip_blocked(compat, Policy::PublicOnly).is_some()); } ``` -------------------------------- ### SafeUrl::as_str Source: https://docs.rs/url_jail/latest/url_jail/struct.SafeUrl.html Returns the full normalized URL as a string, including scheme, host, port (if non-default), path, query, and fragment. ```APIDOC ## SafeUrl::as_str ### Description Gets the complete normalized URL as a string. ### Method `pub fn as_str(&self) -> &str` ### Parameters None ### Request Example None ### Response #### Success Response - `&str`: The full normalized URL string. ### Response Example None ``` -------------------------------- ### Default PolicyBuilder Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Provides a default PolicyBuilder instance. This is useful when a basic builder is needed without explicit initialization. ```rust fn default() -> PolicyBuilder ``` -------------------------------- ### Allow Standard IPv4 Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL with a standard IPv4 address as the host is parsed successfully. ```rust let url = SafeUrl::parse("http://127.0.0.1/").unwrap(); assert_eq!(url.host(), "127.0.0.1"); ``` -------------------------------- ### Asynchronous Validation with Very Short DNS Timeout Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Tests the `validate_with_options` function with a very short DNS timeout to ensure it handles timeouts gracefully without panicking. ```rust #[tokio::test] async fn test_validate_with_very_short_timeout() { let opts = ValidateOptions { dns_timeout: Duration::from_millis(1), }; let result = validate_with_options("https://very-slow-dns-example.invalid/", Policy::PublicOnly, opts).await; assert!(result.is_err()); } ``` -------------------------------- ### Create DnsError Error Source: https://docs.rs/url_jail/latest/src/url_jail/error.rs.html Constructs an Error::DnsError variant. Use this when a DNS resolution fails. ```rust pub(crate) fn dns_error(host: impl Into, message: impl Into) -> Self { Self::DnsError { host: host.into(), message: message.into(), } } ``` -------------------------------- ### Test Red Team Allow Private Allows Private Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Tests that the AllowPrivate policy permits access to private IP ranges, even if DNS resolution fails. ```rust #[tokio::test] async fn test_redteam_allow_private_allows_private() { // Private IPs should work with AllowPrivate // These will fail DNS, but the IP wouldn't be blocked let result = validate("http://10.0.0.1/", Policy::AllowPrivate).await; // Should be SSRF blocked with PublicOnly, but not with AllowPrivate // Will actually fail with DNS error since 10.0.0.1 isn't resolvable match &result { Err(Error::SsrfBlocked { .. }) => { panic!("Private IP should not be SSRF blocked with AllowPrivate") } _ => {} // DNS error or success (if IP is parsed directly) } } ``` -------------------------------- ### Block a hostname or pattern Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Adds a hostname or pattern to the blocklist. Supports wildcard characters like '*'. Patterns are converted to lowercase. ```rust pub fn block_host(mut self, pattern: &str) -> Self { self.blocked_hosts.push(pattern.to_lowercase()); self } ``` -------------------------------- ### Test Red Team Allow Private Blocks Loopback Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Verifies that the AllowPrivate policy still blocks loopback addresses. ```rust #[tokio::test] async fn test_redteam_allow_private_blocks_loopback() { let result = validate("http://127.0.0.1/", Policy::AllowPrivate).await; assert!(result.is_err(), "AllowPrivate must still block loopback"); } ``` -------------------------------- ### Test Backslash as Path Separator in URL Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests if a URL with Windows-style path separators (\) is parsed correctly, ensuring the host remains as expected. The host should be correctly identified. ```rust fn test_redteam_backslash_as_path_separator() { // Windows-style path separator let result = SafeUrl::parse("http://example.com\\path\\to\\file"); match result { Ok(url) => { assert_eq!(url.host(), "example.com"); } Err(_) => {} // Rejection is also acceptable } } ``` -------------------------------- ### Test Blocked Subdomains Source: https://docs.rs/url_jail/latest/src/url_jail/blocklist.rs.html Verifies that subdomains of blocked hostnames are also blocked. This ensures that the blocklist applies recursively to nested hostnames. ```Rust fn test_hostname_blocklist_subdomain() { // Subdomains of blocked hostnames should also be blocked assert!(is_hostname_blocked("sub.metadata.google.internal").is_some()); assert!(is_hostname_blocked("deep.sub.metadata.google.internal").is_some()); } ``` -------------------------------- ### Test File Scheme URL Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Validates that URLs using the 'file://' scheme are rejected when using PublicOnly policy. ```Rust let result = validate("file:///etc/passwd", Policy::PublicOnly).await; assert!(result.is_err()); ``` -------------------------------- ### Test Whitespace-Only URL Parsing Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Verifies that a URL consisting only of whitespace is correctly rejected. ```rust assert!(SafeUrl::parse(" ").is_err()); ``` -------------------------------- ### Handle Subdomain Resembling IP Address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing a URL where a subdomain looks like an IP address. This is considered a valid hostname and requires DNS validation. ```rust let url = SafeUrl::parse("http://127.0.0.1.attacker.com/").unwrap(); assert_eq!(url.host(), "127.0.0.1.attacker.com"); ``` -------------------------------- ### Test AllowPrivate Policy Allows Private IPs Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Confirms that a Policy set to AllowPrivate correctly permits private IP addresses. ```rust #[test] fn test_allow_private_base_allows_private() { let policy = PolicyBuilder::new(Policy::AllowPrivate).build(); assert!(policy.is_ip_allowed("192.168.1.1".parse().unwrap()).is_ok()); assert!(policy.is_ip_allowed("10.0.0.1".parse().unwrap()).is_ok()); } ``` -------------------------------- ### Test Hostname Pattern Matching Case Sensitivity Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Demonstrates that matches_hostname_pattern expects lowercase input and does not handle case mismatches; callers should lowercase hostnames. ```rust #[test] fn test_matches_hostname_pattern_case() { // Function expects lowercase input assert!(matches_hostname_pattern("example.com", "example.com")); // Case mismatch - function doesn't handle, caller should lowercase assert!(!matches_hostname_pattern("EXAMPLE.COM", "example.com")); } ``` -------------------------------- ### Block Unspecified IP with AllowPrivate Policy Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Verifies that unspecified IP addresses are blocked even when the policy allows private IP addresses. ```Rust #[tokio::test] async fn test_block_unspecified_with_allow_private() { // Unspecified should be blocked even with AllowPrivate let result = validate("http://0.0.0.0/", Policy::AllowPrivate).await; assert!(result.is_err()); } ``` -------------------------------- ### Reject Unbalanced Brackets Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests the rejection of URLs with unbalanced brackets around IPv6 addresses. ```rust assert!(SafeUrl::parse("http://[::1/").is_err()); assert!(SafeUrl::parse("http://::1]/").is_err()); ``` -------------------------------- ### Synchronous Validation of Public IP Source: https://docs.rs/url_jail/latest/src/url_jail/validate.rs.html Tests the synchronous `validate_sync` function to confirm it correctly validates a public IP address and extracts host and port information. ```rust #[test] fn test_validate_sync_public_ip() { let result = validate_sync("https://example.com/", Policy::PublicOnly); assert!(result.is_ok()); let validated = result.unwrap(); assert_eq!(validated.host, "example.com"); assert_eq!(validated.port, 443); assert!(validated.https); } ``` -------------------------------- ### Test IPv6 Loopback Variations Source: https://docs.rs/url_jail/latest/src/url_jail/blocklist.rs.html Ensures that various representations of the IPv6 loopback address (::1) are correctly blocked. This includes full, abbreviated, and zero-padded forms. ```Rust fn test_redteam_ipv6_loopback_variations() { // Various representations of ::1 assert!(is_ip_blocked("::1".parse().unwrap(), Policy::PublicOnly).is_some()); assert!(is_ip_blocked("0:0:0:0:0:0:0:1".parse().unwrap(), Policy::PublicOnly).is_some()); assert!(is_ip_blocked( "0000:0000:0000:0000:0000:0000:0000:0001".parse().unwrap(), Policy::PublicOnly ) .is_some()); } ``` -------------------------------- ### Test Empty URL Parsing Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Ensures that parsing an empty string as a URL results in an error. ```rust assert!(SafeUrl::parse("").is_err()); ``` -------------------------------- ### Parse IPv6 loopback address Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests parsing of the IPv6 loopback address `[::1]`. The host is correctly identified. ```rust fn test_ipv6_loopback() { let url = SafeUrl::parse("http://[::1]/").unwrap(); assert_eq!(url.host(), "[::1]"); } ``` -------------------------------- ### Error Handling for URL Validation Source: https://docs.rs/url_jail/latest/index.html Shows how to handle errors returned by `validate_sync` using pattern matching on the `Error` enum. Differentiates between security rejections and temporary errors. Requires `validate_sync` and `Policy` from `url_jail`. ```rust use url_jail::{validate_sync, Policy, Error}; match validate_sync("http://127.0.0.1/", Policy::PublicOnly) { Ok(v) => println!("Safe: {}", v.ip), Err(e) if e.is_blocked() => { // Security rejection: SsrfBlocked, HostnameBlocked, RedirectBlocked println!("Blocked: {}", e); if let Some(url) = e.url() { println!("URL: {}", url); } } Err(e) if e.is_retriable() => { // Temporary error: DnsError, Timeout, HttpError // Retry with caution for untrusted URLs println!("Temporary: {}", e); } Err(e) => println!("Error: {}", e), } ``` -------------------------------- ### Allow Hostname or Pattern with PolicyBuilder Source: https://docs.rs/url_jail/latest/url_jail/struct.PolicyBuilder.html Adds a rule to allow a specific hostname or a pattern, overriding any existing blocklist rules. This method can be chained. ```rust pub fn allow_host(self, pattern: &str) -> Self ``` -------------------------------- ### Test Alibaba Cloud Metadata Source: https://docs.rs/url_jail/latest/src/url_jail/blocklist.rs.html Tests blocking of the Alibaba Cloud metadata endpoint. This snippet also demonstrates checking against `Policy::AllowPrivate`. ```rust let alibaba: IpAddr = "100.100.100.200".parse().unwrap(); assert!(is_ip_blocked(alibaba, Policy::PublicOnly).is_some()); assert!(is_ip_blocked(alibaba, Policy::AllowPrivate).is_some()); ``` -------------------------------- ### Allow an IP range using CIDR notation Source: https://docs.rs/url_jail/latest/src/url_jail/policy_builder.rs.html Adds a CIDR allow rule to the policy, which can override base policy restrictions. Invalid CIDR strings are ignored. ```rust pub fn allow_cidr(mut self, cidr: &str) -> Self { if let Ok(net) = cidr.parse() { self.allowed_cidrs.push(net); } self } ``` -------------------------------- ### Parse Unicode hostname and normalization Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests how `SafeUrl::parse` handles Unicode hostnames. The parser is expected to convert them to Punycode or handle them appropriately. ```rust fn test_unicode_hostname_gets_normalized() { // Unicode hostnames should be converted to punycode by the URL parser let result = SafeUrl::parse("http://münchen.de/"); // Should either work (with punycode) or fail cleanly match result { Ok(url) => { // Should be normalized to punycode let host = url.host(); assert!(host == "münchen.de" || host.contains("xn-- MÜNCHEN.DE")); } Err(_) => { // Or fail cleanly if normalization is not supported } } } ``` -------------------------------- ### Parse URL with Empty Path Source: https://docs.rs/url_jail/latest/src/url_jail/safe_url.rs.html Tests that a URL without an explicit path is parsed correctly. ```rust let url = SafeUrl::parse("http://example.com").unwrap(); assert_eq!(url.host(), "example.com"); ```