### URL Normalization Example using trurl Source: https://curl.se/trurl/manual This example demonstrates the URL normalization feature of trurl. It shows how trurl decodes and re-encodes URL components, processing percent-encoded characters and path segments. ```bash $ trurl 'http://ex%61mple:80/%62ath/a/../b?%2e%FF#tes%74' http://example/bath/b?.%ff#test ``` -------------------------------- ### Normalize IPv6 Address with Trurl Source: https://curl.se/trurl/manual Example of Trurl normalizing an IPv6 address, shortening consecutive zeros and handling the address format. ```bash $ trurl 'http://[2001:9b1:0:0:0:0:7b97:364b]/' http://[2001:9b1::7b97:364b]/ ``` -------------------------------- ### JSON Output of URL Components Source: https://curl.se/trurl/manual Provides an example of the JSON output format generated by trurl, showing various components of a URL like scheme, host, port, path, query, and fragment. The 'parts' object details these components. ```json [ { "url": "http://alpha/?one=real&two=fake", "parts": { "scheme": "http", "host": "alpha", "port": null, "path": "/", "query": "one=real&two=fake", "fragment": null }, "params": [ { "key": "one", "value": "real" }, { "key": "two", "value": "fake" } ] } ] ``` -------------------------------- ### Get Full URL using JSON Output Source: https://curl.se/trurl/manual Demonstrates retrieving the complete URL as a string using the '--json' option and specifying '{url}'. This is useful when you need the entire URL in a structured format. ```bash $ trurl ftps://example.com:2021/p%61th -g '{url}' ftps://example.com:2021/path ``` -------------------------------- ### Extract Default Port from URL Source: https://curl.se/trurl/manual Demonstrates extracting the default port for a given scheme if it's not explicitly set in the URL, using the --get '{default:port}' option. This helps in understanding standard port assignments. ```shell $ trurl --url https://curl.se/we/are.html --get '{default:port}' 443 ``` -------------------------------- ### Extract Path from URL Source: https://curl.se/trurl/manual Shows how to extract specific components from a URL, in this case, the path, using the --get '{path}' option. This is useful for parsing URL information. ```shell $ trurl --url https://curl.se/we/are.html --get '{path}' /we/are.html ``` -------------------------------- ### Extract Fragment from URL Source: https://curl.se/trurl/manual Shows how to extract the fragment part of a URL (excluding the leading hash '#') using the '-g' or '--get' option with '{fragment}'. ```bash $ trurl http://horse#elephant -g '{fragment}' elephant ``` -------------------------------- ### Create URL by Setting Components Source: https://curl.se/trurl/manual Shows how to construct a URL by specifying its components like host and scheme using the --set option. This allows for building URLs programmatically. ```shell $ trurl --set host=example.com --set scheme=ftp ftp://example.com/ ``` -------------------------------- ### Convert Punycode to Hostname with Trurl Source: https://curl.se/trurl/manual Shows how to convert a Punycode hostname back to its Internationalized Domain Name (IDN) version using the '--as-idn' option. ```bash $ trurl http://xn--4cab6c/ --as-idn http://åäö/ ``` -------------------------------- ### Create URL Variations with Different Schemes Source: https://curl.se/trurl/manual Shows how to generate multiple variations of a URL by iterating through a list of different schemes using the --iterate "scheme=..." option. This is useful for testing protocol compatibility. ```shell $ trurl "https://curl.se/path/index.html" --iterate "scheme=http ftp sftp" http://curl.se/path/index.html ftp://curl.se/path/index.html sftp://curl.se/path/index.html ``` -------------------------------- ### Normalize URL Path with Trurl Source: https://curl.se/trurl/manual Explains how Trurl handles URL paths, ensuring a leading slash, normalizing '..', './' sequences, and providing options to append path segments. ```bash $ trurl http://xn--4cab6c -g '[path]' / ``` ```bash $ trurl http://hello -s path="pony" http://hello/pony ``` ```bash $ trurl http://hello -s path="/pony" http://hello/pony ``` ```bash $ trurl http://hej/one/../two/../three/./four http://hej/three/four ``` ```bash $ trurl http://twelve/three?hello --append path=four http://twelve/three/four?hello ``` -------------------------------- ### Iterating Over URL Components with trurl Source: https://curl.se/trurl/manual Demonstrates how to use the --iterate option to generate multiple URL variations based on different component values. This is useful for testing or generating a set of related URLs. ```bash $ trurl example.com --iterate=scheme="ftp https" --iterate=port="22 80" ftp://example.com:22/ ftp://example.com:80/ https://example.com:22/ https://example.com:80/ ``` -------------------------------- ### Handle Default Ports with Trurl Source: https://curl.se/trurl/manual Demonstrates Trurl's behavior regarding default ports for schemes. It can show default ports ('--default-port'), hide them when explicit, and optionally keep them ('--keep-port'). ```bash $ trurl http:/a --default-port http://a:80/ ``` ```bash $ trurl http:/a:80 http://a/ ``` ```bash $ trurl http:/a:80 --keep-port http://a:80/ ``` -------------------------------- ### Read URLs from Stdin Source: https://curl.se/trurl/manual Demonstrates how to read a list of URLs from standard input using the --url-file - option. This allows processing URLs from files or piped output. ```shell $ cat urllist.txt | trurl --url-file - ... ``` -------------------------------- ### Using Punycode Hostnames with trurl Source: https://curl.se/trurl/manual Demonstrates the --punycode option, which converts hostnames to their Punycode representation for Internationalized Domain Names (IDNs). If the hostname is not an IDN, it remains unchanged. ```bash $ trurl http://åäö/ --punycode http://xn--4cab6c/ ``` -------------------------------- ### Extract URL Scheme with Trurl Source: https://curl.se/trurl/manual Demonstrates how to extract the URL scheme using the '{scheme}' format specifier. Trurl automatically guesses the scheme (defaulting to http) if not explicitly provided, unless the '--no-guess-scheme' option is used. ```bash $ trurl https://odd/ -g '{scheme}' https ``` ```bash $ trurl odd -g '{scheme}' http ``` ```bash $ trurl odd -g '{scheme}' --no-guess-scheme trurl note: Bad scheme [odd] ``` -------------------------------- ### Sorting Query Parameters with trurl Source: https://curl.se/trurl/manual Explains how to use the --sort-query option to sort query parameters alphabetically in a case-insensitive manner. This helps in normalizing URLs. -------------------------------- ### Redirecting URLs with trurl Source: https://curl.se/trurl/manual Demonstrates the --redirect option, which redirects the base URL to a new location. The redirection is applied relative to the base URL. ```bash $ trurl --url https://curl.se/we/are.html --redirect ../here.html https://curl.se/here.html ``` -------------------------------- ### Accept Spaces in URL Path Source: https://curl.se/trurl/manual Illustrates how to enable the acceptance of spaces within the URL path, which are then correctly encoded. This is achieved using the --accept-space option. ```shell $ trurl "https://curl.se/this has space/index.html" --accept-space https://curl.se/this%20has%20space/index.html ``` -------------------------------- ### Work with Semicolon Separated Query Source: https://curl.se/trurl/manual Demonstrates how to handle URLs where the query string uses a semicolon as a separator instead of an ampersand, using the --query-separator ";" option. This is combined with --qtrim to remove specific parameters. ```shell $ trurl "https://curl.se?search=fool;page=5" --qtrim "search" --query-separator ";" https://curl.se?page=5 ``` -------------------------------- ### Output URL in JSON Format Source: https://curl.se/trurl/manual Shows how to parse a URL and output its components and parameters in a structured JSON format using the --json option. This is valuable for integrating with other applications. ```shell $ trurl "https://fake.host/search?q=answers&user=me#frag" --json [   {   "url": "https://fake.host/search?q=answers&user=me#frag",   "parts": [   "scheme": "https",   "host": "fake.host",   "path": "/search",   "query": "q=answers&user=me"   "fragment": "frag",   ],   "params": [   {   "key": "q",   "value": "answers"   },   {   "key": "user",   "value": "me"   }   ]   } ] ``` -------------------------------- ### Trim Query Parameters by Pattern Source: https://curl.se/trurl/manual Demonstrates removing query parameters that match a specified pattern using the --qtrim option with a wildcard. This is useful for selectively removing multiple parameters based on their keys. ```bash $ trurl 'https://example.com?a12=hej&a23=moo&b12=foo' --qtrim a* https://example.com/?b12=foo ``` -------------------------------- ### Extract Query String with Trurl Source: https://curl.se/trurl/manual Shows how to extract the query string using '{query}', noting that the leading '?' is not included. It also demonstrates setting the query string, including encoding special characters. ```bash $ trurl http://horse?elephant -g '{query}' elephant ``` ```bash $ trurl http://horse?elephant -s "query=?elephant" http://horse/?%3felephant ``` -------------------------------- ### Redirect a URL Source: https://curl.se/trurl/manual Illustrates how to change the path of a URL to a new specified path using the --redirect option. This is helpful for simple URL rewrites. ```shell $ trurl --url https://curl.se/we/are.html --redirect here.html https://curl.se/we/here.html ``` -------------------------------- ### Extract Password from URL with Trurl Source: https://curl.se/trurl/manual Illustrates extracting the password using the '{password}' specifier. It handles cases where the password might be URL encoded and notes that for IMAP schemes, a semicolon can delimit an options field. ```bash $ trurl https://user:secr%65t@odd/ -g '{password}' secret ``` ```bash $ trurl 'imap://user:pwd;giraffe@odd' -g '{options}' giraffe ``` ```bash $ trurl 'sftp://user:pwd;giraffe@odd' -g '{password}' pwd;giraffe ``` -------------------------------- ### Sort Query Key/Value Pairs Source: https://curl.se/trurl/manual Shows how to sort the key-value pairs within the query component of a URL alphabetically using the --sort-query option. This ensures consistent query string order. ```shell $ trurl "https://example.com?b=a&c=b&a=c" --sort-query https://example.com?a=c&b=a&c=b ``` -------------------------------- ### Replace URL Hostname Source: https://curl.se/trurl/manual Demonstrates how to change the hostname of a given URL using the --set option. This is useful for testing or redirecting traffic. ```shell $ trurl --url https://curl.se --set host=example.com https://example.com/ ``` -------------------------------- ### Sort Query Parameters in URL Source: https://curl.se/trurl/manual Illustrates how to sort the query parameters of a URL alphabetically using the --sort-query option. Sorting can be helpful for comparing URLs or ensuring consistent ordering. ```bash $ trurl "http://alpha/?one=real&two=fake&three=alsoreal" --sort-query http://alpha/?one=real&three=alsoreal&two=fake ``` -------------------------------- ### Append Query Parameters with Trurl Source: https://curl.se/trurl/manual Demonstrates appending new key-value pairs to the query string of a URL using the '--append query=' option. ```bash $ trurl http://host?name=hello --append query=search=life http://host/?name=hello&search=life ``` -------------------------------- ### Remove Tracking Tuples from Query Source: https://curl.se/trurl/manual Demonstrates how to remove query parameters that match a given pattern, using the --qtrim "utm_*" option. This is useful for cleaning URLs for analytics or privacy. ```shell $ trurl "https://curl.se?search=hey&utm_source=tracker" --qtrim "utm_*" https://curl.se/?search=hey ``` -------------------------------- ### Extract User from URL with Trurl Source: https://curl.se/trurl/manual Shows how to extract the username component from a URL. The '{user}' specifier retrieves the username, including any special characters like ':' or '@' that are part of the username itself. ```bash $ trurl https://user%3a%40:secret@odd/ -g '{user}' user:@ ``` -------------------------------- ### Show Specific Query Key Value Source: https://curl.se/trurl/manual Illustrates how to extract the value of a specific query parameter using the -g '{query:key}' option. This allows targeted retrieval of information from URL query strings. ```shell $ trurl "https://example.com?a=home&here=now&thisthen" -g '{query:a}' home ``` -------------------------------- ### Replace or Append Query Parameter in URL Source: https://curl.se/trurl/manual Shows how to replace an existing query parameter or append a new one if it doesn't exist, using the --replace-append option. This provides flexibility in managing URL query parameters. ```bash $ trurl 'http://alpha?one=real&two=fake' --replace-append three=alsoreal http://alpha/?one=real&two=fake&three=alsoreal ``` -------------------------------- ### Set Fragment in URL with Leading Hash Source: https://curl.se/trurl/manual Illustrates how to set the fragment of a URL, including a leading hash symbol, using the '-s' or '--set' option. The fragment is URL-encoded. ```bash $ trurl "http://horse#elephant" -s "fragment=#zebra" http://horse/#%23zebra ``` -------------------------------- ### Normalize IPv4 Address with Trurl Source: https://curl.se/trurl/manual Illustrates Trurl's ability to normalize various representations of IPv4 addresses (decimal, octal, hexadecimal) into the standard four-dotted-decimal format. ```bash $ trurl http://646464646/ http://38.136.68.134/ ``` ```bash $ trurl http://246.646/ http://246.0.2.134/ ``` ```bash $ trurl http://246.46.646/ http://246.46.2.134/ ``` ```bash $ trurl http://0x14.0xb3022/ http://20.11.48.34/ ``` -------------------------------- ### Disabling Scheme Guessing with trurl Source: https://curl.se/trurl/manual Illustrates the use of --no-guess-scheme to disable libcurl's scheme guessing. URLs without a scheme will be treated as invalid, and trurl will report an error. ```bash $ trurl example.com --no-guess-scheme trurl note: Bad scheme [example.com] ``` -------------------------------- ### Specifying Query Separator with trurl Source: https://curl.se/trurl/manual Shows how to use the --query-separator option to define a custom character for separating query parameters. This is useful when URLs use non-standard separators like colons instead of ampersands. ```bash $ trurl "https://curl.se?b=name:a=age" --sort-query --query-separator ":" https://curl.se/?a=age:b=name ``` -------------------------------- ### Extract Zone ID from IPv6 Address with Trurl Source: https://curl.se/trurl/manual Shows how to extract the zone identifier from an IPv6 address, which can be a number or network interface name, using the '{zoneid}' specifier. ```bash $ trurl 'http://[2001:9b1::f358:1ba4:7b97:364b%enp3s0]/' -g '{zoneid}' enp3s0 ``` -------------------------------- ### Append Query Segment to URL Source: https://curl.se/trurl/manual Shows how to append a new key-value pair to the query string of a URL using the --append query= option. This is essential for adding parameters to URLs. ```shell $ trurl --url "https://curl.se?name=hello" --append query=search=string  https://curl.se/?name=hello&search=string ``` -------------------------------- ### Replace Query Parameter in URL Source: https://curl.se/trurl/manual Demonstrates how to replace the value of an existing query parameter in a URL using the --replace option. This is useful for updating specific values within a URL's query string. ```bash $ trurl 'http://alpha?one=real&two=fake' --replace two=alsoreal http://alpha/?one=real&two=alsoreal ``` -------------------------------- ### Append Path Segment to URL Source: https://curl.se/trurl/manual Illustrates how to add a new segment to the existing path of a URL using the --append path= option. This is useful for building longer URLs dynamically. ```shell $ trurl --url https://curl.se/hello --append path=you https://curl.se/hello/you ``` -------------------------------- ### Change URL Port Number Source: https://curl.se/trurl/manual Demonstrates how to modify the port number of a URL using the --set port option. It also highlights trurl's ability to handle and clean up path components like dot-dot sequences. ```shell $ trurl --url https://curl.se/we/../are.html --set port=8080 https://curl.se:8080/are.html ``` -------------------------------- ### Keeping Default Port Numbers with trurl Source: https://curl.se/trurl/manual Shows how the --keep-port option prevents trurl from removing default port numbers from URLs with known schemes. This ensures that explicitly provided default ports are retained in the output. ```bash $ trurl https://example.com:443/ --keep-port https://example.com:443/ ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.