### Install httpx using Go Source: https://github.com/projectdiscovery/httpx/blob/dev/README.md Install the httpx tool using the Go package manager. Ensure you have go version >=1.25.0 installed. This command fetches the latest version of the httpx repository. ```sh go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest ``` -------------------------------- ### Example httpx Library Usage Source: https://github.com/projectdiscovery/httpx/blob/dev/README.md Demonstrates how to use httpx as a library in Go. Create an Option struct, populate it with desired configurations, and pass it to a runner instance to execute the enumeration. ```go package main import ( "fmt" "github.com/projectdiscovery/httpx/runner" ) func main() { options := &runner.Options{ // Populate options here, e.g.: // Methods: "GET,POST", // Threads: 100, // Timeout: 10, } runner, err := runner.NewRunner(options) if err != nil { panic(fmt.Sprintf("unable to create runner: %v", err)) } defer runner.Close() results := runner.RunEnumeration(nil) // Pass nil for stdin input for result := range results { fmt.Printf("%s\n", result.String()) } } ``` -------------------------------- ### Event Listener Setup for httpx Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Sets up various event listeners for httpx, including 'new-xhr', 'open-xhr-end', 'fetch-done', etc. ```javascript t.on("new-xhr",i),t.on("open-xhr-start",o),t.on("open-xhr-end",c),t.on("send-xhr-start",u),t.on("xhr-cb-time",d),t.on("xhr-load-added",l),t.on("xhr-load-removed",f),t.on("xhr-resolved",h),t.on("addEventListener-end",g),t.on("removeEventListener-end",m),t.on("fn-end",b),t.on("fetch-before-start",y),t.on("fetch-start",A),t.on("fn-start",v),t.on("fetch-done",w)}(e,this.ee,this.handler,this.dt),this.importAggregator()}} ``` -------------------------------- ### httpx Secret File Configuration Example Source: https://github.com/projectdiscovery/httpx/blob/dev/README.md Shows an example of a secret file in YAML format for domain-based authentication with httpx. Supports various authentication types like Header, BasicAuth, BearerToken, Cookie, and Query. ```yaml id: example-auth info: name: Example Auth Config static: - type: Header domains: - api.example.com headers: - key: X-API-Key value: secret-key-here - type: BasicAuth domains-regex: - ".*\\.internal\\.com$" username: admin password: secret ``` -------------------------------- ### Agent API Setup Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Sets up the agent's API methods, including error handling, tracing, and custom attribute management. This function ensures all available API methods are exposed. ```javascript function b(){const e=(0,f.gG)();p.forEach((t=>{e[t]=function(){for(var r=arguments.length,n=new Array(r),i=0;i1?r-1:0),i=1;i{e.exposed&&e.api[t]&&o.push(e.api[t](...n))})),o.length>1?o:o[0]}}(t,...n)}}))} ``` -------------------------------- ### Dynamic Concurrency Control via HTTP API Source: https://context7.com/projectdiscovery/httpx/llms.txt Expose an embedded HTTP server to dynamically adjust scan concurrency at runtime. This example demonstrates starting a scan with low concurrency and then increasing it via a PUT request to the API endpoint. ```go package main import ( "bytes" "encoding/json" "fmt" "log" "net/http" "time" "github.com/projectdiscovery/goflags" "github.com/projectdiscovery/httpx/runner" ) func main() { apiAddr := "127.0.0.1:31234" // Build 200 test URLs var urls []string for i := 0; i < 200; i++ { urls = append(urls, fmt.Sprintf("https://scanme.sh/path-%d", i)) } options := runner.Options{ InputTargetHost: goflags.StringSlice(urls), Methods: "GET", Threads: 1, // Start slow HttpApiEndpoint: apiAddr, OnResult: func(r runner.Result) { if r.Err == nil { fmt.Printf("[%d] %s\n", r.StatusCode, r.URL) } }, } if err := options.ValidateOptions(); err != nil { log.Fatal(err) } httpxRunner, err := runner.New(&options) if err != nil { log.Fatal(err) } defer httpxRunner.Close() // Ramp up to 50 threads after 2 seconds time.AfterFunc(2*time.Second, func() { payload, _ := json.Marshal(runner.Concurrency{Threads: 50}) req, _ := http.NewRequest(http.MethodPut, fmt.Sprintf("http://%s/api/concurrency", apiAddr), bytes.NewBuffer(payload)) req.Header.Set("Content-Type", "application/json") resp, err := http.DefaultClient.Do(req) if err != nil { log.Printf("Failed to update concurrency: %v", err) return } defer resp.Body.Close() fmt.Printf("Concurrency updated, HTTP %d\n", resp.StatusCode) }) // Check current concurrency via GET time.AfterFunc(5*time.Second, func() { resp, _ := http.Get(fmt.Sprintf("http://%s/api/concurrency", apiAddr)) if resp != nil { defer resp.Body.Close() var c runner.Concurrency json.NewDecoder(resp.Body).Decode(&c) fmt.Printf("Current threads: %d\n", c.Threads) } }) httpxRunner.RunEnumeration() } ``` -------------------------------- ### Start New Relic Agent Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Starts the New Relic agent. This should be called after all necessary configurations have been set. ```javascript this.api.start(e) ``` -------------------------------- ### XHR Callback Start Timer Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Sets the start time for an XHR callback and tracks if 'onload' has been set. ```javascript function v(e,t,r){t instanceof ne&&("onload"===r&&(this.onload=!0),("load"==(e\[0\]&&e\[0\].type)||this.onload)&&(this.xhrCbStart=(0,P.z)()))} ``` -------------------------------- ### Implement Custom Response Filters with httpx.Filter Source: https://context7.com/projectdiscovery/httpx/llms.txt Implement the `httpx.Filter` interface to create custom logic for matching HTTP responses. This example shows how to filter by Content-Type prefix, string keywords, regex patterns, and custom conditions. ```go package main import ( "fmt" "log" "strings" "github.com/projectdiscovery/httpx/common/httpx" ) // ContentTypeFilter passes only responses with a specific Content-Type prefix type ContentTypeFilter struct { Prefix string } func (f ContentTypeFilter) Filter(resp *httpx.Response) (bool, error) { ct := resp.GetHeaderPart("Content-Type", ";") return strings.HasPrefix(strings.ToLower(ct), strings.ToLower(f.Prefix)), nil } func main() { client, err := httpx.New(&httpx.DefaultOptions) if err != nil { log.Fatal(err) } // Chain multiple filters client.AddFilter(ContentTypeFilter{Prefix: "application/json"}) client.AddFilter(httpx.FilterString{Keywords: []string{"\"status\""}}) client.AddFilter(httpx.FilterRegex{Regexs: []string{`"version"\s*:\s*"\d+\.\d+`}}) client.AddFilter(httpx.FilterCustom{ CallBacks: []httpx.CustomCallback{ func(resp *httpx.Response) (bool, error) { return resp.StatusCode == 200 && resp.ContentLength > 100, nil }, }, }) req, _ := client.NewRequest("GET", "https://api.github.com") client.SetCustomHeaders(req, map[string]string{"Accept": "application/json"}) // Verify returns true only if ALL registered filters match matched, err := client.Verify(req, httpx.UnsafeOptions{}) fmt.Printf("All filters matched: %v, err: %v\n", matched, err) } ``` -------------------------------- ### Initialize New Relic Agent Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Initializes the New Relic agent with a specified agent identifier. If no identifier is provided, a default one is generated. This is typically done once when the application starts. ```javascript new o( (0,r.ky)(16) ) ``` -------------------------------- ### New Relic Agent Webpack Configuration Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Configures Webpack for the New Relic agent, including defining module loading, chunk management, and public path. This setup is essential for the agent's dynamic loading capabilities. ```javascript i.m=r,i.d=(e,t)=>{for(var r in t)i.o(t,r)&&!i.o(e,r)&&Object.defineProperty(e,r,{enumerable:!0,get:t[r]})},i.f={},i.e=e=>Promise.all(Object.keys(i.f).reduce(((t,r)=>(i.f[r](e,t),t)),[])),i.u=e=>({111:"nr-spa",164:"nr-spa-compressor",433:"nr-spa-recorder"}[e]+"-1.260.1.min.js"),i.o=(e,t)=>Object.prototype.hasOwnProperty.call(e,t),e={},t="NRBA-1.260.1.PROD:",i.l=(r,n,o,a)=>{if(e[r])e[r].push(n);else{var s,c;if(void 0!==o)for(var u=document.getElementsByTagName("script"),d=0;d{s.onerror=s.onload=null,clearTimeout(p);var i=e[r];if(delete e[r],s.parentNode&&s.parentNode.removeChild(s),i&&i.forEach((e=>e(n))),t)return t(n)},p=setTimeout(h.bind(null,void 0,{type:"timeout",target:s}),12e4);s.onerror=h.bind(null,s.onerror),s.onload=h.bind(null,s.onload),c&&document.head.appendChild(s)}},i.p="https://js-agent.newrelic.com/",(()=>{var e={801:0,92:0};i.f.j=(t,r)=>{var n=i.o(e,t)?e[t]:void 0;if(0!==n)if(n)r.push(n[2]);else{var o=new Promise(((r,i)=>n=e[t]=[r,i]));r.push(n[2]=o);var a=i.p+i.u(t),s=new Error;i.l(a,(r=>{if(i.o(e,t)&&(0!==(n=e[t])&&(e[t]=void 0),n)){var o=r&&("load"===r.type?"missing":r.type),a=r&&r.target&&r.target.src;s.message="Loading chunk "+t+" failed.\n("+o+": "+a+")",s.name="ChunkLoadError",s.type=o,s.request=a,n[1](s)}}),"chunk-"+t,t)}};var t=(t,r)=>{var n,o,[a,s,c]=r,u=0;if(a.some((t=>0!==e[t]))){for(n in s)i.o(s,n)&&(i.m[n]=s[n]);if(c)c(i)}for(t&&t(r);u 500", // TLS TLSGrab: true, ZTLS: false, // Output JSONOutput: true, Output: "results.json", // Performance Threads: 50, Timeout: 15, RateLimit: 100, Retries: 3, Delay: 50 * time.Millisecond, // Pre-created shared objects Wappalyzer: wap, CDNCheckClient: cdnClient, OnResult: func(r runner.Result) { if r.Err != nil { return } fmt.Printf("URL=%s Status=%d TLS=%v Tech=%v ASN=%v\n", r.URL, r.StatusCode, r.TLSData != nil, r.Technologies, r.ASN) }, OnClose: func() { fmt.Println("Scan complete.") }, } if err := options.ValidateOptions(); err != nil { log.Fatal(err) } r, err := runner.New(&options) if err != nil { log.Fatal(err) } defer r.Close() r.RunEnumeration() } ``` -------------------------------- ### Get Performance Timestamp Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Returns the current high-resolution timestamp from the performance API, typically used for performance measurements. ```javascript function n(){return Math.floor(performance.now())} ``` -------------------------------- ### Configurable Object Initialization Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Handles the creation and merging of configurable objects, ensuring properties are set correctly based on provided models. Useful for initializing complex objects with default and custom settings. ```javascript var n=r(50);function i(e,t){try{if(!e||"object"!=typeof e)return(0,n.Z)("Setting a Configurable requires an object as input");if(!t||"object"!=typeof t)return(0,n.Z)("Setting a Configurable requires a model to set its initial properties");const r=Object.create(Object.getPrototypeOf(t),Object.getOwnPropertyDescriptors(t)),o=0===Object.keys(r).length?e:r;for(let a in o)if(void 0!==e[a])try{if(null===e[a]){r[a]=null;continue}Array.isArray(e[a])&&Array.isArray(t[a])?r[a]=Array.from(new Set([...e[a],...t[a]])):"object"==typeof e[a]&&"object"==typeof t[a]?r[a]=i(e[a],t[a]):r[a]=e[a]}catch(e){(0,n.Z)("An error occurred while setting a property of a Configurable",e)}}catch(e){(0,n.Z)("An error occured while setting a Configurable",e)}} ``` -------------------------------- ### Initialize httpx Agent Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Initializes the httpx agent with provided configuration. Ensures essential agent properties are set and features are activated. ```javascript function _(e){let t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},p=arguments.length>2?arguments[2]:void 0,_=arguments.length>3?arguments[3]:void 0,{init:x,info:R,loader_config:S,runtime:T={loaderType:p},exposed:N=!0}=t;const O=(0,f.gG)();R||(x=O.init,R=O.info,S=O.loader_config),(0,i.Dg)(e.agentIdentifier,x||{}),(0,i.GE)(e.agentIdentifier,S||{}),R.jsAttributes??={},u.v6&&(R.jsAttributes.isWorker=!0),(0,i.CX)(e.agentIdentifier,R);const I=(0,i.P_)(e.agentIdentifier),D=[R.beacon,R.errorBeacon];E||(I.proxy.assets&&(w(I.proxy.assets),D.push(I.proxy.assets)),I.proxy.beacon&&D.push(I.proxy.beacon),b(),(0,f.EZ)("activatedFeatures",A.T),e.runSoftNavOverSpa&&=!0===I.soft_navigations.enabled&&I.feature_flags.includes("soft_nav")),T.denyList=[...I.ajax.deny_list||[],...I.ajax.block_internal?D:[]],T.ptid=e.agentIdentifier,(0,i.sU)(e.agentIdentifier,T),void 0===e.api&&(e.api=function(e,t){let f=arguments.length>2&&void 0!==arguments[2]&&arguments[2];t||(0,s.RP)(e,"api");const p={};var b=a.ee.get(e),A=b.get("tracer");y[e]=v.IK.OFF,b.on(h.Ef.REPLAY_RUNNING,(t=>{y[e]=t}));var w="api-",E=w+"ixn-";function _(t,r,n,o){const a=(0,i.C5)(e);return null===r?delete a.jsAttributes[t]:(0,i.CX)(e,{...a,jsAttributes:{...a.jsAttributes,[t]:r}}),S(w,n,!0,o||null===r?"session":void 0)(t,r)}function x(){}g.forEach((e=>{p[e]=S(w,e,!0,"api")})),p.addPageAction=S(w,"addPageAction",!0,n.D.pageAction),p.setPageViewName=function(t,r){if("string"==typeof t)return"/"!==t.charAt(0)&&(t="/"+t),(0,i.OP)(e).customTransaction=(r||"http://custom.transaction")+" "+t,S(w,"setPageViewName",!0)()},p.setCustomAttribute=function(e,t){let r=arguments.length>2&&void 0!==arguments[2]&&arguments[2];if("string"==typeof e){if("string"==typeof t||"number"==typeof t||"boolean"==typeof t||null===t)return _(e,t,"setCustomAttribute",r);(0,d.Z)("Failed to execute setCustomAttribute.\nNon-null value must be a string, number or boolean type, but a type of <".concat(typeof t,"> was provided."))}else(0,d.Z)("Failed to execute setCustomAttribute.\nName must be a string type, but a type of <".concat(typeof e,"> was provided."))},p.setUserId=function(e){if("string"==typeof e||"object"==typeof e||"number"==typeof e||"boolean"==typeof e||"undefined"==typeof e||"symbol"==typeof e||"bigint"==typeof e||"function"==typeof e||null===e)return _("enduser.id",e,"setUserId",!0);(0,d.Z)("Failed to execute setUserId.\nNon-null value must be a string type, but a type of <".concat(typeof e,"> was provided."))},p.setApplicationVersion=function(e){if("string"==typeof e||"object"==typeof e||"number"==typeof e||"boolean"==typeof e||"undefined"==typeof e||"symbol"==typeof e||"bigint"==typeof e||"function"==typeof e||null===e)return _("application.version",e,"setApplicationVersion",!1);(0,d.Z)("Failed to execute setApplicationVersion. Expected , but got <".concat(typeof e,">."))},p.start=()=>{try{(0,o.p)(l.xS,["API/start/called"],void 0,n.D.metrics,b),b.emit("manual-start-all")}catch(e){(0,d.Z)("An unexpected issue occurred",e)}},p[h.Ef.RECORD]=function(){(0,o.p)(l.xS,["API/recordReplay/called"],void 0,n.D.metrics,b),(0,o.p)(h.Ef.RECORD,[],void 0,n.D.sessionReplay,b)},p[h.Ef.PAUSE]=function(){(0,o.p)(l.xS,["API/pauseReplay/called"],void 0,n.D.metrics,b),(0,o.p)(h.Ef.PAUSE,[],void 0,n.D.sessionReplay,b)},p.interaction=function(e){return(new x).get("object"==typeof e?e:{})};const R=x.prototype={createTracer:function(e,t){var r={},i=this,a="function"==typeof t;return(0,o.p)(l.xS,["API/createTracer/called"],void 0,n.D.metrics,b),f||(0,o.p)(E+"tracer",[(0,m.z)(),e,r],i,n.D.spa,b),function(){if(A.emit((a?"":"no-")+"fn-start",[ (0,m.z)(),i,a ],r),a)try{return t.apply(this,arguments)}catch(e){const t="string"==typeof e?new Error(e):e;throw A.emit("fn-err",[arguments,this,t],r),t}finally{A.emit("fn-end",[ (0,m.z)() ],r)}}}};function S(e,t,r,i){return function(){return(0,o.p)(l.xS,["API/"+t+"/called"],void 0,n.D.metrics,b),i&&(0,o.p)(e+t,[ (0,m.z)(),...arguments ],r?null:this,i,b),r?void 0:this}}function T(){r.e(111).then(r.bind(r,7438)).then((t=>{let{setAPI:r}=t;r(e),(0,s.LP)(e,"api")})).catch((e=>{(0,d.Z)("Downloading runtime APIs failed...",e),b.abort()}))}return["actionText","setName","setAttribute","save","ignore","onEnd","getContext","end","get"].forEach((e=>{R[e]=S(E ``` -------------------------------- ### New Relic Agent Initialization Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Initializes and configures the New Relic agent, setting up essential properties and agent-specific configurations. ```javascript function l(){return function(){let e=a();const t=e.info||{};e.info={beacon:o.beacon,errorBeacon:o.errorBeacon,...t}}(),function(){let e=a();const t=e.init||{};e.init={...t}}(),s(),function(){let e=a();const t=e.loader_config||{};e.loader_config={...t}}(),a()}} ``` -------------------------------- ### httpx Agent Initialization and Resource Timing Source: https://github.com/projectdiscovery/httpx/blob/dev/common/httpx/test-data/hackerone.html Initializes the httpx agent, including distributed tracing, and captures resource timing information upon initialization. It also sets up event listeners for various network events. ```javascript static featureName=J.t; constructor(e,t){let r=!(arguments.length>2&&void 0!==arguments[2])||arguments[2];super(e,t,J.t,r),this.dt=new Y(e),this.handler=(e,t,r,n)=>(0,N.p)(e,t,r,n,this.ee);try{const e={xmlhttprequest:"xhr",fetch:"fetch",beacon:"beacon"}; p._A?.performance?.getEntriesByType("resource").forEach((t=>{ if(t.initiatorType in e&&0!==t.responseStatus){ const r={status:t.responseStatus}, n={rxSize:t.transferSize,duration:Math.floor(t.duration),cbTime:0}; oe(r,t.name),this.handler("xhr",\[r,n,t.startTime,t.responseEnd,e\[t.initiatorType\]\],void 0,a.D.ajax) } }))}catch(e){}(0,W.u5)(this.ee),(0,W.Kf)(this.ee),function(e,t,r,n){function i(e){var t=this; t.totalCbs=0,t.called=0,t.cbTime=0,t.end=E,t.ended=!1,t.xhrGuids={},t.lastSize=null,t.loadCaptureCalled=!1,t.params=this.params||{},t.metrics=this.metrics||{}, e.addEventListener("load",(function(r){ _(t,e) }),(0,I.m$)(!1)), p.IF||e.addEventListener("progress",(function(e){ t.lastSize=e.loaded }),(0,I.m$)(!1)) } function o(e){this.params={method:e[0]}, oe(this,e[1]), this.metrics={}} function c(t,r){ var i=(0,s.DL)(e); i.xpid&&this.sameOrigin&&r.setRequestHeader("X-NewRelic-ID",i.xpid); var o=n.generateTracePayload(this.parsedOrigin); if(o){ var a=!1; o.newrelicHeader&&(r.setRequestHeader("newrelic",o.newrelicHeader),a=!0), o.traceContextParentHeader&&(r.setRequestHeader("traceparent",o.traceContextParentHeader),o.traceContextStateHeader&&r.setRequestHeader("tracestate",o.traceContextStateHeader),a=!0), a&&(this.dt=o) } } function u(e,r){ var n=this.metrics, i=e[0], o=this; if(n&&i){ var a=Z(i); a&&(n.txSize=a) } this.startTime=(0,P.z)(), this.body=i }} ``` -------------------------------- ### Using httpx Runner Result Struct in Go Source: https://context7.com/projectdiscovery/httpx/llms.txt Demonstrates how to configure and run httpx with a custom result callback to access all fields of the runner.Result struct. Ensure all necessary options are validated before running. ```go package main import ( "encoding/json" "fmt" "log" "github.com/projectdiscovery/goflags" "github.com/projectdiscovery/httpx/runner" ) func main() { options := runner.Options{ InputTargetHost: goflags.StringSlice{"https://example.com"}, StatusCode: true, ExtractTitle: true, ContentLength: true, OutputServerHeader: true, TechDetect: true, OutputIP: true, OutputCName: true, OutputResponseTime: true, Favicon: true, TLSGrab: true, Asn: true, Jarm: true, Hashes: "md5,sha256", OutputWordsCount: true, OutputLinesCount: true, FollowRedirects: true, JSONOutput: true, Methods: "GET", OnResult: func(r runner.Result) { if r.Err != nil { fmt.Println("Error:", r.Err) return } // All fields available on Result: fmt.Println("URL: ", r.URL) fmt.Println("Input: ", r.Input) fmt.Println("StatusCode: ", r.StatusCode) fmt.Println("Title: ", r.Title) fmt.Println("WebServer: ", r.WebServer) fmt.Println("ContentLength:", r.ContentLength) fmt.Println("ContentType: ", r.ContentType) fmt.Println("Technologies: ", r.Technologies) fmt.Println("HostIP: ", r.HostIP) fmt.Println("CNAMEs: ", r.CNAMEs) fmt.Println("A records: ", r.A) fmt.Println("ResponseTime: ", r.ResponseTime) fmt.Println("Words: ", r.Words) fmt.Println("Lines: ", r.Lines) fmt.Println("FavIcon: ", r.FavIconMMH3) fmt.Println("JarmHash: ", r.JarmHash) fmt.Println("CDN: ", r.CDN, r.CDNName, r.CDNType) fmt.Println("HTTP2: ", r.HTTP2) fmt.Println("WebSocket: ", r.WebSocket) fmt.Println("Scheme: ", r.Scheme) fmt.Println("Port: ", r.Port) fmt.Println("Method: ", r.Method) fmt.Println("Location: ", r.Location) fmt.Println("FinalURL: ", r.FinalURL) fmt.Println("ChainCodes: ", r.ChainStatusCodes) if r.TLSData != nil { fmt.Println("TLS Version: ", r.TLSData.Version) fmt.Println("TLS Cipher: ", r.TLSData.Cipher) } if r.ASN != nil { fmt.Println("ASN: ", r.ASN.AsNumber, r.ASN.AsName, r.ASN.AsCountry) } // Pretty-print as JSON b, _ := json.MarshalIndent(r, "", " ") fmt.Println(string(b)) }, } if err := options.ValidateOptions(); err != nil { log.Fatal(err) } hr, err := runner.New(&options) if err != nil { log.Fatal(err) } defer hr.Close() hr.RunEnumeration() } ``` -------------------------------- ### High-level httpx Scanner Runner with Go Library Source: https://context7.com/projectdiscovery/httpx/llms.txt Create a configured runner instance and execute enumeration using the httpx Go library. The OnResult callback receives each Result struct as probing completes. ```go package main import ( "fmt" "log" "github.com/projectdiscovery/goflags" "github.com/projectdiscovery/gologger" "github.com/projectdiscovery/gologger/levels" "github.com/projectdiscovery/httpx/runner" ) func main() { gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent) options := runner.Options{ Methods: "GET", InputTargetHost: goflags.StringSlice{"scanme.sh", "projectdiscovery.io", "example.com"}, Threads: 25, Timeout: 10, StatusCode: true, ExtractTitle: true, ContentLength: true, OutputServerHeader: true, TechDetect: true, JSONOutput: true, FollowRedirects: true, MaxRedirects: 5, RandomAgent: true, OnResult: func(r runner.Result) { if r.Err != nil { fmt.Printf("[ERR] %s: %s\n", r.Input, r.Err) return } fmt.Printf("[%d] %s | Title: %q | Server: %s | Tech: %v | Length: %d\n", r.StatusCode, r.URL, r.Title, r.WebServer, r.Technologies, r.ContentLength) }, } if err := options.ValidateOptions(); err != nil { log.Fatal(err) } httpxRunner, err := runner.New(&options) if err != nil { log.Fatal(err) } defer httpxRunner.Close() httpxRunner.RunEnumeration() // Output: // [200] https://scanme.sh | Title: "scanme.sh" | Server: nginx | Tech: [Nginx] | Length: 4096 // [200] https://projectdiscovery.io | Title: "ProjectDiscovery" | Server: cloudflare | Tech: [Cloudflare] | Length: 28000 } ``` -------------------------------- ### httpx.Options Source: https://context7.com/projectdiscovery/httpx/llms.txt Details the configurable options for the httpx.HTTPX client, including defaults and how to override them for custom configurations like timeouts, redirects, proxies, and custom headers. ```APIDOC ## httpx.Options — Low-level client configuration All configurable options for the `httpx.HTTPX` client, with documented defaults. ### Default Options and Overrides ```go package main import ( "fmt" "log" "time" "github.com/projectdiscovery/httpx/common/httpx" "github.com/projectdiscovery/networkpolicy" ) func main() { // Start from defaults and override opts := httpx.DefaultOptions // opts.RandomAgent = true (default) // opts.Threads = 25 (default) // opts.Timeout = 30s (default) // opts.RetryMax = 5 (default) // opts.MaxRedirects = 10 (default) // opts.CdnCheck = "true" (default) // opts.VHostSimilarityRatio = 85 (default) opts.Timeout = 15 * time.Second opts.RetryMax = 2 opts.FollowRedirects = true opts.FollowHostRedirects = false opts.RespectHSTS = true opts.Unsafe = false opts.TLSGrab = true opts.ZTLS = false opts.TlsImpersonate = false opts.Proxy = "http://127.0.0.1:8080" opts.Resolvers = []string{"udp:8.8.8.8:53", "udp:1.1.1.1:53"} opts.SniName = "example.com" opts.CustomHeaders = map[string]string{ "Authorization": "Bearer mytoken", "Accept": "application/json", } opts.MaxResponseBodySizeToRead = 1024 * 1024 // 1MB // Network policy to block private IPs np, _ := networkpolicy.New(networkpolicy.Options{ DenyList: []string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"}, }) opts.NetworkPolicy = np client, err := httpx.New(&opts) if err != nil { log.Fatal(err) } req, _ := client.NewRequest("GET", "https://api.example.com/v1/status") client.SetCustomHeaders(req, opts.CustomHeaders) resp, err := client.Do(req, httpx.UnsafeOptions{}) if err != nil { log.Fatal(err) } fmt.Printf("Status: %d, Body length: %d bytes\n", resp.StatusCode, len(resp.Data)) } ``` ```