### Install Katana on Ubuntu
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Installs Katana on Ubuntu after prerequisites have been set up. Requires Go to be installed.
```console
go install github.com/projectdiscovery/katana/cmd/katana@latest
```
--------------------------------
### Install Prerequisites for Ubuntu
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Installs necessary prerequisites on Ubuntu, including Go and Google Chrome, before installing Katana.
```sh
sudo apt update
sudo apt install zip curl wget git snapd
sudo snap refresh
sudo snap install golang --classic
sudo install -d -m 0755 /etc/apt/keyrings
curl -fsSL https://dl.google.com/linux/linux_signing_key.pub \
| sudo gpg --dearmor -o /etc/apt/keyrings/google-chrome.gpg
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/google-chrome.gpg] \
http://dl.google.com/linux/chrome/deb/ stable main" \
| sudo tee /etc/apt/sources.list.d/google-chrome.list > /dev/null
sudo apt update
sudo apt install google-chrome-stable
```
--------------------------------
### Example Katana execution and output
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
This example demonstrates running Katana with a single URL and shows the typical output, including discovered URLs and informational messages.
```console
katana -u https://youtube.com
__ __
/ /_____ _/ /____ ____ ___ _
/ '_/ _ / __/ _ / _ \/ _ /
/_/\_\\_,_/
projectdiscovery.io
[WRN] Use with caution. You are responsible for your actions.
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://www.youtube.com/
https://www.youtube.com/about/
https://www.youtube.com/about/press/
https://www.youtube.com/about/copyright/
https://www.youtube.com/t/contact_us/
https://www.youtube.com/creators/
https://www.youtube.com/ads/
https://www.youtube.com/t/terms
https://www.youtube.com/t/privacy
https://www.youtube.com/about/policies/
https://www.youtube.com/howyoutubeworks?utm_campaign=ytgen&utm_source=ythp&utm_medium=LeftNav&utm_content=txt&u=https%3A%2F%2Fwww.youtube.com%2Fhowyoutubeworks%3Futm_source%3Dythp%26utm_medium%3DLeftNav%26utm_campaign%3Dytgen
https://www.youtube.com/new
https://m.youtube.com/
https://www.youtube.com/s/desktop/4965577f/jsbin/desktop_polymer.vflset/desktop_polymer.js
https://www.youtube.com/s/desktop/4965577f/cssbin/www-main-desktop-home-page-skeleton.css
https://www.youtube.com/s/desktop/4965577f/cssbin/www-onepick.css
https://www.youtube.com/s/_/ytmainappweb/_/ss/k=ytmainappweb.kevlar_base.0Zo5FUcPkCg.L.B1.O/am=gAE/d=0/rs=AGKMywG5nh5Qp-BGPbOaI1evhF5BVGRZGA
https://www.youtube.com/opensearch?locale=en_GB
https://www.youtube.com/manifest.webmanifest
https://www.youtube.com/s/desktop/4965577f/cssbin/www-main-desktop-watch-page-skeleton.css
https://www.youtube.com/s/desktop/4965577f/jsbin/web-animations-next-lite.min.vflset/web-animations-next-lite.min.js
https://www.youtube.com/s/desktop/4965577f/jsbin/custom-elements-es5-adapter.vflset/custom-elements-es5-adapter.js
https://www.youtube.com/s/desktop/4965577f/jsbin/webcomponents-sd.vflset/webcomponents-sd.js
https://www.youtube.com/s/desktop/4965577f/jsbin/intersection-observer.min.vflset/intersection-observer.min.js
https://www.youtube.com/s/desktop/4965577f/jsbin/scheduler.vflset/scheduler.js
https://www.youtube.com/s/desktop/4965577f/jsbin/www-i18n-constants-en_GB.vflset/www-i18n-constants.js
https://www.youtube.com/s/desktop/4965577f/jsbin/www-tampering.vflset/www-tampering.js
https://www.youtube.com/s/desktop/4965577f/jsbin/spf.vflset/spf.js
https://www.youtube.com/s/desktop/4965577f/jsbin/network.vflset/network.js
https://www.youtube.com/howyoutubeworks/
https://www.youtube.com/trends/
https://www.youtube.com/jobs/
https://www.youtube.com/kids/
```
--------------------------------
### Install Katana via Go
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Installs the latest version of Katana using the Go build tool. Ensure Go 1.25+ is installed.
```console
CGO_ENABLED=1 go install github.com/projectdiscovery/katana/cmd/katana@latest
```
--------------------------------
### Basic Katana Usage Example
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
This is the basic command structure for running Katana. Replace '[flags]' with the desired configuration options.
```console
./katana [flags]
```
--------------------------------
### Raw HTTP Response Example
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
This is an example of a raw HTTP response, including headers and HTML content, as might be stored or displayed by Katana.
```http
HTTP/1.1 200 OK
Content-Length: 1256
Accept-Ranges: bytes
Age: 331239
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Mon, 20 Mar 2023 10:53:58 GMT
Etag: "3147526947"
Expires: Mon, 27 Mar 2023 10:53:58 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7EA3)
Vary: Accept-Encoding
X-Cache: HIT
Example Domain
Example Domain
This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.
More information...
```
--------------------------------
### Load Custom Headers from File
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Supply custom headers or cookies from a file to perform authenticated crawling. Ensure the file format matches the example.
```bash
$ cat cookie.txt
Cookie: PHPSESSIONID=XXXXXXXXX
X-API-KEY: XXXXX
TOKEN=XX
```
```bash
katana -u https://tesla.com -H cookie.txt
```
--------------------------------
### Run Katana with Docker (Standard Mode)
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Executes Katana in standard mode using Docker, crawling a specified URL. Ensure Docker is installed.
```sh
docker pull projectdiscovery/katana:latest
docker run projectdiscovery/katana:latest -u https://tesla.com
```
--------------------------------
### Run Katana with a single URL
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use the -u flag to specify a single URL for Katana to crawl. This is the most basic way to start a crawl.
```sh
katana -u https://tesla.com
```
--------------------------------
### Run Katana with Docker (Headless Mode)
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Executes Katana in headless mode using Docker, enabling JavaScript rendering for crawling. Ensure Docker is installed.
```sh
docker run projectdiscovery/katana:latest -u https://tesla.com -system-chrome -headless
```
--------------------------------
### Use Local Chrome Browser
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use a locally installed Chrome browser instead of the one bundled with Katana by specifying the -system-chrome flag.
```console
katana -u https://tesla.com -headless -system-chrome
```
--------------------------------
### Example: Filter URLs with Query Parameters
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Demonstrates how to use the `-f qurl` flag to display only URLs that contain query parameters. The `-silent` flag is used to suppress additional output.
```bash
katana -u https://tesla.com -f qurl -silent
https://shop.tesla.com/en_au?redirect=no
https://shop.tesla.com/en_nz?redirect=no
https://shop.tesla.com/product/men_s-raven-lightweight-zip-up-bomber-jacket?sku=1740250-00-A
https://shop.tesla.com/product/tesla-shop-gift-card?sku=1767247-00-A
https://shop.tesla.com/product/men_s-chill-crew-neck-sweatshirt?sku=1740176-00-A
https://www.tesla.com/about?redirect=no
https://www.tesla.com/about/legal?redirect=no
https://www.tesla.com/findus/list?redirect=no
```
--------------------------------
### Start Chrome with Remote Debugging
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Initiate a Chrome browser instance with remote debugging enabled on a specific port. This allows Katana to connect to an active session.
```console
$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
DevTools listening on ws://127.0.0.1:9222/devtools/browser/c5316c9c-19d6-42dc-847a-41d1aeebf7d6
```
--------------------------------
### Custom Field Configuration Example
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Defines custom fields for extracting email addresses and phone numbers using regular expressions. This YAML configuration can be loaded by Katana.
```yaml
- name: email
type: regex
regex:
- '([a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)'
- '([a-zA-Z0-9+._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)'
- name: phone
type: regex
regex:
- '\d{3}-\d{8}|\d{4}-\d{7}'
```
--------------------------------
### Connect Katana to Active Browser Session
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Crawl a website by connecting Katana to an active Chrome browser session. Use the provided websocket URL obtained from starting Chrome with remote debugging.
```console
katana -headless -u https://tesla.com -cwu ws://127.0.0.1:9222/devtools/browser/c5316c9c-19d6-42dc-847a-41d1aeebf7d6 -no-incognito
```
--------------------------------
### Run Without Incognito Mode
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Disable incognito mode for headless Chrome using the -no-incognito flag. This is beneficial when using a local browser installation to preserve session data.
```console
katana -u https://tesla.com -headless -no-incognito
```
--------------------------------
### Katana Output Options Help
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Display help information for Katana's output-related command-line options.
```bash
katana -h output
```
```bash
OUTPUT:
-o, -output string file to write output to
-sr, -store-response store http requests/responses
-srd, -store-response-dir string store http requests/responses to custom directory
-lof, -list-output-fields list available fields for jsonl output format
-eof, -exclude-output-fields exclude fields from jsonl output
-j, -json write output in JSON Lines format
-nc, -no-color disable output content coloring (ANSI escape codes)
-silent display output only
-v, -verbose display verbose output
-version display project version
```
--------------------------------
### Initialize and Use Katana Crawler as Library
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Demonstrates how to configure and run the Katana crawler programmatically. Set various crawling options such as depth, scope, timeouts, and concurrency. The `OnResult` callback is used to process each discovered URL.
```go
package main
import (
"math"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/katana/pkg/engine/standard"
"github.com/projectdiscovery/katana/pkg/output"
"github.com/projectdiscovery/katana/pkg/types"
)
func main() {
options := &types.Options{
MaxDepth: 3, // Maximum depth to crawl
FieldScope: "rdn", // Crawling Scope Field
BodyReadSize: math.MaxInt, // Maximum response size to read
Timeout: 10, // Timeout is the time to wait for request in seconds
Concurrency: 10, // Concurrency is the number of concurrent crawling goroutines
Parallelism: 10, // Parallelism is the number of urls processing goroutines
Delay: 0, // Delay is the delay between each crawl requests in seconds
RateLimit: 150, // Maximum requests to send per second
Strategy: "depth-first", // Visit strategy (depth-first, breadth-first)
OnResult: func(result output.Result) { // Callback function to execute for result
gologger.Info().Msg(result.Request.URL)
},
}
crawlerOptions, err := types.NewCrawlerOptions(options)
if err != nil {
gologger.Fatal().Msg(err.Error())
}
defer crawlerOptions.Close()
crawler, err := standard.New(crawlerOptions)
if err != nil {
gologger.Fatal().Msg(err.Error())
}
defer crawler.Close()
var input = "https://www.hackerone.com"
err = crawler.Crawl(input)
if err != nil {
gologger.Warning().Msgf("Could not crawl %s: %s", input, err.Error())
}
}
```
--------------------------------
### Scope Control: Help Command
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Display all available CLI options for scope control by running 'katana -h scope'.
```console
katana -h scope
```
--------------------------------
### Display Katana Help Information
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Run this command to display the help message and view all supported command-line switches for the Katana tool.
```console
katana -h
```
--------------------------------
### Run Katana with a list of URLs from a file
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Specify a file containing a list of URLs using the -list option. Each URL should be on a new line. This is useful for scanning a large number of targets.
```bash
$ cat url_list.txt
https://tesla.com
https://google.com
```
```bash
katana -list url_list.txt
```
--------------------------------
### Return-to-Origin Algorithm Steps
Source: https://github.com/projectdiscovery/katana/blob/dev/pkg/engine/headless/TODOS.md
Details the three-level approach for navigating back to a specific origin state. It prioritizes element re-use, then browser history, and finally graph shortest path, with specific exit conditions and fallback mechanisms.
```go
Step 0 Fast-fail: if currentID == target → done.
Step 1 Element re-use
• If `action.Element` is non-nil, locate by XPath, ensure Visible & Interactable,
*plus* DOM equality check under the canonicalizer to avoid false positives.
• If match, return targetOriginID.
Step 2 Browser history
• page.GetNavigationHistory()
• Walk back until (url == origin.URL && title == origin.Title).
• Limit: max 10 steps to avoid long loops.
• After each back() call wait with WaitForRouteChange() (new detector described below).
• Recompute fingerprint; if equal (exact or fuzzy) → success.
Step 3 Graph shortest path
• crawlerGraph.ShortestPath(currentID, targetID).
• If unreachable, retry from emptyPageHash (fresh tab).
• Execute each Action; after each, WaitForRouteChange().
• After final step verify state (same equality logic as Step 2).
• Failure → ErrNoNavigationPossible.
```
--------------------------------
### Katana Rate Limit CLI Options
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Lists all long and short CLI options available for rate limit control in Katana.
```console
katana -h rate-limit
Flags:
RATE-LIMIT:
-c, -concurrency int number of concurrent fetchers to use (default 10)
-p, -parallelism int number of concurrent inputs to process (default 10)
-rd, -delay int request delay between each request in seconds
-rl, -rate-limit int maximum requests to send per second (default 150)
-rlm, -rate-limit-minute int maximum number of requests to send per minute
-hrl, -host-rate-limit int maximum requests to send per second per host
-hrlm, -host-rate-limit-minute int maximum number of requests to send per minute per host
```
--------------------------------
### newPageState Algorithm
Source: https://github.com/projectdiscovery/katana/blob/dev/pkg/engine/headless/TODOS.md
Outlines the algorithm for creating a new PageState. It involves grabbing page info, normalizing the HTML, computing ExactHash and FuzzyHash, and optionally saving diagnostics. Handles edge cases like empty pages or normalizer failures.
```go
Algorithm newPageState(page, causingAction):
1. Grab `page.Info()`; bail out if URL is empty or about:blank.
2. outerHTML := page.HTML().
3. stripped := domNormalizer.Apply(outerHTML).
4. Build PageState as above.
5. Compute hashes as described.
6. Diagnostics hook (save stripped DOM, screenshots, etc.).
7. Return the fully populated PageState.
```
--------------------------------
### Crawler Configuration: Known Files
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Enable crawling of 'robots.txt' and 'sitemap.xml' by specifying them with the '-kf' option. This is disabled by default.
```console
katana -u https://tesla.com -kf robotstxt,sitemapxml
```
--------------------------------
### Crawler Configuration: Filter Similar Threshold
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Tune the promotion threshold for filtering similar URLs with '-fst'. Lower values are more aggressive, higher values are more permissive. Default is 10.
```console
katana -u https://tesla.com -fsu -fst 5
```
--------------------------------
### Configure Katana CLI Options
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
View all configuration-related CLI options for Katana. Use the '-h config' command to access this information.
```console
katana -h config
Flags:
CONFIGURATION:
-r, -resolvers string[] list of custom resolver (file or comma separated)
-d, -depth int maximum depth to crawl (default 3)
-jc, -js-crawl enable endpoint parsing / crawling in javascript file
-ct, -crawl-duration int maximum duration to crawl the target for
-kf, -known-files string enable crawling of known files (all,robotstxt,sitemapxml)
-mrs, -max-response-size int maximum response size to read (default 9223372036854775807)
-timeout int time to wait for request in seconds (default 10)
-aff, -automatic-form-fill enable automatic form filling (experimental)
-fx, -form-extraction enable extraction of form, input, textarea & select elements
-retry int number of times to retry the request (default 1)
-proxy string http/socks5 proxy to use
-H, -headers string[] custom header/cookie to include in request
-config string path to the katana configuration file
-fc, -form-config string path to custom form configuration file
-flc, -field-config string path to custom field configuration file
-s, -strategy string Visit strategy (depth-first, breadth-first) (default "depth-first")
-iqp, -ignore-query-params Ignore crawling same path with different query-param values
-fsu, -filter-similar filter crawling of similar looking URLs (e.g., /users/123 and /users/456)
-fst, -filter-similar-threshold int number of distinct values before a path position is treated as parameter (default 10)
-mdp, -max-domain-pages int maximum number of pages to crawl per domain (default unlimited)
```
--------------------------------
### List Available JSONL Output Fields
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Employ the -list-output-fields or -lof flag to display all fields available for JSONL output. This is helpful for understanding data for custom output templates or field exclusion.
```bash
katana -lof
```
--------------------------------
### Show Browser Window
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Display the browser window on the screen during headless mode execution with the -show-browser flag. Useful for debugging.
```console
katana -u https://tesla.com -headless -show-browser
```
--------------------------------
### Match Extensions with Katana
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use the `-em` or `-extension-match` option to display only output containing the specified file extensions. Include `none` to also include URLs without extensions.
```console
katana -u https://tesla.com -silent -em js,jsp,json
```
```console
katana -u https://tesla.com -silent -em js,jsp,json,none
```
--------------------------------
### Run Katana with piped input (STDIN)
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Pipe URLs directly into Katana using standard input (STDIN). This is convenient for integrating Katana into existing command-line workflows.
```sh
echo https://tesla.com | katana
```
```sh
cat domains | httpx | katana
```
--------------------------------
### Scope Control: Crawl Scope from File
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
For multiple in-scope rules, provide a file path to '-cs'. Each line in the file is treated as a regex pattern.
```bash
$ cat in_scope.txt
login/
admin/
app/
wordpress/
```
```console
katana -u https://tesla.com -cs in_scope.txt
```
--------------------------------
### Configure Captcha Solving via Environment Variables
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Set the captcha solver provider and API key using environment variables CAPTCHA_SOLVER_PROVIDER and CAPTCHA_SOLVER_KEY for headless crawling.
```console
export CAPTCHA_SOLVER_PROVIDER=capsolver
export CAPTCHA_SOLVER_KEY=YOUR_API_KEY
katana -u https://example.com -headless
```
--------------------------------
### Customize Output Format with Templates
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Employ the -output-template option to define custom output structures using templates. This allows for flexible data presentation, and non-existent fields are omitted.
```sh
katana -u https://example.com -output-template '{{email}} - {{url}}'
```
--------------------------------
### Run Katana with Custom Fields
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use the `-f` or `-field` option to specify which fields to display in the output. This is useful for filtering output at runtime.
```console
katana -u https://tesla.com -f email,phone
```
--------------------------------
### Run Katana with multiple URLs
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Provide multiple URLs for crawling by separating them with commas after the -u flag. This allows for scanning several targets in one command.
```sh
katana -u https://tesla.com,https://google.com
```
--------------------------------
### Run Without Sandbox
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Execute headless Chrome in --no-sandbox mode using the -no-sandbox flag. This is particularly useful when running Katana as a root user.
```console
katana -u https://tesla.com -headless -no-sandbox
```
--------------------------------
### Page Ready Detector (WaitForRouteChange)
Source: https://github.com/projectdiscovery/katana/blob/dev/pkg/engine/headless/TODOS.md
Describes the replacement for brittle WaitPageLoadHeuristics. This involves injecting JavaScript into the tab to detect when the page is considered 'ready', likely by monitoring specific DOM events or network activity.
```go
Replace the brittle WaitPageLoadHeuristics with:
Injected JS once per tab:
```
--------------------------------
### Match Endpoints by Technology using DSL
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use the `-mdc` or `-match-condition` flag with a DSL expression to match endpoints based on their technologies (case-insensitive). Requires the `dsl` project.
```shell
katana -u https://www.hackerone.com -mdc 'contains(to_lower(technologies), "php")'
```
--------------------------------
### Enable Headless Mode
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Enable headless hybrid crawling using the -headless option. This mode allows for better HTTP fingerprinting and analysis of both raw and JavaScript-rendered responses.
```console
katana -u https://tesla.com -headless
```
--------------------------------
### Match Endpoints by Status Code using DSL
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Use the `-mdc` or `-match-condition` flag with a DSL expression to match endpoints based on their status code. Requires the `dsl` project.
```shell
katana -u https://www.hackerone.com -mdc 'status_code == 200'
```
--------------------------------
### Output Results in JSONL Format
Source: https://github.com/projectdiscovery/katana/blob/dev/README.md
Enable JSON Lines output with the -jsonl flag for structured data, which can be piped to tools like jq for further processing.
```console
katana -u https://example.com -jsonl | jq .
```
```json
{
"timestamp": "2023-03-20T16:23:58.027559+05:30",
"request": {
"method": "GET",
"endpoint": "https://example.com",
"raw": "GET / HTTP/1.1\r\nHost: example.com\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36\r\nAccept-Encoding: gzip\r\n\r\n"
},
"response": {
"status_code": 200,
"headers": {
"accept_ranges": "bytes",
"expires": "Mon, 27 Mar 2023 10:53:58 GMT",
"last_modified": "Thu, 17 Oct 2019 07:18:26 GMT",
"content_type": "text/html; charset=UTF-8",
"server": "ECS (dcb/7EA3)",
"vary": "Accept-Encoding",
"etag": "\"3147526947\"",
"cache_control": "max-age=604800",
"x_cache": "HIT",
"date": "Mon, 20 Mar 2023 10:53:58 GMT",
"age": "331239"
},
"body": "\n\n\n Example Domain\n\n \n \n \n \n\n\n\n\n
Example Domain
\n
This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.
\n
More information...
\n