### Install WhatWeb System-Wide (Makefile)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs WhatWeb across the system using the Makefile. This command is typically run on Linux or macOS after obtaining the WhatWeb source code.

```bash
make install
```

--------------------------------

### Verify WhatWeb Installation (Bash)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Checks if WhatWeb has been installed correctly by running the version command. This command should be executed from the WhatWeb source folder after dependency installation.

```bash
./whatweb --version
```

--------------------------------

### Install Ruby Dependencies using Bundler (Bash)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs necessary Ruby dependencies for WhatWeb using Bundler. This involves installing Bundler itself, updating it, and then executing 'bundle install' within the WhatWeb source directory.

```bash
gem install bundler
```

```bash
bundle update
```

```bash
bundle install
```

--------------------------------

### WhatWeb Output Performance Optimization Examples

Source: https://github.com/urbanadventurer/whatweb/blob/master/README.md

Provides examples of command-line arguments for optimizing WhatWeb's output performance, including forcing real-time output, setting custom buffer sizes, and disabling buffering entirely for immediate file writes.

```bash
# Force real-time output to screen and logs (slower but immediate)
./whatweb --output-sync -t 100 --log-brief=results.txt targets.txt
```

```bash
# Custom buffer size for high-volume logging
./whatweb --output-buffer-size=50 -t 100 --log-verbose=detailed.log targets.txt
```

```bash
# Disable buffering entirely for immediate file writes
./whatweb --output-buffer-size=0 -t 10 --log-json=results.json targets.txt
```

--------------------------------

### Enable Verbose Output in WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

To get more detailed output that explains the meaning of detected plugins, use the '--verbose' or '-v' flag. This option provides insights into the plugin matching process.

```bash
whatweb --verbose reddit.com
```

```bash
whatweb -v reddit.com
```

--------------------------------

### Install All Optional Dependencies for WhatWeb

Source: https://github.com/urbanadventurer/whatweb/blob/master/README.md

Installs all optional dependencies for WhatWeb, including MongoDB support and character set detection. It uses `bundle install --with mongo rchardet`. For older Bundler versions, it specifies both `mongo` and `rchardet` groups.

```bash
# Install all optional groups
bundle install --with mongo rchardet

# If you're using an older version of Bundler
GEMFILE_GROUPS="default mongo rchardet" bundle install
```

--------------------------------

### Scan for Specific Files in a List of Hosts with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

This example demonstrates scanning a list of hosts from a file for a specific file (e.g., '/crossdomain.xml') using '--url-suffix' and targeting a particular plugin ('-p crossdomain_xml').

```bash
whatweb -i plugin-development/alexa-top-100.txt --url-suffix /crossdomain.xml -p crossdomain_xml
```

--------------------------------

### Install Ruby and Dependencies on Debian/Ubuntu (Bash)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs Ruby and essential development libraries on Debian-based systems like Ubuntu using the apt package manager. These are prerequisites for running WhatWeb.

```bash
sudo apt install ruby ruby-dev libopenssl-ruby
```

--------------------------------

### Define Example Websites (Ruby)

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

This snippet shows how to define an array of example website URLs for scanning. The %w| syntax creates an array of strings separated by whitespace. If a URL is missing a scheme (http:// or https://), http:// is assumed.

```ruby
examples %w| amnesty.org/ appel.nasa.gov/ beta.worldbank.org/ entergy.pewclimate.org/ labs.divx.com/ lindenlab.com/ littlestarprints.com moby.com/ myplay.com/ sequelnaturals.com/ teen.secondlife.com/ www.artwaves.de www.asys.com.br/ www.atomicbop.net www.cristal.com.pe/?adulto=si www.dutchbutnotfromholland.eu/ www.elespectador.com/ www.ensembles.com.ph/ www.foxsearchlight.com/index.php www.freshbrain.org/ www.icsalabs.com/ www.johnnycashonline.com/ www.journalismcenter.org/ www.jovenscriativos.com.br/ www.koalafoundation.org.au/ www.la2day.com/ www.moove.be www.mtv.co.uk/channel/flux www.mulinobianco.it/ www.multiways.com/ www.nowpublic.com/ www.pravda.lt/ www.realismssoftware.com/ www.save-your-future.com www.shock.com.co/ www.sosojuicy.com/ www.spreadfirefox.com/ www.tidningenresultat.se www.ubuntu.com/ www.universitytowers.net/ www.warnerbrosrecords.com |
```

```ruby
examples %w|
http://www.example.com
http://www.example2.com
http://www.site.com/blah/
|
```

--------------------------------

### WhatWeb Profiling Examples

Source: https://github.com/urbanadventurer/whatweb/blob/master/README.md

Shows how to enable and configure performance profiling in WhatWeb to identify bottlenecks. It includes options for directing profile output to stderr or a file, and notes the requirement for the `ruby-prof` gem.

```bash
# Basic profiling to stderr
WHATWEB_PROFILE=1 ./whatweb -t 25 targets.txt
```

```bash
# Save profile to file
WHATWEB_PROFILE=1 WHATWEB_PROFILE_FILE=profile.txt ./whatweb -t 25 targets.txt
```

--------------------------------

### WhatWeb Banner Grabbing Example

Source: https://github.com/urbanadventurer/whatweb/wiki/Advanced-Usage

Shows how to perform banner grabbing by appending port numbers to domain names or by using --url-pattern with stdin input. This example attempts to grab banners from SSH and FTP ports.

```bash
# Banner grabbing by appending port numbers
./whatweb -a 1 whatweb.net:21 whatweb.net:22

# Banner grabbing using stdin and --url-pattern
echo "21,22" | tr ',' '\n' | ./whatweb -a 1 --url-pattern whatweb.net:%insert% --input-file /dev/stdin
```

--------------------------------

### SilverStripe WhatWeb Plugin Example

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

An example of a WhatWeb plugin for SilverStripe CMS. It demonstrates how to fill in the plugin details and define matches using regular expressions and text patterns to identify SilverStripe instances.

```ruby
Plugin.define "SilverStripe" do
    author "Andrew Horton"
    version "0.1"
    description "SilverStripe is an opensource CMS written in PHP. It can run on Apache, IIS or lighthttpd. Homepage: http://www.silverstripe.com"

    examples %w|http://beatone.co.uk/ http://charcoalinteriors.com.au/ http://customcanvas.fritzandandre.com/ http://hungryhearts.no http://maungataniwha.co.nz/ http://unbounded.org/ http://victoriaoruwari.com/ http://weonline.in http://www.arprostatecancer.org/ http://www.benpearce.co.nz/ http://www.bradyinc.com/ http://www.cavendishimaging.com/ http://www.chapmansurfboards.com/ http://www.choidoco.com/demo/ http://www.clockwork.co.nz/ http://www.enamaine.org/ http://www.executivemediasearch.com/ http://www.fairtradenap.net/ http://www.firstgalaxies.org/ http://www.frussian.com.ar/ http://www.fuel.ie/silverstripe http://www.gsbc.edu/ http://www.holistichealth.com/ http://www.hutmacherin.com/ http://www.infinitestillness.ie/ss http://www.intandemtheatre.org/ http://www.kitesurfnelson.co.nz/ http://www.latenightdisco.com/ http://www.lisamarieelliott.com/ http://www.maklerservice-greiz.de/ http://www.moerakihavenmotel.co.nz/ http://www.monjasantner.de/ http://www.moonlitekustoms.com/ http://www.moto-racepaint.com/ http://www.naciondnb.com/ http://www.nadabakery.co.nz/ http://www.peterpanvakantieclub.nl/ http://www.rcaforum.org.nz/ http://www.robert80.de/ http://www.silverstripe.com/ http://www.silverstripe.org.pl/ http://www.stillrunnin.com/ http://www.textiprints.com/ http://www.thelightboxdesigns.com/ http://www.tobychampion.co.uk/ http://www.upstreamgroup.com/ http://www.verus.com.tr/ http://www.wend.nl/ http://www.whileyouwait.co.nz/ |

    # a comment block here is a good place to make notes for yourself and others

    # There are four types of matches: regexp, text, ghdb
    # Matches are enclosed in {} brackets and separated by commas
    matches [
    {:name=>"a brief description of the match, eg. powered by in footer",
    :certainty=>100, # 100 is certain, 75 is probably and 25 is maybe. Optional
    :regexp=>/This page was generated by <a href="http:\/\/www.genericcms.com\/en\/products\/generic-cms\/">Generic CMS<\/a>/ },

    {:name=>"title",
    :certainty=>75,
    :text=>"<title>Generic Homepage</title>" }
    ]
    end
```

--------------------------------

### WhatWeb Plugin Selection Example

Source: https://github.com/urbanadventurer/whatweb/wiki/Advanced-Usage

Demonstrates how to select plugins using the -p or --plugin command-line option. Plugins can be specified by directory, file, or name, with modifiers to add or remove from the default set.

```bash
# Select plugins by directory, file, or name with modifiers
./whatweb --plugins +plugins-disabled,-foobar
./whatweb --plugins +/tmp/moo.rb
./whatweb --plugins foobar
./whatweb -p title,md5,+./plugins-disabled/
./whatweb -p ./plugins-disabled,-md5
```

--------------------------------

### Install Ruby and Dependencies on Fedora (Bash)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs Ruby and related packages on Fedora-based systems using dnf (or yum). This ensures the Ruby environment is set up correctly for WhatWeb.

```bash
sudo dnf -y install ruby ruby-dev rubygems
```

--------------------------------

### Install MongoDB Dependencies for WhatWeb

Source: https://github.com/urbanadventurer/whatweb/blob/master/README.md

Installs the necessary Ruby gems for MongoDB logging in WhatWeb. It uses `bundle install` with the `--with mongo` flag. For older Bundler versions, it specifies the `mongo` group.

```bash
# Install dependencies
bundle install --with mongo

# If you're using an older version of Bundler
GEMFILE_GROUPS="default mongo" bundle install
```

--------------------------------

### Download and Extract WhatWeb Release (Bash)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Instructions for downloading a stable release of WhatWeb as a ZIP or tar.gz archive and extracting it. This method is suitable for users who prefer not to use Git.

```bash
unzip WhatWeb-v0.5.2.zip
cd WhatWeb-0.5.2/
```

```bash
tar -zxvf WhatWeb-v0.5.2.tar.gz
cd WhatWeb-0.5.2/
```

--------------------------------

### Install Optional MongoDB Dependencies (Ruby Gems)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs Ruby gems required for WhatWeb to log data to MongoDB databases. These are optional dependencies and not strictly necessary for WhatWeb's core functionality.

```ruby
sudo gem install bson
```

```ruby
sudo gem install bson_ext
```

```ruby
sudo gem install mongo
```

--------------------------------

### Enable Really Verbose Output in WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

For even more detailed information, especially for debugging plugin behavior, you can use the '--verbose' flag twice ('-vv'). This shows precisely how each plugin is matched.

```bash
whatweb -vv reddit.com
```

--------------------------------

### Scan Multiple Websites with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

WhatWeb allows scanning multiple websites simultaneously by listing their URLs on the command line. This is useful for quickly assessing several targets.

```bash
whatweb reddit.com slashdot.org twitter.com
```

--------------------------------

### WhatWeb Plugin Template

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

This is the basic template for creating a WhatWeb plugin. It includes fields for plugin name, author, version, description, examples, and match definitions. The 'matches' array defines how the plugin identifies software.

```ruby
Plugin.define "Plugin-Template" do
    author "Enter Your Name"
    version "0.1"
    description "Describe what the plugin identifies. Include the homepage of the software package"
    examples %w| include-some.net example-websites.com here.com |

    # a comment block here is a good place to make notes for yourself and others

    # There are four types of matches: regexp, text, ghdb
    # Matches are enclosed in {} brackets and separated by commas
    matches [
    {:name=>"a brief description of the match, eg. powered by in footer",
    :certainty=>100, # 100 is certain, 75 is probably and 25 is maybe. if omitted, it defaults to 100.
    :regexp=>/This page was generated by <a href="http:\/\/www.genericcms.com\/en\/products\/generic-cms\/">Generic CMS<\/a>/ },

    {:name=>"title",
    :certainty=>75,
    :text=>"<title>Generic Homepage</title>" }
    ]
    end
```

--------------------------------

### Scan a Single Website with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

The most basic usage of WhatWeb involves specifying a single website URL on the command line for scanning. This command initiates a scan of the provided domain.

```bash
./whatweb reddit.com
http://reddit.com [301 Moved Permanently] Country[UNITED STATES][US], HTTPServer[snooserv], IP[151.101.65.140], RedirectLocation[https://www.reddit.com/], UncommonHeaders[retry-after,x-served-by,x-cache-hits,x-timer], Via-Proxy[1.1 varnish]
https://www.reddit.com/ [200 OK] Cookies[edgebucket,eu_cookie_v2,loid,rabt,rseor3,session_tracker,token], Country[UNITED STATES][US], Email[banner@2x.png,snoo-home@2x.png], Frame, HTML5, HTTPServer[snooserv], HttpOnly[token], IP[151.101.37.140], Open-Graph-Protocol[website], Script[text/javascript], Strict-Transport-Security[max-age=15552000; includeSubDomains; preload], Title[reddit: the front page of the internet], UncommonHeaders[fastly-restarts,x-served-by,x-cache-hits,x-timer], Via-Proxy[1.1 varnish], X-Frame-Options[SAMEORIGIN]
```

```bash
whatweb example.com
```

--------------------------------

### Pipe URLs to WhatWeb via STDIN

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

WhatWeb can accept a list of URLs piped from standard input (STDIN) using the '-i /dev/stdin' option. This enables integration with other command-line tools that generate lists of URLs.

```bash
echo -e "facebook.com\ninstagram.com\ntwitter.com" | ./whatweb -i /dev/stdin
```

--------------------------------

### Basic Netcat and Grep for Title Extraction

Source: https://github.com/urbanadventurer/whatweb/wiki/Discussion

A simple command-line example using netcat to fetch a webpage and grep to extract the content within the <title> tags. This illustrates a manual approach that WhatWeb's custom plugin feature aims to simplify.

```bash
echo "\n\n" | netcat whatweb.net 80 | grep -Eo "<title>([^<]+)<\/title>"
```

--------------------------------

### Match CSS Files for Layout, Typography, and Forms

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

This example demonstrates how to identify a website by matching specific CSS file names in the HTML. It looks for 'layout.css', 'typography.css', and 'form.css' linked within the page.

```html
<link rel="stylesheet" type="text/css" href="http://www.textiprints.com/themes/textiprints/css/layout.css?m=1266347738" />
```

--------------------------------

### WhatWeb Automation with Gggooglescan

Source: https://github.com/urbanadventurer/whatweb/wiki/Advanced-Usage

Demonstrates automating WhatWeb by piping the output of gggooglescan to WhatWeb for fingerprinting. This example searches for WordPress sites and logs the results.

```bash
# Pipe gggooglescan output to WhatWeb for fingerprinting
./gggooglescan -v "powered by wordpress" | ./whatweb -i /dev/stdin --log-brief wordpress.log
```

--------------------------------

### Scan HTTPS Websites in a Network Range with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

To scan for HTTPS websites within a network range, use the '--url-prefix https://' option along with the network range. This ensures that requests are made over HTTPS.

```bash
whatweb --no-errors --url-prefix https:// 192.168.0.0/24
```

--------------------------------

### Get Latest WhatWeb Development Version (Git)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Clones the latest development version of WhatWeb from its GitHub repository. This is useful for developers or users who want the most recent features and bug fixes.

```bash
git clone https://github.com/urbanadventurer/WhatWeb.git
cd WhatWeb/
```

--------------------------------

### Install Optional RChardet Dependency (Ruby Gem)

Source: https://github.com/urbanadventurer/whatweb/wiki/Installation

Installs the RChardet gem, which is used for character set detection. This is required for converting results to UTF-8, a dependency for logging to MongoDB, but it may slow performance.

```ruby
sudo gem install rchardet
```

--------------------------------

### WhatWeb Plugin Testing Commands

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

These commands demonstrate how to test a WhatWeb plugin during development. The first command lists all loaded plugins, while the second runs the specific SilverStripe plugin against collected sample files with verbose output.

```bash
$ ./whatweb -l
```

```bash
$ ./whatweb -v -psilverstripe ./plugin-development/tests/silverstripe/*html
```

--------------------------------

### Match Image Asset URL Structure with Regex

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

This snippet uses a regular expression to match a specific image asset URL structure commonly found on SilverStripe websites. The pattern targets URLs starting with '/assets/' followed by a directory, then '/_resampled/', and ending with a .jpg extension. It's assigned a certainty of 75% due to some non-SilverStripe examples found.

```regex
<img src="/assets/[^/]+/_resampled/[^"].jpg"
```

--------------------------------

### Ruby Plugin Matches Array Example

Source: https://github.com/urbanadventurer/whatweb/wiki/How-to-develop-WhatWeb-plugins

This snippet shows an example of how matches are defined within a Ruby plugin for the whatweb tool. It includes the name, certainty, and the regular expression used for detection.

```ruby
matches [
    {:name=>"meta generator tag",
    :certainty=>100,
    :regexp=>/<meta name="generator"[^>]*content="SilverStripe/},

    {:name=>"layout, typography, form css files",
    :certainty=>75,
    :regexp=>/<link[^>]*stylesheet[^>]*layout.css[^>]*>[^<]*<link[^>]*stylesheet[^>]*typography.css[^>]*>[^<]*<link[^>]*stylesheet[^>]*form.css[^>]*>/}
    ]
```

--------------------------------

### Scan Websites from a File with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Example-Usage

To scan a large number of websites, you can provide a file containing a list of URLs. WhatWeb reads each URL from the specified file for scanning. The short option '-i' is equivalent to '--input-file'.

```bash
whatweb --input-file list.txt
```

```bash
whatweb -i list.txt
```

--------------------------------

### Custom Plugin Example using Regex

Source: https://github.com/urbanadventurer/whatweb/wiki/Discussion

Demonstrates how to use the --custom-plugin option with a regular expression to extract specific data from a web page. This allows WhatWeb to function as a grep-like tool for HTTP(S) content. Be mindful of shell escaping, especially for nested quotes.

```bash
./whatweb --custom-plugin "{:string=>/<title>([^<]+)<\/title>/i}" whatweb.net
```

--------------------------------

### Extract Injection Points for Security Testing

Source: https://github.com/urbanadventurer/whatweb/wiki/Discussion

This functionality aims to identify potential insertion points within web pages for security testing. It focuses on extracting GET and POST parameters, as well as identifying HTML elements that might be vulnerable. The extraction of GET parameters involves parsing the URL, while POST parameters can be handled by a dedicated plugin. HTML element extraction may lead to false positives without specific GET parameter values.

```regex
/base_uri[^'"].*\?([^=]+)=([^&]+)/
```

--------------------------------

### Install Character Set Detection Dependencies for WhatWeb

Source: https://github.com/urbanadventurer/whatweb/blob/master/README.md

Installs the `rchardet` gem required for character set detection, which is necessary for JSON and MongoDB logging. It also includes instructions for copying the charset plugin. For older Bundler versions, it specifies the `rchardet` group.

```bash
# Install dependencies
bundle install --with rchardet

# If you're using an older version of Bundler
GEMFILE_GROUPS="default rchardet" bundle install

# Copy the charset plugin
cp plugins-disabled/charset.rb my-plugins/
```

--------------------------------

### Basic XSS Scanning with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Advanced-Usage

Illustrates using WhatWeb for basic Cross-Site Scripting (XSS) detection by injecting a simple alert script into GET parameters. URL encoding is necessary for special characters. This functionality is restricted to GET requests.

```bash
$ ./whatweb -a 1 -p Vulnerable-To-XSS example.com/vuln.php?param1=<script>alert(1)</script> example.com/vuln.php?param2=<script>alert(1)</script>
```

```bash
$ echo "vuln.php?param1=<script>alert(1)</script>,vuln.php?param2=<script>alert(1)</script>" | tr ',' '\n' | ./whatweb -a 1 -p Vulnerable-To-XSS --url-pattern example.com/%insert% --input-file /dev/stdin
```

--------------------------------

### Example POST data for framework error detection

Source: https://github.com/urbanadventurer/whatweb/wiki/Discussion

This snippet demonstrates how to send a POST request with specific data to potentially trigger unique error responses from certain frameworks, aiding in their identification. It's a method for fingerprinting frameworks by observing their behavior with invalid requests.

```ruby
:url_post=>'/', :post_data=>'null=null'
```

--------------------------------

### Basic SQL Injection Scanning with WhatWeb

Source: https://github.com/urbanadventurer/whatweb/wiki/Advanced-Usage

Demonstrates using WhatWeb to perform basic SQL Injection scans by injecting SQL queries into GET request parameters. Special characters in the query must be URL encoded. This method is limited to GET requests.

```bash
$ ./whatweb -a 1 "example.com/vuln.php?param1=1'order%20by%2010--" "example.com/vuln.php?param1=1'order%20by%209--" "example.com/vuln.php?param1=1'order%20by%208--" "example.com/vuln.php?param1=1'order%20by%207--" "example.com/vuln.php?param1=1'order%20by%206--" "example.com/vuln.php?param1=1'order%20by%205--"
```

```bash
$ echo "vuln.php?param1=1'order%20by%2010--,vuln.php?param1=1'order%20by%209--,vuln.php?param1=1'order%20by%208--,vuln.php?param1=1'order%20by%207--,vuln.php?param1=1'order%20by%206--,vuln.php?param1=1'order%20by%205--" | tr ',' '\n' | ./whatweb -a 1 --url-pattern example.com/%insert% --input-file /dev/stdin
```

--------------------------------

### GHDB Match Example - Ruby

Source: https://github.com/urbanadventurer/whatweb/wiki/Writing-WhatWeb-Plugins

This Ruby code snippet provides an example of a GHDB match, often used for porting Google Hacking Database queries. It shows how to include Google search operators like 'intitle' and 'inurl' within the GHDB signature. GHDB queries are case-insensitive.

```ruby
# A typical GHDB match looks like :

# http://johnny.ihackstuff.com/ghdb?function=detail&id=1840
{ :ghdb=>'"Powered by Vsns Lemon" intitle:"Vsns Lemon"' }
```