### Setup Script Output Example

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

This is an example of the output you will see when running the setup script, indicating which files are being modified and what includes are being added.

```text
Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

INFO:      /etc/nginx/conf.d/* detected               => /etc/nginx/nginx.conf
inserting: include /etc/nginx/bots.d/blockbots.conf;  => /etc/nginx/sites-available/mydomain2.com.vhost
inserting: include /etc/nginx/bots.d/ddos.conf;       => /etc/nginx/sites-available/mydomain2.com.vhost
inserting: include /etc/nginx/bots.d/blockbots.conf;  => /etc/nginx/sites-available/mydomain1.com.vhost
inserting: include /etc/nginx/bots.d/ddos.conf;       => /etc/nginx/sites-available/mydomain1.com.vhost

Whitelisting ip:  x.x.x.x  => /etc/nginx/bots.d/whitelist-ips.conf
```

--------------------------------

### Actual installation output example

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

This output shows the successful download and installation of configuration files when the install script is run with the '-x' parameter.

```text
Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Creating directory: /etc/nginx/bots.d

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/conf.d/globalblacklist.conf            [TO]=>  /etc/nginx/conf.d/globalblacklist.conf...OK
Downloading [FROM]=>  [REPO]/conf.d/botblocker-nginx-settings.conf  [TO]=>  /etc/nginx/conf.d/botblocker-nginx-settings.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/bots.d/blockbots.conf              [TO]=>  /etc/nginx/bots.d/blockbots.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/ddos.conf                   [TO]=>  /etc/nginx/bots.d/ddos.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/whitelist-ips.conf          [TO]=>  /etc/nginx/bots.d/whitelist-ips.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/whitelist-domains.conf      [TO]=>  /etc/nginx/bots.d/whitelist-domains.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/blacklist-user-agents.conf  [TO]=>  /etc/nginx/bots.d/blacklist-user-agents.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/blacklist-ips.conf          [TO]=>  /etc/nginx/bots.d/blacklist-ips.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/bad-referrer-words.conf     [TO]=>  /etc/nginx/bots.d/bad-referrer-words.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/custom-bad-referrers.conf   [TO]=>  /etc/nginx/bots.d/custom-bad-referrers.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

```

--------------------------------

### Dry-run output example

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

This is an example of the output you will see when running the install script in dry-run mode, indicating files to be downloaded and directories to be created.

```text
** Dry Run ** | not updating files | run  as 'install-ngxblocker -x' to install files.

Creating directory: /etc/nginx/bots.d

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/conf.d/globalblacklist.conf            [TO]=>  /etc/nginx/conf.d/globalblacklist.conf
Downloading [FROM]=>  [REPO]/conf.d/botblocker-nginx-settings.conf  [TO]=>  /etc/nginx/conf.d/botblocker-nginx-settings.conf

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/bots.d/blockbots.conf              [TO]=>  /etc/nginx/bots.d/blockbots.conf
Downloading [FROM]=>  [REPO]/bots.d/ddos.conf                   [TO]=>  /etc/nginx/bots.d/ddos.conf
Downloading [FROM]=>  [REPO]/bots.d/whitelist-ips.conf          [TO]=>  /etc/nginx/bots.d/whitelist-ips.conf
Downloading [FROM]=>  [REPO]/bots.d/whitelist-domains.conf      [TO]=>  /etc/nginx/bots.d/whitelist-domains.conf
Downloading [FROM]=>  [REPO]/bots.d/blacklist-user-agents.conf  [TO]=>  /etc/nginx/bots.d/blacklist-user-agents.conf
Downloading [FROM]=>  [REPO]/bots.d/blacklist-ips.conf          [TO]=>  /etc/nginx/bots.d/blacklist-ips.conf
Downloading [FROM]=>  [REPO]/bots.d/bad-referrer-words.conf     [TO]=>  /etc/nginx/bots.d/bad-referrer-words.conf
Downloading [FROM]=>  [REPO]/bots.d/custom-bad-referrers.conf   [TO]=>  /etc/nginx/bots.d/custom-bad-referrers.conf

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/setup-ngxblocker      [TO]=>  /usr/local/sbin/setup-ngxblocker
Downloading [FROM]=>  [REPO]/update-ngxblocker     [TO]=>  /usr/local/sbin/update-ngxblocker
```

--------------------------------

### Download and Make Install Script Executable

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Installs the `install-ngxblocker` script to `/usr/local/sbin/` and makes it executable using `wget`.

```sh
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker
sudo chmod +x /usr/local/sbin/install-ngxblocker
```

--------------------------------

### Execute Install Script with Download Parameter

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Runs the install-ngxblocker script with the -x parameter to download all necessary files from the repository and perform the installation.

```shell
cd /usr/local/sbin/
sudo ./install-ngxblocker -x
```

--------------------------------

### Install via pkg

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Use this command to install the blocker via the FreeBSD package manager.

```sh
pkg install www/nginx-ultimate-bad-bot-blocker
```

--------------------------------

### Download and Make Install Script Executable (using curl)

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Installs the `install-ngxblocker` script to `/usr/local/sbin/` and makes it executable using `curl` as an alternative to `wget`.

```sh
curl -sL https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -o /usr/local/sbin/install-ngxblocker
```

--------------------------------

### Install via portmaster

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Alternatively, install the blocker using portmaster.

```sh
portmaster www/nginx-ultimate-bad-bot-blocker
```

--------------------------------

### Dry Run Setup Script

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Execute the setup script in dry-run mode to preview changes without modifying any files. This helps in understanding what the script will do, such as inserting include directives into vhost files.

```bash
cd /usr/local/sbin/
sudo ./setup-ngxblocker
```

--------------------------------

### Start Fail2Ban Client Verbose

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

This command starts the Fail2Ban client with verbose output, useful for debugging.

```bash
sudo fail2ban-client -vvv -x start
```

--------------------------------

### Set Executable Permissions for Scripts

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

These commands are crucial for making the downloaded setup and update scripts executable. Ensure these are run before proceeding with the setup process.

```bash
sudo chmod +x /usr/local/sbin/setup-ngxblocker
sudo chmod +x /usr/local/sbin/update-ngxblocker
```

--------------------------------

### Run Nginx Bot Blocker Setup Script

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Execute the setup script to activate the Bot Blocker on all Nginx sites. This script modifies vhost files and whitelists IPs.

```bash
cd /usr/local/sbin/
sudo ./setup-ngxblocker -x
```

--------------------------------

### Install Blocker to Non-Standard Nginx Folders

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Installs the Nginx blocker using custom paths for configuration and bot data directories.

```bash
sudo ./install-ngxblocker -x -c /usr/local/nginx/conf.d -b /usr/local/nginx/bots.d
```

--------------------------------

### Download and Make Install Script Executable

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Downloads the install-ngxblocker script using wget and makes it executable. If wget is not available, curl can be used as an alternative.

```shell
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker
sudo chmod +x /usr/local/sbin/install-ngxblocker
```

```shell
curl -sL https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -o /usr/local/sbin/install-ngxblocker
```

--------------------------------

### Setup Blocker in Non-Standard Nginx Folders

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Sets up the Nginx blocker with custom paths for configuration and bot data directories.

```bash
sudo ./setup-ngxblocker -x -c /usr/local/nginx/conf.d -b /usr/local/nginx/bots.d
```

--------------------------------

### Dry Run Output Example

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

This output demonstrates the dry-run results, showing detected Nginx configurations and planned insertions into vhost files. It also indicates IP whitelisting actions.

```bash
Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

** Dry Run ** | not updating files | run  as 'setup-ngxblocker -x' to setup files. 

INFO:      /etc/nginx/conf.d/* detected               => /etc/nginx/nginx.conf
inserting: include /etc/nginx/bots.d/blockbots.conf;  => /etc/nginx/sites-available/mydomain2.com.vhost
inserting: include /etc/nginx/bots.d/ddos.conf;       => /etc/nginx/sites-available/mydomain2.com.vhost
inserting: include /etc/nginx/bots.d/blockbots.conf;  => /etc/nginx/sites-available/mydomain1.com.vhost
inserting: include /etc/nginx/bots.d/ddos.conf;       => /etc/nginx/sites-available/mydomain1.com.vhost

Whitelisting ip:  x.x.x.x  => /etc/nginx/bots.d/whitelist-ips.conf
```

--------------------------------

### Dry Run of Install Script

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Executes the install-ngxblocker script in dry-run mode to preview the changes and files that will be downloaded without making any actual modifications.

```shell
cd /usr/local/sbin
sudo ./install-ngxblocker
```

--------------------------------

### Fail2Ban Jail Configuration Includes

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

These examples show how to include common path configurations in your Fail2Ban jail.conf or jail.local. It is recommended to set 'enabled = false' to manually enable jails later.

```ini
[INCLUDES]
before = paths-common.conf
enabled = false
```

```ini
[INCLUDES]
before = paths-debian.conf
enabled = false
```

--------------------------------

### Block GoogleBot Example

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Add GoogleBot to the blacklist-user-agents.conf file to override its default whitelisted status and block it.

```nginx
GoogleBot
```

--------------------------------

### Test Referrer Blocking (Second Example)

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Another test for referrer blocking, simulating a different malicious referrer. Should result in an empty reply.

```bash
curl -I http://yourdomain.com -e http://zx6.ru
```

--------------------------------

### Block Specific User Agents

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Add user agents to the 'blacklist-user-agents.conf' file to block them. This example shows how to block GoogleBot.

```nginx
/etc/nginx/bots.d/blacklist-user-agents.conf
```

--------------------------------

### Disallow Specific Bots in Robots.txt

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/robots.txt/README.md

This example demonstrates how to explicitly disallow specific user agents, such as '360Spider' and '404checker', from crawling any part of the website by adding a Disallow: / directive for each.

```robots.txt
User-agent: 360Spider
Disallow:/
User-agent: 404checker
Disallow:/
```

--------------------------------

### Fail2Ban Nginx Repeat Offender Jail Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

Example configuration for a custom Fail2Ban jail to block repeat offenders hitting Nginx.

```ini
[nginxrepeatoffender]
enabled = true
logpath = %(nginx_access_log)s
filter = nginxrepeatoffender
banaction = nginxrepeatoffender
bantime  = 86400   ; 1 day
findtime = 604800   ; 1 week
maxretry = 20
```

--------------------------------

### Create Bots Directory and Download Bot Lists

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Creates the necessary directory for bot-related configuration files and downloads essential bot list configurations using wget.

```bash
sudo mkdir /etc/nginx/bots.d 
```

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blockbots.conf -O /etc/nginx/bots.d/blockbots.conf
```

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/ddos.conf -O /etc/nginx/bots.d/ddos.conf
```

--------------------------------

### Download Domain Whitelist Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the domain whitelist configuration file. This file is used to specify domain names that should not be blocked by the bot blocker.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-domains.conf -O /etc/nginx/bots.d/whitelist-domains.conf
```

--------------------------------

### Download Global Blacklist Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the global blacklist configuration file using wget. This file contains a comprehensive list of known bad bots and IP addresses.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -O /etc/nginx/conf.d/globalblacklist.conf
```

--------------------------------

### Test Nginx Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Run this command to check your Nginx configuration syntax before reloading. Ensure there are no errors before proceeding to reload the service.

```bash
sudo nginx -t
```

--------------------------------

### Download Global Blacklist Configuration with Curl

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the global blacklist configuration file using curl as an alternative to wget. This is useful if wget is not available on your system.

```bash
curl -sL https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -o /etc/nginx/conf.d/globalblacklist.conf
```

--------------------------------

### Download IP Whitelist Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the IP whitelist configuration file. This file is used to specify IP addresses that should not be blocked by the bot blocker.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/whitelist-ips.conf -O /etc/nginx/bots.d/whitelist-ips.conf
```

--------------------------------

### Customizable Bot Blocker Configuration Files

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

List of configuration files that can be customized to tailor the bot blocker's behavior. These files are not overwritten during automatic updates.

```text
/etc/nginx/bots.d/whitelist-ips.conf
/etc/nginx/bots.d/whitelist-domains.conf
/etc/nginx/bots.d/blacklist-user-agents.conf
/etc/nginx/bots.d/blacklist-ips.conf
/etc/nginx/bots.d/bad-referrer-words.conf
/etc/nginx/bots.d/custom-bad-referrers.conf
```

--------------------------------

### Download Nginx Settings Configuration File

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the botblocker-nginx-settings.conf file, which includes essential rate limiting and hash bucket settings for Nginx. This file should be placed in the /etc/nginx/conf.d folder.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/botblocker-nginx-settings.conf -O /etc/nginx/conf.d/botblocker-nginx-settings.conf
```

--------------------------------

### Test Xenu Link Sleuth Access

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Simulates a request from Xenu Link Sleuth. This should result in an empty reply from the server.

```bash
curl -A "Xenu Link Sleuth/1.3.8" http://yourdomain.com
```

--------------------------------

### Download Bot Blocker Files

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

This output shows the files being downloaded from the repository to the specified Nginx directories. It confirms successful download of configuration and script files.

```bash
Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Creating directory: /etc/nginx/bots.d

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/conf.d/globalblacklist.conf            [TO]=>  /etc/nginx/conf.d/globalblacklist.conf...OK
Downloading [FROM]=>  [REPO]/conf.d/botblocker-nginx-settings.conf  [TO]=>  /etc/nginx/conf.d/botblocker-nginx-settings.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/bots.d/blockbots.conf              [TO]=>  /etc/nginx/bots.d/blockbots.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/ddos.conf                   [TO]=>  /etc/nginx/bots.d/ddos.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/whitelist-ips.conf          [TO]=>  /etc/nginx/bots.d/whitelist-ips.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/whitelist-domains.conf      [TO]=>  /etc/nginx/bots.d/whitelist-domains.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/blacklist-user-agents.conf  [TO]=>  /etc/nginx/bots.d/blacklist-user-agents.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/blacklist-ips.conf          [TO]=>  /etc/nginx/bots.d/blacklist-ips.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/bad-referrer-words.conf     [TO]=>  /etc/nginx/bots.d/bad-referrer-words.conf...OK
Downloading [FROM]=>  [REPO]/bots.d/custom-bad-referrers.conf   [TO]=>  /etc/nginx/bots.d/custom-bad-referrers.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=>  [REPO]/setup-ngxblocker      [TO]=>  /usr/local/sbin/setup-ngxblocker...OK
Downloading [FROM]=>  [REPO]/update-ngxblocker     [TO]=>  /usr/local/sbin/update-ngxblocker...OK
```

--------------------------------

### DNS Lookup for Punycode Domain

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Demonstrates how to perform a DNS lookup for a Punycode-encoded domain name to verify its correct format and resolution.

```bash
nslookup xn--lifehacer-1rb.com
```

--------------------------------

### Test Bot Blocker with Referer (Blocked Referers)

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

These commands test if specific referers are blocked. Use the '-I' option for a HEAD request. An 'Empty reply from server' response confirms the blocking is active.

```bash
curl -I http://yourdomain.com -e http://100dollars-seo.com
```

```bash
curl -I http://yourdomain.com -e http://zx6.ru
```

--------------------------------

### Download Custom Bad Referrers Blacklist

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the custom bad referrers blacklist configuration file. This enables blocking based on specific referrer domains.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/custom-bad-referrers.conf -O /etc/nginx/bots.d/custom-bad-referrers.conf
```

--------------------------------

### Ensure Nginx Configuration Include

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

This directive in your main nginx.conf file ensures that all configuration files within the /etc/nginx/conf.d directory are loaded. It is crucial for the bot blocker to function.

```nginx
include /etc/nginx/conf.d/*;
```

--------------------------------

### Test Xenu Link Sleuth Access

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Simulates a request from Xenu Link Sleuth to test the blocker's response. Expected responses are connection errors.

```bash
curl -A "Xenu Link Sleuth/1.3.8" -I http://yourdomain.com
```

--------------------------------

### Include Custom Nginx Configurations on FreeBSD

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

This configuration snippet shows how to include custom Nginx configuration files from .vhost files and the conf.d directory. Ensure these paths are correctly set up in your Nginx environment.

```nginx
include /etc/nginx/conf.d/*;

```

--------------------------------

### Specify VHost File Extension

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Use the -e parameter to specify a custom extension for your vhost files if they do not end in .vhost.

```bash
sudo ./setup-ngxblocker -x -e conf
```

--------------------------------

### Include Bot Blocker Configuration Files

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Include the main configuration files for the bot blocker. These are typically placed in the Nginx configuration.

```nginx
include /etc/nginx/bots.d/ddos.conf;
include /etc/nginx/bots.d/blockbots.conf;
```

--------------------------------

### Test Bot Blocker with Curl (Good Bot)

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Use this command to test if a known good bot, like Googlebot, is correctly allowed access. A 200 OK response indicates successful configuration.

```bash
curl -A "googlebot" http://yourdomain.com
```

--------------------------------

### Download Custom User-Agent Blacklist

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the custom user-agent blacklist configuration file. This allows you to block specific user agents not covered by the default lists.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-user-agents.conf -O /etc/nginx/bots.d/blacklist-user-agents.conf
```

--------------------------------

### Download Custom IP Blacklist

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Downloads the custom IP blacklist configuration file. This allows you to block specific IP addresses or IP ranges.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/blacklist-ips.conf -O /etc/nginx/bots.d/blacklist-ips.conf
```

--------------------------------

### Update Global Blacklist Configuration

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Manually update the global blacklist by downloading the latest version. This ensures your blocker has the most up-to-date list of malicious bots and referers. Remember to test and reload Nginx after updating.

```bash
cd /etc/nginx/conf.d
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/conf.d/globalblacklist.conf -O /etc/nginx/conf.d/globalblacklist.conf
```

--------------------------------

### Create Fail2Ban Nginx Repeater Offender File

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

This command creates the necessary empty file for Fail2Ban's nginx.repeatoffender configuration. Ensure the file has execute permissions.

```bash
sudo touch /etc/fail2ban/nginx.repeatoffender

sudo chmod +x /etc/fail2ban/nginx.repeatoffender
```

--------------------------------

### Test Googlebot Access

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Simulates a Googlebot request to check if your domain responds with a 200 OK status.

```bash
curl -A "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" http://yourdomain.com
```

--------------------------------

### Stop and Debug Fail2Ban Service

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

Use these commands in sequence to stop the Fail2Ban service and then restart it in verbose mode to identify and debug loading errors. This helps pinpoint configuration issues.

```bash
sudo service fail2ban stop

sudo fail2ban-client -vvv -x stop

sudo fail2ban-client -vvv -x start
```

--------------------------------

### Monitor Daily Referers on Nginx

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

A cron job to monitor the top 1000 referers for a website daily, excluding common search engines and the website's own domain. It emails the results to a specified address.

```bash
00 08 * * * tail -10000 /var/log/nginx/mydomain-access.log | awk '$11 !~ /google|bing|yahoo|yandex|mywebsite.com/' | awk '{print $11}' | tr -d '"' | sort | uniq -c | sort -rn | head -1000 | mail -s "Top 1000 Referers for Mydomain.com" me@mydomain.com
```

--------------------------------

### Nginx Rate Limiting and Hash Settings

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

These are the core Nginx configuration directives for rate limiting and hash bucket sizes. They are typically included via the botblocker-nginx-settings.conf file.

```nginx
server_names_hash_bucket_size 64;
server_names_hash_max_size 4096;
limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;
limit_conn_zone $binary_remote_addr zone=addr:50m;
```

--------------------------------

### Download Custom Bad Referrer Words File

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

Copies the custom bad referrer words configuration file to the /etc/nginx/bots.d folder. This file can be edited to customize scanning for bad words.

```bash
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/bots.d/bad-referrer-words.conf -O /etc/nginx/bots.d/bad-referrer-words.conf
```

--------------------------------

### Restart Nginx Service

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Alternatively, restart the Nginx service to apply configuration changes.

```bash
sudo service nginx restart
```

--------------------------------

### Schedule Daily Bot Blocker Updates

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Set up a cron job to automatically update the Nginx bot blocker daily at 10 PM. Replace 'yourname@youremail.com' with your actual email address to receive update reports.

```bash
00 22 * * * sudo /usr/local/sbin/update-ngxblocker -e yourname@youremail.com
```

--------------------------------

### Restart Fail2Ban Service

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

After making configuration changes or resolving errors, restart the Fail2Ban service to apply the new settings.

```bash
sudo service fail2ban restart
```

--------------------------------

### Test Bingbot Access

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/AUTO-CONFIGURATION.md

Simulates a Bingbot request to verify your domain's response.

```bash
curl -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" http://yourdomain.com
```

--------------------------------

### Stop Fail2Ban Service

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/_fail2ban_addon/README.md

Use this command to stop the Fail2Ban service during troubleshooting or configuration changes.

```bash
sudo service fail2ban stop
```

--------------------------------

### Monitor Daily User Agents with Cron

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

This cron job monitors Nginx access logs to identify and email the top 1000 User-Agents visiting your site daily. It helps in spotting suspicious user agent strings.

```bash
00 08 * * * tail -50000 /var/log/nginx/mydomain-access.log | awk '{print $12}' | tr -d '"' | sort | uniq -c | sort -rn | head -1000 | mail -s "Top 1000 Agents for Mydomain.com" me@mydomain.com
```

--------------------------------

### Include Bot Blocker Configuration Files

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/MANUAL-CONFIGURATION.md

These include directives must be placed within a server block in your Nginx virtual host configuration to avoid EMERG errors. They load the main bot blocking and DDoS protection rules.

```nginx
include /etc/nginx/bots.d/blockbots.conf;
```

```nginx
include /etc/nginx/bots.d/ddos.conf;
```

--------------------------------

### Schedule Updates with Mailgun Notification

Source: https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/blob/master/README.md

Configure a cron job to update the Nginx bot blocker daily at 10 PM, using Mailgun for email notifications.

```bash
00 22 * * * sudo /usr/local/sbin/update-ngxblocker -g yourname@yourdomain.com -d yourdomain.com -a mailgun api key -f from@yourdomain.com
```