### Setup HFS+ Filesystem for Testing on Linux

Source: https://github.com/rockdaboot/wget2/wiki/HFS--testing-on-Linux

Installs necessary tools, creates an HFS+ filesystem image, mounts it, and sets ownership for testing filename decomposition.

```bash
$ sudo apt-get install hfsprogs

$ dd if=/dev/zero of=hfsimage bs=512 count=500k
$ /sbin/mkfs.hfsplus hfsimage
$ mkdir mnt
$ sudo mount -o loop -t hfsplus hfsimage mnt
$ sudo chown tim:users mnt

$ cd /mnt
$ git clone git://git.sv.gnu.org/wget.git
$ cd wget
$ ./bootstrap
$ ./configure
$ make 
$ make check
```

--------------------------------

### Install wget2 and libwget

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Installs wget2 and its associated library. Use 'sudo' for system-wide installation or 'su -c "make install"' if root privileges are required.

```bash
sudo make install (or su -c "make install")
```

--------------------------------

### Build wget2 on Haiku

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Specific instructions for building wget2 on the Haiku operating system. This includes setting the architecture, configuring the installation prefix, and managing the installation directory.

```bash
setarch x86
./configure --prefix=/boot/home/config/non-packaged
rm /boot/home/config/non-packaged/wget2 && mv /boot/home/config/non-packaged/wget2_noinstall /boot/home/config/non-packaged/wget2
```

--------------------------------

### Sample wget2 Build Configuration

Source: https://github.com/rockdaboot/wget2/blob/master/docs/development/ContributingGuide.md

Example build commands for wget2, including bootstrap, configure with specific flags, and make check.

```bash
$ ./bootstrap
$ ./configure --enable-manywarnings --disable-silent-rules --enable-assert
$ make check
```

--------------------------------

### Build wget2 from Tarball

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Standard procedure for downloading, extracting, configuring, and installing wget2 from a tarball. Ensure you have build tools like make and a C compiler installed.

```bash
wget https://gnuwget.gitlab.io/wget2/wget2-latest.tar.gz
tar xf wget2-latest.tar.gz
cd wget2-*
./configure
make
make check
sudo make install
```

--------------------------------

### Example Wget2 Command for Bug Reproduction

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

When reporting a bug, try to reproduce it with a simplified command. This example shows a complex command that might be simplified to isolate the issue.

```bash
wget2 -rl0 -kKE -t5 --no-proxy https://example.com -o /tmp/log
```

--------------------------------

### Build wget2 with Make

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

After preparing sources, this command configures and builds wget2. Ensure you have the necessary development dependencies installed.

```bash
./configure
make
```

--------------------------------

### Get wget2 Help

Source: https://github.com/rockdaboot/wget2/wiki/Contributing-guide-for-GSoC-students

Display the help message for the wget2 command-line tool. This provides information on available options and usage.

```bash
wget2 --help
```

--------------------------------

### Configure Wget2 with GnuTLS support

Source: https://github.com/rockdaboot/wget2/blob/master/docs/faq.md

Check the configure summary to ensure GnuTLS support is enabled. Install gnutls-dev on Debian/Ubuntu if TLS support is missing.

```bash
apt-get install gnutls-dev
```

```text
SSL/TLS support:    yes
```

--------------------------------

### Download LibreOffice Writer compatible files with wget2

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

This example demonstrates downloading files compatible with LibreOffice Writer by dynamically generating the MIME type list from a desktop file. It uses sed to extract and format the MimeType entry.

```bash
wget2 -r https://<site>/<document> --filter-mime-type=$(sed -r '/^MimeType=/!d;s/^MimeType=//;s/;/,/g' /usr/share/applications/libreoffice-writer.desktop)
```

--------------------------------

### Add Personal Fork as Remote

Source: https://github.com/rockdaboot/wget2/wiki/Git:-Syncing-forks

Add your own GitHub fork as a remote repository, typically named 'darnir' in this example.

```bash
git remote add darnir https://github.com/darnir/wget2
```

--------------------------------

### Configure and Build wget2 with Custom GnuTLS

Source: https://github.com/rockdaboot/wget2/wiki/Building-with-alternate-GnuTLS-version

Use these environment variables to point the build process to your custom GnuTLS installation. Ensure absolute paths are used for `GNUTLS_CFLAGS` and `GNUTLS_LIBS`.

```bash
$ GNUTLS_CFLAGS="-I/home/tim/src/gnutls/lib/includes" GNUTLS_LIBS=-L/home/tim/src/gnutls/lib ./configure && make
```

--------------------------------

### Add Upstream Repository

Source: https://github.com/rockdaboot/wget2/blob/master/docs/development/ContributingGuide.md

Add the official wget2 repository as a remote named 'upstream'. This is a one-time setup.

```bash
$ git remote add upstream https://gitlab.com/gnuwget/wget2.git
```

--------------------------------

### Broken Progress Bar Example

Source: https://github.com/rockdaboot/wget2/wiki/Progress-Bar

Demonstrates the incorrect percentage calculation and output overwriting issue in the Wget2 progress bar. This output is generated when downloading multiple files.

```text
233% [=====================================================================>]
     [Files: 2  Bytes: 0  Redirects: 1  Todo: 0                             ]
```

--------------------------------

### Enable Debugging Output in Wget2

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Start Wget2 with the -d option to enable debug output. This is crucial for tracing bugs and should be included in bug reports.

```bash
wget2 -d
```

--------------------------------

### Download all except images using wget2

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use the --filter-mime-type option to specify MIME types to download. Wildcards are supported, and a leading '!' excludes types. This example downloads everything except files of type 'image/*'.

```bash
wget2 -r https://<site>/<document> --filter-mime-type=*,"!image/*"
```

--------------------------------

### View Configure Help Options

Source: https://github.com/rockdaboot/wget2/wiki/Building-with-alternate-GnuTLS-version

Run `./configure --help` to see all available configuration options, including those related to libraries supported by pkg-config.

```bash
$ ./configure --help
```

--------------------------------

### Download and Prepare Sources with Git

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Clones the wget2 repository and prepares the build environment using the bootstrap script. If the script fails, try executing it with bash.

```bash
git clone https://gitlab.com/gnuwget/wget2.git
cd wget2
./bootstrap
# on shell failure try 'bash ./bootstrap'
```

--------------------------------

### Configure and Build with libFuzzer

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Set up the environment variables and build the project with libFuzzer support. Ensure correct CFLAGS and LIB_FUZZING_ENGINE are set.

```bash
export CC=clang
#export CFLAGS "-O1 -g -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined,integer,nullability,bool,alignment,null,enum,address,leak,nonnull-attribute -fsanitize=address -fsanitize-address-use-after-scope -fsanitize-coverage=trace-pc-guard,trace-cmp"

export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fno-sanitize-recover=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fsanitize=fuzzer-no-link"

export LIB_FUZZING_ENGINE="-lFuzzer -lstdc++"
./configure --enable-fuzzing --disable-doc --disable-manywarnings
make clean
make -j$(nproc)
cd fuzz
```

--------------------------------

### Run Fuzzer with AFL

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Execute a fuzzer using AFL. This command starts the AFL fuzzer for the specified target.

```bash
./run-afl.sh libwget_xml_parse_buffer_fuzzer
```

--------------------------------

### Configure and Build with AFL

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Set up the environment and build the project for use with AFL. This involves setting the CC environment variable and building the all target.

```bash
$CC=afl-clang-fast ./configure --disable-doc
$ make -j$(nproc) clean all
$ cd fuzz
```

--------------------------------

### Build wget2

Source: https://github.com/rockdaboot/wget2/wiki/Contributing-guide-for-GSoC-students

Build the wget2 project after cloning. This involves bootstrapping, configuring with specific options, and making the project.

```bash
$ ./bootstrap
$ ./configure --enable-fsanitize-msan --enable-fsanitize-ubsan --enable-manywarnings
$ make check-coverage
```

--------------------------------

### Create wget_options_fuzzer.dict

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Generate a dictionary file for the wget_options_fuzzer by extracting command-line options from the wget2 --help output. This can improve fuzzer effectiveness.

```bash
for i in `../src/wget2 --help|tr ' ' '\n'|grep ^--|cut -c 3-|sort`;do echo "\"$i\""; done >wget_options_fuzzer.dict
```

--------------------------------

### Build Static wget2 for Linux/amd64 on GNU/Linux

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Builds a static wget2 executable for Linux/amd64 on a GNU/Linux system using Docker and MUSL libc. This results in a lightweight, self-contained binary.

```bash
(cd contrib; docker build -t wget2/static/amd64 -f Dockerfile.musl.static.amd64 .)
docker run --rm -v $PWD:/tmp wget2/static/amd64 cp /usr/local/wget2/src/wget2 /tmp
sudo chown $USER: wget2
# now you have `wget2.exe` with debug symbols in your current directory
# optional: remove debug symbols with
strip wget2
# optional: pack executable
upx wget2
```

--------------------------------

### wget2: Set Directory Prefix

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use `-P prefix` or `--directory-prefix=prefix` to specify the top-level directory where all downloaded files and subdirectories will be saved. If the directory does not exist, it will be created.

```bash
wget2 -P /path/to/save/files https://example.com/index.html
```

--------------------------------

### Run Wget2 test suite with Valgrind

Source: https://github.com/rockdaboot/wget2/blob/master/docs/faq.md

Execute the complete test suite using Valgrind memcheck for memory error detection. This requires enabling Valgrind tests during configuration.

```bash
make check-valgrind
```

```bash
./configure --enable-valgrind-tests
make check
```

--------------------------------

### Wget2 Clearing and Setting Exclude Directories

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Demonstrates how to clear existing exclude-directories settings and set new ones using the `--no-` prefix.

```bash
wget2 --no-exclude-directories -X /priv,/trash
```

--------------------------------

### Test wget2 Functionality

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Runs the test suite to verify the functionality of the built wget2. This step is crucial for ensuring the build integrity.

```bash
make check
```

--------------------------------

### wget2 Download (Before Patch)

Source: https://github.com/rockdaboot/wget2/wiki/Speed-and-CPU-measurements

This command performs a simple download using wget2 with a single thread, before a specific patch was applied. It is used for performance benchmarking.

```bash
wget2 http://oms-4/file.gz
```

--------------------------------

### Wget2 Options After Arguments

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Explains how to use the `--` delimiter to specify options that might be mistaken for URLs or arguments.

```bash
wget2 -o log -- -x
```

--------------------------------

### Download HTML page and its requisites with Wget2

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use -p to download all files necessary to display an HTML page properly, including images and stylesheets. This is useful for offline viewing.

```bash
wget2 -r -l 2 https://<site>/1.html
```

```bash
wget2 -r -l 2 -p https://<site>/1.html
```

```bash
wget2 -r -l 1 -p https://<site>/1.html
```

```bash
wget2 -p https://<site>/1.html
```

```bash
wget2 -E -H -k -K -p https://<site>/<document>
```

--------------------------------

### Generate Certificates

Source: https://github.com/rockdaboot/wget2/blob/master/tests/certs/ocsp/README.md

Execute the generate_certs.sh script to create necessary certificates.

```bash
$ bash generate_certs.sh
```

--------------------------------

### Build Static wget2.exe for Windows on GNU/Linux

Source: https://github.com/rockdaboot/wget2/blob/master/README.md

Cross-compiles a static wget2 executable for Windows on a GNU/Linux system using Docker. This creates a self-contained executable without external dependencies.

```bash
(cd contrib; docker build -t wget2/win32/static -f Dockerfile.win32.static .)
docker run --rm -v $PWD:/tmp wget2/win32/static cp /usr/local/wget2/src/wget2.exe /tmp
sudo chown $USER: wget2.exe
# now you have `wget2.exe` with debug symbols in your current directory
# optional: remove debug symbols with
#   strip wget2.exe   # (needs mingw64 version of `strip`)
# optional: pack executable
upx wget2.exe
```

--------------------------------

### wget2: Use Protocol Directories

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use `--protocol-directories` to include the protocol name (e.g., 'https') as a directory component in the local file path. This helps differentiate downloads from the same host using different protocols.

```bash
wget2 -r --protocol-directories https://example.com
```

--------------------------------

### Wget2 Mixed Option Styles

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Demonstrates mixing long and short option styles, including options with arguments, in a single command.

```bash
wget2 -r --tries=10 https://example.com/ -o log
```

--------------------------------

### Tag Repository for Release

Source: https://github.com/rockdaboot/wget2/wiki/Release-todo-list

Use these Git commands to commit changes, create a signed tag for the release, and push the tag to the repository.

```bash
git commit -a -m "Release Vx.y.z"
git tag -s -m "Release Vx.y.z" wget2-x.y.z
git push --follow-tags
```

--------------------------------

### Wget2 Combined Short Options

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Illustrates combining multiple short options that do not require arguments.

```bash
wget2 -drc <URL>
```

--------------------------------

### wget2: Force Directory Creation

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use `-x` or `--force-directories` to create a directory hierarchy even if one would not normally be created. This ensures files are saved in a structure reflecting their remote path.

```bash
wget2 -x https://example.com/robots.txt
```

--------------------------------

### Generate Wget2 test coverage report

Source: https://github.com/rockdaboot/wget2/blob/master/docs/faq.md

Create and view a test code coverage report using 'make check-coverage'. This process is compatible with GCC but not with Clang.

```bash
make check-coverage
xdg-open lcov/index.html
```

--------------------------------

### wget Download

Source: https://github.com/rockdaboot/wget2/wiki/Speed-and-CPU-measurements

This command uses wget to download a file quietly and save it to a local file. It is included for performance benchmarking against other tools.

```bash
wget -q http://oms-4/file.gz
```

--------------------------------

### wget2: Do Not Create Directories

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use `-nd` or `--no-directories` to save all files to the current directory without creating a hierarchy. Files with duplicate names will receive extensions like .1, .2, etc.

```bash
wget2 -nd https://example.com/file.html
```

--------------------------------

### Wget2 Mirror Website

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use this command to mirror a website. It enables recursion, time-stamping, and sets infinite recursion depth, equivalent to -r -N -l inf.

```bash
wget2 -m https://example.com/
```

--------------------------------

### Load Cookies from File

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use `--load-cookies` to load cookies from a Netscape-formatted file before the first HTTP request. This is useful for accessing sites that require authentication via cookies.

```bash
wget2 --load-cookies cookies.txt https://example.com/
```

--------------------------------

### Continue Partially Downloaded File with Wget2

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Use the `-c` option to resume interrupted downloads. Wget2 will check the local file size and request the remaining bytes from the server.

```bash
wget2 -c https://example.com/tarball.gz
```

--------------------------------

### Clone Official Repository

Source: https://github.com/rockdaboot/wget2/wiki/Git:-Syncing-forks

Use this command to clone the official wget2 repository from its Savannah URL.

```bash
git clone git://git.savannah.gnu.org/wget/wget2.git
```

--------------------------------

### Generate Certificates and CRLs

Source: https://github.com/rockdaboot/wget2/blob/master/tests/certs/README.md

Execute the `generate.sh` script to automatically create CA and server certificates, private keys, and a CRL. This is the primary method for setting up the required cryptographic files.

```bash
$ bash generate.sh
```

--------------------------------

### Download OSS-Fuzz Corpora

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Use this command to download corpora from OSS-Fuzz for a specific fuzzer. After downloading, stage and commit the updated corpora to your git repository.

```bash
./get_ossfuzz_corpora libwget_xml_parse_buffer_fuzzer
git add libwget_xml_parse_buffer_fuzzer.in/*
git commit -a -m "Update OSS-Fuzz corpora"
(create a branch and push if something changed)
(create a MR)
```

--------------------------------

### Run single Wget2 tests with Valgrind

Source: https://github.com/rockdaboot/wget2/blob/master/docs/faq.md

Debug individual tests with Valgrind by setting the VALGRIND_TESTS environment variable. This allows focused memory checking on specific test cases.

```bash
cd tests
VALGRIND_TESTS=1 ./test-k
```

```bash
TESTS_ENVIRONMENT="VALGRIND_TESTS=1" make check -C tests TESTS="test-k"
```

--------------------------------

### Wget2 Negating Boolean Options

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

Shows how to negate affirmative boolean options by prepending `--no-`.

```bash
wget2 --no-timestamping
```

--------------------------------

### Clone Collaborator Repository and Add Upstream

Source: https://github.com/rockdaboot/wget2/wiki/Git:-Syncing-forks

Clone a collaborator's repository and add the official repository as an 'upstream' remote for synchronization.

```bash
git clone https://github.com/darnir/wget2
```

```bash
git remote add upstream git://git.savannah.gnu.org/wget/wget2.git
```

--------------------------------

### Verify Linked GnuTLS Library

Source: https://github.com/rockdaboot/wget2/wiki/Building-with-alternate-GnuTLS-version

After building, use `ldd` to confirm that `wget2_noinstall` is linked against the custom GnuTLS library. This command helps in debugging linking issues.

```bash
$ ldd src/wget2_noinstall|grep gnutls
        libgnutls.so.30 => /usr/oms/src/gnutls/lib/.libs/libgnutls.so.30 (0x00007f627cc35000)
```

--------------------------------

### Add ccache to PATH

Source: https://github.com/rockdaboot/wget2/wiki/Developer-hints:-Increasing-speed-of-GNU-toolchain

Ensure ccache is available in your system's PATH for it to be used automatically by the build system. This is typically done by exporting the path to the ccache directory.

```bash
export PATH="/usr/lib/ccache:$PATH"
```

--------------------------------

### Run libwget_xml_parse_buffer_fuzzer with libFuzzer

Source: https://github.com/rockdaboot/wget2/blob/master/fuzz/README.md

Execute a specific libFuzzer target. Ensure UBSAN_OPTIONS and ASAN_SYMBOLIZER_PATH are set for detailed crash reporting.

```bash
export UBSAN_OPTIONS=print_stacktrace=1:report_error_type=1
export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer
./run-clang.sh libwget_xml_parse_buffer_fuzzer
```

--------------------------------

### rsync Download

Source: https://github.com/rockdaboot/wget2/wiki/Speed-and-CPU-measurements

This command uses rsync to download a file from a remote server. It is included for comparison of CPU consumption during the transfer.

```bash
rsync oms-4:/var/www/html/file.gz .
```

--------------------------------

### Wget2 Link Conversion for Local Viewing

Source: https://github.com/rockdaboot/wget2/blob/master/docs/wget2.md

This option converts links in downloaded documents to make them suitable for local viewing. It adjusts links to downloaded files to be relative and links to non-downloaded files to be absolute.

```bash
wget2 -k https://example.com/some/document.html
```

--------------------------------

### Build dash with lineno support

Source: https://github.com/rockdaboot/wget2/wiki/Developer-hints:-Increasing-speed-of-GNU-toolchain

If Debian's dash lacks lineno support, which can cause configure scripts to fall back to bash, recompile dash from source. This involves modifying the debian/rules file to remove the --disable-lineno flag.

```bash
apt-get source dash
cd dash-0.5.8/
vi debian/rules (removing --disable-lineno)
LC_ALL=C debuild -b -uc -us
sudo dpkg -i ../dash_0.5.8-2.3_amd64.deb
```

--------------------------------

### Clone wget2 Fork

Source: https://github.com/rockdaboot/wget2/wiki/Contributing-guide-for-GSoC-students

Clone your personal fork of the wget2 repository from GitHub to your local machine.

```bash
$ git clone https://github.com/<your username>/wget2.git
```