n2disk

https://www.ntop.org/guides/n2disk/
Admin
n2disk is a high-speed network traffic recorder application designed to capture full-sized network...
Tokens:53,059
Snippets:401
Trust Score:10
Update:1 week ago
Show doc for...
Tokens:
Raw
### List n2disk config and start files

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

Example listing showing the .start file required for automatic startup.

```plaintext
# ls -lha /etc/n2disk
-rw-r--r--   1 root root  211 mar 15 17:54 n2disk-eth0.conf
-rw-r--r--   1 root root    0 mar 17 15:44 n2disk-eth0.start
```

--------------------------------

### Start n2disk with init script (start)

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Demonstrates the 'start' option: starts only interfaces with a .start file. Without it, an error is shown.

```bash
# ls -lha /etc/n2disk
   -rw-r--r--   1 root root  211 mar 15 17:54 n2disk-eth0.conf
   -rw-r--r--   1 root root    0 mar 17 15:44 n2disk-eth0.start
   -rw-r--r--   1 root root  195 Jan  8 17:17 n2disk-eth4.conf
   # /etc/init.d/n2disk start eth0
   Starting n2disk eth0
   # /etc/init.d/n2disk start eth4
   n2disk eth4 not started: missing /etc/n2disk/n2disk-eth4.start
```

--------------------------------

### Start n2disk instance (requires .start)

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

Starting an instance with a .start file succeeds; starting one without it fails with the error message shown.

```plaintext
# ls -lha /etc/n2disk
-rw-r--r--   1 root root  211 mar 15 17:54 n2disk-eth0.conf
-rw-r--r--   1 root root    0 mar 17 15:44 n2disk-eth0.start
-rw-r--r--   1 root root  195 Jan  8 17:17 n2disk-eth4.conf
# /etc/init.d/n2disk start eth0
Starting n2disk eth0
# /etc/init.d/n2disk start eth4
n2disk eth4 not started: missing /etc/n2disk/n2disk-eth4.start
```

--------------------------------

### Start Napatech Driver with ntstart.sh

Source: https://www.ntop.org/guides/n2disk/tuning/napatech.html

Starts the Napatech driver after configuration changes have been completed.

```bash
/opt/napatech3/bin/ntstart.sh

```

--------------------------------

### Force start n2disk (force-start)

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

force-start starts the daemon even when no .start file exists.

```bash
# /etc/init.d/n2disk force-start eth4
   Starting n2disk eth4
```

--------------------------------

### n2disk command-line configuration for multi-directory capture

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt

Use this n2disk command-line example to configure a multi-directory capture setup with eight dump and timeline directories, a disk limit of 80%, a maximum file length of 1024 MB, a buffer length of 8192, indexing enabled, and CPU affinities for writers, readers, and indexers.

```bash
--interface=mlx:mlx_5@[0-7]
--dump-directory=/storage1/n2disk/pcap
--dump-directory=/storage2/n2disk/pcap
--dump-directory=/storage3/n2disk/pcap
--dump-directory=/storage4/n2disk/pcap
--dump-directory=/storage5/n2disk/pcap
--dump-directory=/storage6/n2disk/pcap
--dump-directory=/storage7/n2disk/pcap
--dump-directory=/storage8/n2disk/pcap
--timeline-dir=/storage1/n2disk/timeline
--timeline-dir=/storage2/n2disk/timeline
--timeline-dir=/storage3/n2disk/timeline
--timeline-dir=/storage4/n2disk/timeline
--timeline-dir=/storage5/n2disk/timeline
--timeline-dir=/storage6/n2disk/timeline
--timeline-dir=/storage7/n2disk/timeline
--timeline-dir=/storage8/n2disk/timeline
--disk-limit=80%
--max-file-len=1024
--buffer-len=8192
--index
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--writer-cpu-affinity=0
--reader-cpu-affinity=1
--reader-cpu-affinity=2
--reader-cpu-affinity=3
--reader-cpu-affinity=4
--reader-cpu-affinity=5
--reader-cpu-affinity=6
--reader-cpu-affinity=7
--reader-cpu-affinity=8
--indexer-cpu-affinity=9,10
--indexer-cpu-affinity=11,12
--indexer-cpu-affinity=13,14
--indexer-cpu-affinity=15,0
--indexer-cpu-affinity=25,26
--indexer-cpu-affinity=27,28
--indexer-cpu-affinity=29,30
--indexer-cpu-affinity=31,16
```

--------------------------------

### Start Napatech driver

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt

Starts the Napatech driver after configuration changes have been made.

```console
/opt/napatech3/bin/ntstart.sh
```

--------------------------------

### L7 filter configuration file example

Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt

Example configuration file for PF_RING FT filtering rules, specifying default action and per-protocol actions (discard, forward, slice).

```text
[global]
default = forward

[filter]
YouTube = discard
Netflix = discard
SSH = slice
```

--------------------------------

### npcapindex example

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapindex.rst.txt

Example command that indexes the pcap file dummy.pcap and writes the index to dummy.index.

```console
npcapindex -i dummy.pcap -o dummy.index
```

--------------------------------

### Force-start n2disk instance

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

force-start starts an instance even without a .start file.

```plaintext
# /etc/init.d/n2disk force-start eth4
Starting n2disk eth4
```

--------------------------------

### npcapdecompress example command

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapdecompress.rst.txt

Example command to decompress a compressed .npcap file named dummy.npcap into standard pcap format as dummy.pcap.

```console
npcapdecompress -i dummy.npcap -o dummy.pcap
```

--------------------------------

### Start PF_RING driver

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt

Loads the PF_RING driver using systemctl.

```console
systemctl start pf_ring
```

--------------------------------

### List n2disk configuration files

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Example listing of /etc/n2disk/ with two configuration files for interfaces eth0 and eth4.

```bash
# ls -lha /etc/n2disk/
   4.0K -rw-r--r--   1 root root  211 Mar 15 17:54 n2disk-eth0.conf
   4.0K -rw-r--r--   1 root root  195 Jan  8 17:17 n2disk-eth4.conf
```

--------------------------------

### npcapmove relative links example

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt

Example of moving a pcap file using relative symbolic links, which is the default and recommended mode for portability.

```console
npcapmove /storage/1.pcap /new-storage /new-storage/timeline
```

--------------------------------

### Show n2disk configuration files

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

Example listing of two configuration files, one per interface (eth0 and eth4).

```plaintext
# ls -lha /etc/n2disk/
4.0K -rw-r--r--   1 root root  211 Mar 15 17:54 n2disk-eth0.conf
4.0K -rw-r--r--   1 root root  195 Jan  8 17:17 n2disk-eth4.conf
```

--------------------------------

### Example output: human-readable format

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt

Sample output showing index version, packet details, and L7 protocol.

```text
Found index version 2 [packet index]
Index includes timestamps
1) [14:23:45.123456] len: 74, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 192.168.1.100:49152 -> 10.0.0.1:80, l7proto: HTTP
2) [14:23:45.123789] len: 60, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 10.0.0.1:80 -> 192.168.1.100:49152, l7proto: HTTP
3) [14:23:45.124012] len: 1514, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 10.0.0.1:80 -> 192.168.1.100:49152, l7proto: HTTP
```

--------------------------------

### disk2n example command (console)

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/disk2n.rst.txt

Example invocation of disk2n replaying a playlist of pcap files on interface zc:eth1 with a 4096 KB chunk, 2000 MB buffer, time pulse on core 0, reader on core 1, and sender on core 2.

```console
disk2n -i zc:eth1 -m playlist.txt -C 4096 -b 2000 -S 0 -c 1 -w 2
```

--------------------------------

### Example output: JSON format

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt

Sample JSON output for a single packet.

```json
{ "epoch" : 1234567890.123456, "length" : 74, "tuple" : { "vlan" : 0, "vlan_qinq" : 0, "ip_version" : 4, "protocol" : 6, "src_ip" : "192.168.1.100", "dst_ip" : "10.0.0.1", "src_port" : 49152, "dst_port" : 80, "l7protocol" : "HTTP" } }
```

--------------------------------

### BPF filter example using l7proto

Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt

Example of a BPF-like filter using the l7proto primitive to match SSH traffic from a specific IP address.

```text
ip host 192.168.1.1 and l7proto SSH
```

--------------------------------

### Load PF_RING driver and start n2disk

Source: https://www.ntop.org/guides/n2disk/tuning/fiberblaze.html

Ensure the PF_RING driver is loaded, then start n2disk with the specified options for Silicom FPGA (Fiberblaze) adapters. chunk mode is configured automatically.

```bash
systemctl start pf_ring

```

```bash
n2disk -o /storage -p 1000 -b 4000 -i fbcard:0:a00 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100

```

--------------------------------

### npcapmove absolute links example

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt

Example of moving a pcap file using the -a flag to create absolute symbolic links, useful when storage and timeline are on different filesystems.

```console
npcapmove -a /storage/1.pcap /external-storage /external-storage/timeline
```

--------------------------------

### Enable n2disk on boot

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Starts n2disk on boot if it was previously disabled.

```bash
systemctl enable n2disk@eth0
```

--------------------------------

### Example output: protocol statistics

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt

Sample statistics output when compiled with nDPI support and using index V2.

```text
--- Protocols stats ---
HTTP: 1523 pkts (45.32%)
TLS: 892 pkts (26.54%)
DNS: 421 pkts (12.53%)
SSH: 267 pkts (7.95%)
Unknown: 257 pkts (7.65%)
```

--------------------------------

### Create Napatech stream and start n2disk

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt

Instructs the Napatech card to create a stream aggregating ports 0 and 1 (Assign[streamid=0] = port == 0,1) and then starts n2disk using that stream via the interface name nt:stream0. The nt:stream<ID> syntax is used to open a stream.

```console
ntpl -e "Assign[streamid=0] = port == 0,1"
```

```console
n2disk -o /storage -p 1000 -b 4000 -i nt:stream0 -q 1 -C 1024 -w 0 -c 1 --disk-limit 80%
```

--------------------------------

### Repair after storage migration

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt

Example command to repair after moving storage to a new location.

```console
# After moving storage to new location
npcaprepair /new-storage /new-storage/timeline
```

--------------------------------

### List configuration and .start files

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Example listing showing a .start file next to the configuration file for automatic startup on init-based systems.

```bash
# ls -lha /etc/n2disk
   -rw-r--r--   1 root root  211 mar 15 17:54 n2disk-eth0.conf
   -rw-r--r--   1 root root    0 mar 17 15:44 n2disk-eth0.start
```

--------------------------------

### Run n2disk with command line vs config file

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

The command line example (using -i and -o) equals using the configuration file, whose contents are shown. In the config file each option is on its own line.

```plaintext
n2disk -i eth0 -o /storage
```

```plaintext
n2disk /etc/n2disk/n2disk-eth0.conf
```

```plaintext
 # cat /etc/n2disk/n2disk-eth0.conf
-i=eth0
-o=/storage
```

--------------------------------

### Complex capture filter examples

Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt

Examples of complex BPF filter expressions using AND, OR, and NOT operators with parentheses for grouping.

```text
ip host 192.168.1.1 and 192.168.1.2
ip src 192.168.1.1 and dst 192.168.1.2

ip host 192.168.1.1 and tcp port (80 or 443)
(ip host 192.168.1.1 or 192.168.1.2) and (port 80 or 443)
```

--------------------------------

### Start n2disk with Napatech port

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt

Starts n2disk with a Napatech port (nt:0) as the capture interface. The port number is the numeric suffix after 'nt:'.

```console
n2disk -o /storage -p 1000 -b 4000 -i nt:0 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100
```

--------------------------------

### Show init script usage

Source: https://www.ntop.org/guides/n2disk/how_to_start.html

Calling the script without options prints the usage line.

```plaintext
sudo /etc/init.d/n2disk

Usage: /etc/init.d/n2disk {start|force-start|stop|restart|status} [interface(s)]
```

--------------------------------

### Usage: npcapprintindex -i <index file> [options]

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt

Shows the basic command-line syntax for running npcapprintindex.

```text
npcapprintindex -i <index file> [options]
```

--------------------------------

### Install tshark (Wireshark command-line tool)

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt

Commands to install tshark on Debian/Ubuntu and Red Hat/CentOS. tshark is required for reading packet timestamps from pcap files.

```console
# Debian/Ubuntu
sudo apt-get install tshark

# Red Hat/CentOS
sudo yum install wireshark
```

--------------------------------

### Show init script usage

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Invoking the init script without options prints usage and available commands.

```bash
sudo /etc/init.d/n2disk
   
   Usage: /etc/init.d/n2disk {start|force-start|stop|restart|status} [interface(s)]
```

--------------------------------

### Dump 10-20 Gbps: configuration file

Source: https://www.ntop.org/guides/n2disk/tuning/examples.html

This configuration file corresponds to the 10-20 Gbps dump example. It uses the same parameters as the command line and sets CPU affinities for reader and writer. Requires a 4-core 3Ghz system.

```config
--interface=zc:eth1
--dump-directory=/storage/n2disk/pcap
--disk-limit=80%
--max-file-len=1024
--buffer-len=8192
--reader-cpu-affinity=1
--writer-cpu-affinity=2
--time-pulse=0

```

--------------------------------

### Start PF_RING then run n2disk on Silicom FPGA

Source: https://www.ntop.org/guides/n2disk/_sources/tuning/fiberblaze.rst.txt

First, ensure the PF_RING driver is loaded as a systemd service. Then launch n2disk with the specified options to capture from the Fiberblaze card. The -i fbcard:0:a00 designates the Fibreblaze interface and the other flags configure output directory, packet size, buffer size, queue, and other capture parameters.

```console
systemctl start pf_ring
```

```console
n2disk -o /storage -p 1000 -b 4000 -i fbcard:0:a00 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100
```

--------------------------------

### IPv6 Abbreviated Representation Example

Source: https://www.ntop.org/guides/n2disk/utilities/npcapprintindex.html

Due to index optimization, IPv6 addresses are hashed and displayed in abbreviated form showing only the last 4 bytes. This example shows the output format for a tunneled IPv6 flow.

```text
*:a1b2:c3d4:45152 -> *:e5f6:7890:443

```

--------------------------------

### Primitive BPF filter examples

Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt

Examples of primitive BPF-like filter expressions using allowed qualifiers: protocol (ether, ip, ip6, tcp, udp, sctp), direction (src, dst), type (host, net, port), and vlan.

```text
ether host 00:11:22:33:44:55
ether src host 00:11:22:33:44:55

ip host 192.168.0.1
ip dst host 192.168.0.1

ip6 host 2001:0db8:85a3:0042:0000:8a2e:0370:7334
ip6 src host 2001:0db8:85a3:0042:0000:8a2e:0370:7334

ip net 192.168.1.0/24
ip src net 192.168.1

port 80
udp port 9000
tcp src port 80
vlan 32
```

--------------------------------

### Disable n2disk on boot

Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt

Prevents n2disk from starting on boot.

```bash
systemctl disable n2disk@eth0
```

--------------------------------

### IPv6 representation example

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt

Shows how IPv6 addresses are displayed in abbreviated form.

```text
*:a1b2:c3d4:45152 -> *:e5f6:7890:443
```

--------------------------------

### Example output of npcaprepair

Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt

Sample output showing the checks and repairs performed by npcaprepair, including index creation, timeline slot creation, and link creation.

```text
Checking storage /storage/n2disk and timeline /storage/n2disk/timeline
Checking /storage/n2disk/1632826800.123456.pcap [date 2021/09/28] [time 14:20:00.123456] [epoch 1632826800.123456] [slot 20]
Creating index /storage/n2disk/1632826800.123456.pcap.idx
Creating timeline slot /storage/n2disk/timeline/2021/09/28/14/20
Creating link /storage/n2disk/timeline/2021/09/28/14/20/1632826800.123456.pcap
Creating link /storage/n2disk/1632826800.123456.pcap.timeline
Creating link /storage/n2disk/timeline/2021/09/28/14/20/1632826800.123456.pcap.idx
Creating link /storage/n2disk/1632826800.123456.pcap.idx.timeline
Checking /storage/n2disk/1632827400.654321.pcap [date 2021/09/28] [time 14:30:00.654321] [epoch 1632827400.654321] [slot 30]
```
Tokens:
Raw