### List n2disk config and start files Source: https://www.ntop.org/guides/n2disk/how_to_start.html Example listing showing the .start file required for automatic startup. ```plaintext # ls -lha /etc/n2disk -rw-r--r-- 1 root root 211 mar 15 17:54 n2disk-eth0.conf -rw-r--r-- 1 root root 0 mar 17 15:44 n2disk-eth0.start ``` -------------------------------- ### Start n2disk with init script (start) Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Demonstrates the 'start' option: starts only interfaces with a .start file. Without it, an error is shown. ```bash # ls -lha /etc/n2disk -rw-r--r-- 1 root root 211 mar 15 17:54 n2disk-eth0.conf -rw-r--r-- 1 root root 0 mar 17 15:44 n2disk-eth0.start -rw-r--r-- 1 root root 195 Jan 8 17:17 n2disk-eth4.conf # /etc/init.d/n2disk start eth0 Starting n2disk eth0 # /etc/init.d/n2disk start eth4 n2disk eth4 not started: missing /etc/n2disk/n2disk-eth4.start ``` -------------------------------- ### Start n2disk instance (requires .start) Source: https://www.ntop.org/guides/n2disk/how_to_start.html Starting an instance with a .start file succeeds; starting one without it fails with the error message shown. ```plaintext # ls -lha /etc/n2disk -rw-r--r-- 1 root root 211 mar 15 17:54 n2disk-eth0.conf -rw-r--r-- 1 root root 0 mar 17 15:44 n2disk-eth0.start -rw-r--r-- 1 root root 195 Jan 8 17:17 n2disk-eth4.conf # /etc/init.d/n2disk start eth0 Starting n2disk eth0 # /etc/init.d/n2disk start eth4 n2disk eth4 not started: missing /etc/n2disk/n2disk-eth4.start ``` -------------------------------- ### Start Napatech Driver with ntstart.sh Source: https://www.ntop.org/guides/n2disk/tuning/napatech.html Starts the Napatech driver after configuration changes have been completed. ```bash /opt/napatech3/bin/ntstart.sh ``` -------------------------------- ### Force start n2disk (force-start) Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt force-start starts the daemon even when no .start file exists. ```bash # /etc/init.d/n2disk force-start eth4 Starting n2disk eth4 ``` -------------------------------- ### n2disk command-line configuration for multi-directory capture Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt Use this n2disk command-line example to configure a multi-directory capture setup with eight dump and timeline directories, a disk limit of 80%, a maximum file length of 1024 MB, a buffer length of 8192, indexing enabled, and CPU affinities for writers, readers, and indexers. ```bash --interface=mlx:mlx_5@[0-7] --dump-directory=/storage1/n2disk/pcap --dump-directory=/storage2/n2disk/pcap --dump-directory=/storage3/n2disk/pcap --dump-directory=/storage4/n2disk/pcap --dump-directory=/storage5/n2disk/pcap --dump-directory=/storage6/n2disk/pcap --dump-directory=/storage7/n2disk/pcap --dump-directory=/storage8/n2disk/pcap --timeline-dir=/storage1/n2disk/timeline --timeline-dir=/storage2/n2disk/timeline --timeline-dir=/storage3/n2disk/timeline --timeline-dir=/storage4/n2disk/timeline --timeline-dir=/storage5/n2disk/timeline --timeline-dir=/storage6/n2disk/timeline --timeline-dir=/storage7/n2disk/timeline --timeline-dir=/storage8/n2disk/timeline --disk-limit=80% --max-file-len=1024 --buffer-len=8192 --index --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --writer-cpu-affinity=0 --reader-cpu-affinity=1 --reader-cpu-affinity=2 --reader-cpu-affinity=3 --reader-cpu-affinity=4 --reader-cpu-affinity=5 --reader-cpu-affinity=6 --reader-cpu-affinity=7 --reader-cpu-affinity=8 --indexer-cpu-affinity=9,10 --indexer-cpu-affinity=11,12 --indexer-cpu-affinity=13,14 --indexer-cpu-affinity=15,0 --indexer-cpu-affinity=25,26 --indexer-cpu-affinity=27,28 --indexer-cpu-affinity=29,30 --indexer-cpu-affinity=31,16 ``` -------------------------------- ### Start Napatech driver Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt Starts the Napatech driver after configuration changes have been made. ```console /opt/napatech3/bin/ntstart.sh ``` -------------------------------- ### L7 filter configuration file example Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Example configuration file for PF_RING FT filtering rules, specifying default action and per-protocol actions (discard, forward, slice). ```text [global] default = forward [filter] YouTube = discard Netflix = discard SSH = slice ``` -------------------------------- ### npcapindex example Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapindex.rst.txt Example command that indexes the pcap file dummy.pcap and writes the index to dummy.index. ```console npcapindex -i dummy.pcap -o dummy.index ``` -------------------------------- ### Force-start n2disk instance Source: https://www.ntop.org/guides/n2disk/how_to_start.html force-start starts an instance even without a .start file. ```plaintext # /etc/init.d/n2disk force-start eth4 Starting n2disk eth4 ``` -------------------------------- ### List configuration and .start files Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Example listing showing a .start file next to the configuration file for automatic startup on init-based systems. ```bash # ls -lha /etc/n2disk -rw-r--r-- 1 root root 211 mar 15 17:54 n2disk-eth0.conf -rw-r--r-- 1 root root 0 mar 17 15:44 n2disk-eth0.start ``` -------------------------------- ### npcapdecompress example command Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapdecompress.rst.txt Example command to decompress a compressed .npcap file named dummy.npcap into standard pcap format as dummy.pcap. ```console npcapdecompress -i dummy.npcap -o dummy.pcap ``` -------------------------------- ### Start PF_RING driver Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt Loads the PF_RING driver using systemctl. ```console systemctl start pf_ring ``` -------------------------------- ### List n2disk configuration files Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Example listing of /etc/n2disk/ with two configuration files for interfaces eth0 and eth4. ```bash # ls -lha /etc/n2disk/ 4.0K -rw-r--r-- 1 root root 211 Mar 15 17:54 n2disk-eth0.conf 4.0K -rw-r--r-- 1 root root 195 Jan 8 17:17 n2disk-eth4.conf ``` -------------------------------- ### npcapmove relative links example Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt Example of moving a pcap file using relative symbolic links, which is the default and recommended mode for portability. ```console npcapmove /storage/1.pcap /new-storage /new-storage/timeline ``` -------------------------------- ### Show n2disk configuration files Source: https://www.ntop.org/guides/n2disk/how_to_start.html Example listing of two configuration files, one per interface (eth0 and eth4). ```plaintext # ls -lha /etc/n2disk/ 4.0K -rw-r--r-- 1 root root 211 Mar 15 17:54 n2disk-eth0.conf 4.0K -rw-r--r-- 1 root root 195 Jan 8 17:17 n2disk-eth4.conf ``` -------------------------------- ### Example output: human-readable format Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Sample output showing index version, packet details, and L7 protocol. ```text Found index version 2 [packet index] Index includes timestamps 1) [14:23:45.123456] len: 74, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 192.168.1.100:49152 -> 10.0.0.1:80, l7proto: HTTP 2) [14:23:45.123789] len: 60, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 10.0.0.1:80 -> 192.168.1.100:49152, l7proto: HTTP 3) [14:23:45.124012] len: 1514, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 10.0.0.1:80 -> 192.168.1.100:49152, l7proto: HTTP ``` -------------------------------- ### Run n2disk with command line vs config file Source: https://www.ntop.org/guides/n2disk/how_to_start.html The command line example (using -i and -o) equals using the configuration file, whose contents are shown. In the config file each option is on its own line. ```plaintext n2disk -i eth0 -o /storage ``` ```plaintext n2disk /etc/n2disk/n2disk-eth0.conf ``` ```plaintext # cat /etc/n2disk/n2disk-eth0.conf -i=eth0 -o=/storage ``` -------------------------------- ### disk2n example command (console) Source: https://www.ntop.org/guides/n2disk/_sources/utilities/disk2n.rst.txt Example invocation of disk2n replaying a playlist of pcap files on interface zc:eth1 with a 4096 KB chunk, 2000 MB buffer, time pulse on core 0, reader on core 1, and sender on core 2. ```console disk2n -i zc:eth1 -m playlist.txt -C 4096 -b 2000 -S 0 -c 1 -w 2 ``` -------------------------------- ### BPF filter example using l7proto Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Example of a BPF-like filter using the l7proto primitive to match SSH traffic from a specific IP address. ```text ip host 192.168.1.1 and l7proto SSH ``` -------------------------------- ### Example output: JSON format Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Sample JSON output for a single packet. ```json { "epoch" : 1234567890.123456, "length" : 74, "tuple" : { "vlan" : 0, "vlan_qinq" : 0, "ip_version" : 4, "protocol" : 6, "src_ip" : "192.168.1.100", "dst_ip" : "10.0.0.1", "src_port" : 49152, "dst_port" : 80, "l7protocol" : "HTTP" } } ``` -------------------------------- ### Load PF_RING driver and start n2disk Source: https://www.ntop.org/guides/n2disk/tuning/fiberblaze.html Ensure the PF_RING driver is loaded, then start n2disk with the specified options for Silicom FPGA (Fiberblaze) adapters. chunk mode is configured automatically. ```bash systemctl start pf_ring ``` ```bash n2disk -o /storage -p 1000 -b 4000 -i fbcard:0:a00 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100 ``` -------------------------------- ### npcapmove absolute links example Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt Example of moving a pcap file using the -a flag to create absolute symbolic links, useful when storage and timeline are on different filesystems. ```console npcapmove -a /storage/1.pcap /external-storage /external-storage/timeline ``` -------------------------------- ### Enable n2disk on boot Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Starts n2disk on boot if it was previously disabled. ```bash systemctl enable n2disk@eth0 ``` -------------------------------- ### Example output: protocol statistics Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Sample statistics output when compiled with nDPI support and using index V2. ```text --- Protocols stats --- HTTP: 1523 pkts (45.32%) TLS: 892 pkts (26.54%) DNS: 421 pkts (12.53%) SSH: 267 pkts (7.95%) Unknown: 257 pkts (7.65%) ``` -------------------------------- ### Start n2disk with Napatech port Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt Starts n2disk with a Napatech port (nt:0) as the capture interface. The port number is the numeric suffix after 'nt:'. ```console n2disk -o /storage -p 1000 -b 4000 -i nt:0 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100 ``` -------------------------------- ### Create Napatech stream and start n2disk Source: https://www.ntop.org/guides/n2disk/_sources/tuning/napatech.rst.txt Instructs the Napatech card to create a stream aggregating ports 0 and 1 (Assign[streamid=0] = port == 0,1) and then starts n2disk using that stream via the interface name nt:stream0. The nt:stream syntax is used to open a stream. ```console ntpl -e "Assign[streamid=0] = port == 0,1" ``` ```console n2disk -o /storage -p 1000 -b 4000 -i nt:stream0 -q 1 -C 1024 -w 0 -c 1 --disk-limit 80% ``` -------------------------------- ### Dump 10-20 Gbps: configuration file Source: https://www.ntop.org/guides/n2disk/tuning/examples.html This configuration file corresponds to the 10-20 Gbps dump example. It uses the same parameters as the command line and sets CPU affinities for reader and writer. Requires a 4-core 3Ghz system. ```config --interface=zc:eth1 --dump-directory=/storage/n2disk/pcap --disk-limit=80% --max-file-len=1024 --buffer-len=8192 --reader-cpu-affinity=1 --writer-cpu-affinity=2 --time-pulse=0 ``` -------------------------------- ### Example output of npcaprepair Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt Sample output showing the checks and repairs performed by npcaprepair, including index creation, timeline slot creation, and link creation. ```text Checking storage /storage/n2disk and timeline /storage/n2disk/timeline Checking /storage/n2disk/1632826800.123456.pcap [date 2021/09/28] [time 14:20:00.123456] [epoch 1632826800.123456] [slot 20] Creating index /storage/n2disk/1632826800.123456.pcap.idx Creating timeline slot /storage/n2disk/timeline/2021/09/28/14/20 Creating link /storage/n2disk/timeline/2021/09/28/14/20/1632826800.123456.pcap Creating link /storage/n2disk/1632826800.123456.pcap.timeline Creating link /storage/n2disk/timeline/2021/09/28/14/20/1632826800.123456.pcap.idx Creating link /storage/n2disk/1632826800.123456.pcap.idx.timeline Checking /storage/n2disk/1632827400.654321.pcap [date 2021/09/28] [time 14:30:00.654321] [epoch 1632827400.654321] [slot 30] ``` -------------------------------- ### Repair after storage migration Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt Example command to repair after moving storage to a new location. ```console # After moving storage to new location npcaprepair /new-storage /new-storage/timeline ``` -------------------------------- ### Start PF_RING then run n2disk on Silicom FPGA Source: https://www.ntop.org/guides/n2disk/_sources/tuning/fiberblaze.rst.txt First, ensure the PF_RING driver is loaded as a systemd service. Then launch n2disk with the specified options to capture from the Fiberblaze card. The -i fbcard:0:a00 designates the Fibreblaze interface and the other flags configure output directory, packet size, buffer size, queue, and other capture parameters. ```console systemctl start pf_ring ``` ```console n2disk -o /storage -p 1000 -b 4000 -i fbcard:0:a00 -q 1 -C 1024 -c 1 -z 2 -w 3 -n 50 -m 100 ``` -------------------------------- ### Install tshark (Wireshark command-line tool) Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt Commands to install tshark on Debian/Ubuntu and Red Hat/CentOS. tshark is required for reading packet timestamps from pcap files. ```console # Debian/Ubuntu sudo apt-get install tshark # Red Hat/CentOS sudo yum install wireshark ``` -------------------------------- ### IPv6 representation example Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Shows how IPv6 addresses are displayed in abbreviated form. ```text *:a1b2:c3d4:45152 -> *:e5f6:7890:443 ``` -------------------------------- ### Example with default = discard and MDNS = forward - packets on the network Source: https://www.ntop.org/guides/n2disk/filters.html Shows the packet statistics on the network for a configuration where default is discard and MDNS is forward. This is the input before filtering. ```text Unknown packets: 567 bytes: 64030 flows: 4 DNS packets: 8 bytes: 864 flows: 2 MDNS packets: 6 bytes: 480 flows: 2 SSDP packets: 8 bytes: 1728 flows: 2 SSH packets: 1755 bytes: 702890 flows: 4 Dropbox packets: 3 bytes: 639 flows: 1 ``` -------------------------------- ### Complex capture filter examples Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Examples of complex BPF filter expressions using AND, OR, and NOT operators with parentheses for grouping. ```text ip host 192.168.1.1 and 192.168.1.2 ip src 192.168.1.1 and dst 192.168.1.2 ip host 192.168.1.1 and tcp port (80 or 443) (ip host 192.168.1.1 or 192.168.1.2) and (port 80 or 443) ``` -------------------------------- ### IPv6 Abbreviated Representation Example Source: https://www.ntop.org/guides/n2disk/utilities/npcapprintindex.html Due to index optimization, IPv6 addresses are hashed and displayed in abbreviated form showing only the last 4 bytes. This example shows the output format for a tunneled IPv6 flow. ```text *:a1b2:c3d4:45152 -> *:e5f6:7890:443 ``` -------------------------------- ### Primitive BPF filter examples Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Examples of primitive BPF-like filter expressions using allowed qualifiers: protocol (ether, ip, ip6, tcp, udp, sctp), direction (src, dst), type (host, net, port), and vlan. ```text ether host 00:11:22:33:44:55 ether src host 00:11:22:33:44:55 ip host 192.168.0.1 ip dst host 192.168.0.1 ip6 host 2001:0db8:85a3:0042:0000:8a2e:0370:7334 ip6 src host 2001:0db8:85a3:0042:0000:8a2e:0370:7334 ip net 192.168.1.0/24 ip src net 192.168.1 port 80 udp port 9000 tcp src port 80 vlan 32 ``` -------------------------------- ### Dump 10-20 Gbps: configuration file Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt Configuration file equivalent for the 10-20 Gbps dump example. Sets interface, dump directory, disk limit, max file length, buffer length, and CPU affinities. ```console --interface=zc:eth1 --dump-directory=/storage/n2disk/pcap --disk-limit=80% --max-file-len=1024 --buffer-len=8192 --reader-cpu-affinity=1 --writer-cpu-affinity=2 --time-pulse=0 ``` -------------------------------- ### Example output: timestamp-only (-T) Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Sample output showing first and last timestamps. ```text First-TS [14:23:45.123456] Last-TS [18:45:12.987654] ``` -------------------------------- ### Repair dump set with timeline Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt Example command to repair a dump set with a timeline using npcaprepair. ```console npcaprepair /storage/n2disk /storage/n2disk/timeline ``` -------------------------------- ### Disable n2disk on boot Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Prevents n2disk from starting on boot. ```bash systemctl disable n2disk@eth0 ``` -------------------------------- ### Manage n2disk service with systemctl Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Commands to start, stop, and restart a n2disk service on systemd-based systems. ```bash systemctl start n2disk@eth0 systemctl stop n2disk@eth0 systemctl restart n2disk@eth0 ``` -------------------------------- ### Basic n2disk capture command Source: https://www.ntop.org/guides/n2disk/_sources/usage.rst.txt Starts n2disk capturing from eth1, writing to /storage with a buffer of 1024 MB and 512 packet blocks. Enables indexing and file rotation with a 50% disk limit for automatic storage management. ```console n2disk -i eth1 -o /storage -b 1024 -p 512 --disk-limit 50% -I -A /storage ``` -------------------------------- ### Move files with absolute links Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt Example of moving files with absolute links using the -a flag. ```console npcapmove -a /storage/1.pcap /mnt/archive /mnt/archive/timeline ``` -------------------------------- ### FPGA 2-stream n2disk configuration for 100 Gbps Source: https://www.ntop.org/guides/n2disk/tuning/examples.html Complete setup for 100 Gbps on FPGA adapters with 2 streams. First configure the Napatech adapter to hash traffic into two streams, then run two n2disk instances (stream0 and stream1), each with separate dump directories, timelines, and CPU affinities. Requires 12 cores at 3 GHz; tested up to 148 Mpps with no packet loss. ```bash /opt/napatech3/bin/ntpl -e "Delete=All" /opt/napatech3/bin/ntpl -e "HashMode = Hash5TupleSorted" /opt/napatech3/bin/ntpl -e "Setup[NUMANode=0]=Streamid==(0..1)" /opt/napatech3/bin/ntpl -e "Assign[streamid=(0..1)] = All" ``` ```text --interface=nt:stream0 --dump-directory=/storage1/n2disk/pcap --dump-directory=/storage2/n2disk/pcap --dump-directory=/storage3/n2disk/pcap --dump-directory=/storage4/n2disk/pcap --timeline-dir=/storage1/n2disk/timeline --disk-limit=80% --max-file-len=2048 --buffer-len=16384 --index --reader-cpu-affinity=1 --indexer-cpu-affinity=2,3,4,5 --writer-cpu-affinity=0,0,0,0 ``` ```text --interface=nt:stream1 --dump-directory=/storage5/n2disk/pcap --dump-directory=/storage6/n2disk/pcap --dump-directory=/storage7/n2disk/pcap --dump-directory=/storage8/n2disk/pcap --timeline-dir=/storage5/n2disk/timeline --disk-limit=80% --max-file-len=2048 --buffer-len=16384 --index --reader-cpu-affinity=6 --indexer-cpu-affinity=7,8,9,10 --writer-cpu-affinity=0,0,0,0 ``` -------------------------------- ### n2disk command to enable layer 7 indexing and extraction Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Command line example for n2disk that enables layer 7 indexing and extraction using the -E 2 option. It sets up packet capture with a disk limit and other parameters. ```console n2disk -i eth1 -o /storage -b 1024 -p 512 --disk-limit 50% -I -E 2 -A /storage -Z -w 0 -r 1 -z 1 ``` -------------------------------- ### Restart n2disk instance Source: https://www.ntop.org/guides/n2disk/how_to_start.html Restarts the daemon associated with the interface, showing stop then start output. ```plaintext # /etc/init.d/n2disk restart eth0 Stopping n2disk eth0 Starting n2disk eth0 ``` -------------------------------- ### FPGA 100 Gbps: n2disk configuration (stream 0) Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt n2disk configuration for stream 0 of a 100 Gbps FPGA setup. Uses four dump directories, timeline, and CPU affinities for reader, indexer, and writer. ```text --interface=nt:stream0 --dump-directory=/storage1/n2disk/pcap --dump-directory=/storage2/n2disk/pcap --dump-directory=/storage3/n2disk/pcap --dump-directory=/storage4/n2disk/pcap --timeline-dir=/storage1/n2disk/timeline --disk-limit=80% --max-file-len=2048 --buffer-len=16384 --index --reader-cpu-affinity=1 --indexer-cpu-affinity=2,3,4,5 --writer-cpu-affinity=0,0,0,0 ``` -------------------------------- ### Move multiple pcap files with shell loop Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt Example of moving multiple pcap files using a shell loop with npcapmove. ```console for pcap in /storage/n2disk/*.pcap; do npcapmove "$pcap" /external-storage /external-storage/timeline done ``` -------------------------------- ### FPGA 100 Gbps: n2disk configuration (stream 1) Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt n2disk configuration for stream 1 of a 100 Gbps FPGA setup. Uses four dump directories, timeline, and CPU affinities for reader, indexer, and writer. ```text --interface=nt:stream1 --dump-directory=/storage5/n2disk/pcap --dump-directory=/storage6/n2disk/pcap --dump-directory=/storage7/n2disk/pcap --dump-directory=/storage8/n2disk/pcap --timeline-dir=/storage5/n2disk/timeline --disk-limit=80% --max-file-len=2048 --buffer-len=16384 --index --reader-cpu-affinity=6 --indexer-cpu-affinity=7,8,9,10 --writer-cpu-affinity=0,0,0,0 ``` -------------------------------- ### ntopng collector mode command to receive n2disk flows Source: https://www.ntop.org/guides/n2disk/_sources/using_with_ntopng.rst.txt Run ntopng in collector mode to collect flows exported by n2disk. The example listens on tcp://*:5556c. ```bash ntopng -i "tcp://*:5556c" ``` -------------------------------- ### Run n2disk with --zmq to export flows to ntopng Source: https://www.ntop.org/guides/n2disk/using_with_ntopng.html Starts n2disk with ZMQ flow export enabled, sending TLV or JSON formatted flows to the specified endpoint. The --zmq option is required to enable export, and --max-file-duration controls how often PCAP files are produced, which affects flow export delay. ```bash n2disk -i eno1 -o /storage --zmq tcp://127.0.0.1:5556 -I -Z -z 0 --max-file-duration 10 ``` -------------------------------- ### npcapextract command to extract SSH traffic Source: https://www.ntop.org/guides/n2disk/_sources/filters.rst.txt Full example using npcapextract to retrieve SSH traffic generated by 192.168.1.1 within a specified time range, using the l7proto filter. ```console npcapextract -t /storage -b "2020-09-16 12:05:32" -e "2020-09-16 12:10:32" -o output.pcap -f "ip host 192.168.1.1 and l7proto SSH" ``` -------------------------------- ### n2disk command to export flows over ZMQ Source: https://www.ntop.org/guides/n2disk/_sources/using_with_ntopng.rst.txt Use the --zmq option to export flows and statistics in TLV or JSON format over ZMQ. The example captures from eno1, writes to /storage, and exports to tcp://127.0.0.1:5556 with -I -Z -z 0 and --max-file-duration 10. ```bash n2disk -i eno1 -o /storage --zmq tcp://127.0.0.1:5556 -I -Z -z 0 --max-file-duration 10 ``` -------------------------------- ### Storage directory with reverse links Source: https://www.ntop.org/guides/n2disk/utilities/npcaprepair.html Storage directory showing reverse links created by npcaprepair. Each pcap and idx file gets a .timeline symlink pointing back to the timeline directory. ```text storage/ ├── file.pcap ├── file.pcap.idx ├── file.pcap.timeline -> ../timeline/2021/09/28/14/20/1632826800.123456.pcap └── file.pcap.idx.timeline -> ../timeline/2021/09/28/14/20/1632826800.123456.pcap.idx ``` -------------------------------- ### Layer 7 Protocol Statistics Output Source: https://www.ntop.org/guides/n2disk/utilities/npcapprintindex.html When compiled with nDPI support and using index V2, npcapprintindex appends protocol statistics at the end of its output. The example shows the format with protocol names, packet counts, and percentages. ```text --- Protocols stats --- HTTP: 1523 pkts (45.32%) TLS: 892 pkts (26.54%) DNS: 421 pkts (12.53%) SSH: 267 pkts (7.95%) Unknown: 257 pkts (7.65%) ``` -------------------------------- ### Show init script usage Source: https://www.ntop.org/guides/n2disk/how_to_start.html Calling the script without options prints the usage line. ```plaintext sudo /etc/init.d/n2disk Usage: /etc/init.d/n2disk {start|force-start|stop|restart|status} [interface(s)] ``` -------------------------------- ### Usage: npcapprintindex -i [options] Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapprintindex.rst.txt Shows the basic command-line syntax for running npcapprintindex. ```text npcapprintindex -i [options] ``` -------------------------------- ### Show init script usage Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Invoking the init script without options prints usage and available commands. ```bash sudo /etc/init.d/n2disk Usage: /etc/init.d/n2disk {start|force-start|stop|restart|status} [interface(s)] ``` -------------------------------- ### Configuration file format equivalence Source: https://www.ntop.org/guides/n2disk/_sources/how_to_start.rst.txt Shows that a configuration file contains the same options as the command line, one per line. The file /etc/n2disk/n2disk-eth0.conf with lines -i=eth0 and -o=/storage is equivalent to the command 'n2disk -i eth0 -o /storage'. ```bash n2disk -i eth0 -o /storage ``` ```bash n2disk /etc/n2disk/n2disk-eth0.conf ``` ```bash # cat /etc/n2disk/n2disk-eth0.conf -i=eth0 -o=/storage ``` -------------------------------- ### disk2n command-line options (text block) Source: https://www.ntop.org/guides/n2disk/_sources/utilities/disk2n.rst.txt Lists all disk2n command-line options grouped by category: TX settings, buffering, CPU affinity, packet reforging, and other. Each option shows its long and short forms, argument, and description. ```text TX SETTINGS [--interface|-i] | Egress interface. [--pcap-file|-f] | Pcap file to replay. [--multiple-pcap-files|-m] | Text file containing a list of pcap files to replay | in sequence. [--timeline-dir|-A] | Timeline directory created by n2disk (-A). [--begin-time|-B] | Begin time on which packets will be selected. [--end-time|-E] | End time on which packets will be selected. [--coherent-ts-across-pcaps|-H] | Time coherence across pcap files (do not collapse | inter-pcap gap). [--inter-packet-time|-I] | Inter-packet time nsec (ignore pcap timestamps). [--precise-timing|-p] | High time accuracy (lower throughput, forces | active wait). [--active-wait|-g] | Active wait (higher cpu load). [--one-shot|-O] | Send pcap(s) once. BUFFERING SETTINGS [--chunk-len|-C] | (KBytes) Size of the chunk read from disk. [--buffer-len|-b] | (MBytes) Buffer length. [--hugepages|-U] | Use hugepages for memory allocation. [--disable-direct-io|-r] | Disable Direct I/O (experts only). CPU AFFINITY [--reader-cpu-affinity|-c] | Binds the reader thread to the specified core. [--sender-cpu-affinity|-w] | Binds the sender thread to the specified core. [--time-pulse|-S] [] | Bind time pulse thread to the specified core. PACKET REFORGING [--src-mac] | Reforge source MAC address. [--dst-mac] | Reforge destination MAC address. [--src-ip] | Reforge source IP address. [--dst-ip] | Reforge destination IP address. [--src-port] | Reforge source port. [--dst-port] | Reforge destination port. | (comma-separated list in case of multiple egress | interfaces, one per interface) OTHER [--simulation-mode|-e] | Simulation mode (disk simulation, synthetic traffic is | generated). [--unprivileged-user|-u] | Use instead of nobody when dropping | privileges. [--pid|-P] | Save the pid into the specified path. [--version|-V] | Print application version. [--help|-h] | Help. [--verbose|-v] | Verbose. [--show-system-id] | Print the system identifier. [--check-license] | Checks if the license is present and valid. [--syslog|-l] | Dump trace messages to syslog. [--event-log|-Q] | Save relevant events onto the specified file. ``` -------------------------------- ### Run ndedup with RSS across 4 queues Source: https://www.ntop.org/guides/n2disk/utilities/ndedup.html Uses RSS short notation to run ndedup across four queues (eth1@0-3 and eth2@0-3) on Intel ZC adapters.$tIt keeps the same 10 msec window and 10 Gbps speed, with Time Pulse on core 0 and binds to cores 1 through 4 in bridge mode. ```bash ndedup -i zc:eth1@[0-3] -o zc:eth2@[0-3] -d 10 -s 10 -S 0 -g 1:2:3:4 -B ``` -------------------------------- ### Fix broken timeline after manual file operations Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcaprepair.rst.txt Example command to fix a broken timeline after manual file operations. ```console npcaprepair /data/pcaps /data/timeline ``` -------------------------------- ### Configure HostBuffersRx = [4,128,0] Source: https://www.ntop.org/guides/n2disk/tuning/napatech.html Optionally increases the RX host buffer to 128 MB in the Napatech configuration file. ```ini HostBuffersRx = [4,128,0] ``` -------------------------------- ### Move single pcap file to external storage Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapmove.rst.txt Example of moving a single pcap file to external storage using npcapmove. ```console npcapmove /storage/n2disk/1632826800.123456.pcap /external-storage /external-storage/timeline ``` -------------------------------- ### n2disk miscellaneous command-line options Source: https://www.ntop.org/guides/n2disk/usage.html Lists the miscellaneous n2disk command-line options, including daemon mode, simulation modes, hole detection, and logging options. Each option is shown with its syntax and description. ```text [--daemon] | Daemonize this app at startup. [--disable-chunk-mode] | Do not enable chunk mode on supported adapters [--simulation-mode|-e] | Simulation mode (debug only) | 1 - Test max dump speed (simulate capture). | 2 - Test max capture speed (simulate storage). | 3 - Test max capture speed (simulate storage, produce real index). | 4 - Test both capture and storage (test the core engine). [--hole-detection] | Enable detection of traffic holes ( is the min hole size). [--ts-lsb-interf-encoding] | Encode ingress interface index in LSB of the timestamp. [--unprivileged-user|-u] | Use instead of nobody when dropping privileges. [--pid|-P] | Save the pid into the specified path. [--version|-V] | Print application version. [--help|-h] | Help. [--verbose|-v] | Verbose. [--show-system-id] | Print the system identifier [--check-license] | Checks if the license is present and valid [--check-maintenance] | Checks the maintenance status for the specified license [--syslog|-l] | Dump trace messages to syslog [--trace-log] | Dump trace messages to the specified file [--event-log|-Q] | Save relevant events (i.e. drops) onto the specified file ``` -------------------------------- ### Check n2disk status (init script) Source: https://www.ntop.org/guides/n2disk/how_to_start.html Prints the status of the daemon for the given interface. ```plaintext # /etc/init.d/n2disk status eth0 n2disk running ``` -------------------------------- ### Format and mount HDD with XFS for n2disk Source: https://www.ntop.org/guides/n2disk/filesystem.html Format and mount an HDD partition for n2disk using XFS with recommended parameters. Adjust parameters for small partition sizes. The mount options include noatime and nodiratime to reduce writes. ```bash mkfs.xfs -f -d sunit=128,swidth=1024,agcount=6000 -l size=64m /dev/sda1 ``` ```bash mount -o noatime,nodiratime,attr2,nobarrier,logbufs=8,logbsize=256k,osyncisdsync /dev/sda1 /storage/ ``` -------------------------------- ### GTP Tunnel Information Output Source: https://www.ntop.org/guides/n2disk/utilities/npcapprintindex.html For GTP indexes, both outer and tunneled packet information is displayed. The example shows the format with outer IP/UDP and inner IP/protocol details. ```text vlan: 0, ipv4, proto: 17, 10.1.1.1:2152 -> 10.1.1.2:2152, tunneled ipv4, proto: 6, 192.168.1.100:49152 -> 172.16.0.1:80 ``` -------------------------------- ### npcapextract command-line options Source: https://www.ntop.org/guides/n2disk/_sources/utilities/npcapextract.rst.txt Complete command-line usage of npcapextract. It describes all option groups: choosing timeline (-t) or single pcap/index (-a/-i), BPF filtering (-f) with a note that times must be formatted as YYYY-MM-DD hh:mm:ss, dump settings (-o, -s, -P, -N, -x), and other options including privilege dropping (-u), output to stdout (-O), listing (-l), performance test (-T), empty pcap on no match (-0), verbosity (-v), and help (-h). ```text FIND FROM TIMELINE [-t] | Timeline directory created by n2disk with -A (multiple -t are allowed) [-b] | Begin time on which packets will be selected [-e] | Endtime on which packets will be selected FIND FROM SINGLE FILE/INDEX [-a] | Pcap file from which extract packets [-i] | Index corresponding to the pcap file FILTERING [-f] | BPF ingress packet filter (tcpdump-like) | NOTE: -b/-e time must specified as YYYY-MM-DD hh:mm:ss DUMP SETTINGS [-o] | Pcap file or directory where matched packets will be saved [-s] | Cut packets to len (Bytes) [-P] | Max pcap file length (MBytes) [-N] | Max packets per pcap file. (Default: 0 - no max duration) [-x] | Dump file prefix (to be used with -P or -N) OTHER [-g] | CPU core affinity [-u] | Use instead of nobody when dropping privileges [-O] | Write output to stdout (npcapextract -O .. | tshark -i -) [-l] | List matching pcap files only (no packet extraction) [-T] | Test index scan performance (no packet extraction) [-0] | Write empty pcap on no match [-v] | Verbosity level: 0..6 (default = 2) [-h] | Help ``` -------------------------------- ### n2disk -e simulation mode options Source: https://www.ntop.org/guides/n2disk/_sources/performance.rst.txt Shows the command line option for n2disk simulation mode. Mode 1 tests max dump speed by simulating capture; mode 2 tests max capture speed by simulating storage. Use this to evaluate performance before deploying n2disk. ```text [--simulation-mode|-e] | Simulation mode (debug only) | 1 - Test max dump speed (simulate capture) | 2 - Test max capture speed (simulate storage) ``` -------------------------------- ### Run nProbe Cento with egress configuration file Source: https://www.ntop.org/guides/n2disk/_sources/netflow.rst.txt Runs nProbe Cento with an egress configuration file (egress.example) to control which traffic is forwarded to the aggregated egress queue, e.g., to avoid forwarding encrypted traffic or to forward only suspicious address ranges. The configuration file follows the INI standard and is specified with the --egress-conf option. ```console cento-ids -i zc:eth1@[0-3] --aggregated-egress-queue —egress-conf egress.example --dpi-level 2 --v9 192.168.2.221:2055 ``` -------------------------------- ### Dump and Index 10-20 Gbps: configuration file Source: https://www.ntop.org/guides/n2disk/_sources/tuning/examples.rst.txt Configuration file for dumping and indexing 10-20 Gbps on a 6-core system. Includes timeline directory, index flag, and indexer CPU affinity. ```text --interface=zc:eth1 --dump-directory=/storage/n2disk/pcap --timeline-dir=/storage/n2disk/timeline --disk-limit=80% --max-file-len=1024 --buffer-len=8192 --index --reader-cpu-affinity=1 --writer-cpu-affinity=2 --indexer-cpu-affinity=3,4,5 --time-pulse=0 ``` -------------------------------- ### Format and mount EXT4 SSD RAID 0 for n2disk Source: https://www.ntop.org/guides/n2disk/_sources/filesystem.rst.txt Formats an EXT4 filesystem on an SSD RAID 0 array with 16 SSDs, mounts it with discard and other options, and sets the 'noop' scheduler. The mkfs.ext4 command uses block size 4096, discard, stride=16 (chunk size 64K / block size 4K), and stripe-width=256 (number of disks x stride). The mount command uses rw,user,auto,discard; the scheduler is set with echo noop. ```console mkfs.ext4 -F -b 4096 -E discard,stride=16,stripe-width=256 /dev/sda1 ``` ```console mount -o rw,user,auto,discard /dev/sda1 /storage/ ``` ```console echo noop > /sys/block/sda/queue/scheduler ```