### Build nfdump with Optional Features Source: https://context7.com/phaag/nfdump/llms.txt Provides commands for compiling nfdump from source, including a basic build process and examples for enabling various optional features like MaxMind, Tor, sFlow, and NfSen integration. Also shows how to verify the installation. ```bash # Basic build ./autogen.sh ./configure make sudo make install # Build with all features ./configure \ --enable-maxmind \ --enable-tor \ --enable-sflow \ --enable-nfpcapd \ --enable-ja4 \ --enable-readpcap \ --with-lz4path=/usr/local \ --with-zstdpath=/usr/local make sudo make install # Build for NfSen integration ./configure \ --enable-nfprofile \ --enable-nftrack \ --with-rrdpath=/usr/local make sudo make install # Verify installation nfdump -V nfcapd -h ``` -------------------------------- ### Collecting NetFlow Data with nfcapd Source: https://context7.com/phaag/nfdump/llms.txt Examples for starting the nfcapd collector daemon to capture NetFlow data. Shows background operation, per-source splitting, buffer configuration, and basic Cisco router configuration for NetFlow export. ```bash # Start nfcapd collector in background nfcapd -D -w /var/log/netflow/router1 -p 9995 -S 5 -z=lz4 # Start collector with per-source splitting nfcapd -D -p 9995 -S 5 \ -n router1,192.168.1.1,/var/log/netflow/router1 \ -n router2,192.168.1.2,/var/log/netflow/router2 # Start with increased buffer and compression nfcapd -D -B 2048000 -w /var/log/netflow/all -p 9995 -S 2 -z=zstd # Configure Cisco router to export NetFlow v5 # On router: # interface FastEthernet0/0 # ip route-cache flow # ip flow-export destination 192.168.1.10 9995 # ip flow-export version 5 # ip flow-cache timeout active 5 ``` -------------------------------- ### Collect NetFlow Data with nfcapd Source: https://github.com/phaag/nfdump/blob/master/README.md Example command to start the nfcapd daemon for collecting NetFlow data. It runs in the background, sets up a specific number of buckets for processing, and defines the output directory and port. ```shell nfcapd -D -S 2 -w /flow_base_dir/router1 -p 23456 ``` -------------------------------- ### Flow Aggregation and Statistics with nfdump Source: https://context7.com/phaag/nfdump/llms.txt Examples for aggregating flow records and generating traffic statistics using nfdump. Shows how to find top talkers by source IP, top destination ports, create traffic matrices, and get protocol statistics. ```bash # Top talkers by source IP nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ -A srcip -n 20 -s bytes # Top destination ports nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ -A dstport -n 20 -s packets # Traffic matrix (source and destination IPs) nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ -A srcip,dstip -n 50 -s bytes # Protocol statistics nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ -s proto ``` -------------------------------- ### Build nfdump using GNU Autotools Source: https://github.com/phaag/nfdump/blob/master/README.md Standard build process for nfdump using GNU autotools. This involves generating configuration scripts, configuring the build, compiling the source code, and installing the binaries. ```shell ./autogen.sh ./configure make sudo make install ``` -------------------------------- ### Collect sFlow Data with sfcapd Source: https://context7.com/phaag/nfdump/llms.txt Start the sfcapd daemon to collect sFlow data from network devices. This tool can run as a daemon, write data to specified directories, and configure multiple sFlow sources. It also provides examples for configuring sFlow on network devices. ```bash # Start sFlow collector daemon sfcapd -D -w /var/log/sflow/switch1 -p 6343 -S 5 -z=lz4 # Collect from multiple sFlow sources sfcapd -D -p 6343 -S 5 \ -n switch1,192.168.2.1,/var/log/sflow/switch1 \ -n switch2,192.168.2.2,/var/log/sflow/switch2 # Configure sFlow on network device (example): # sflow agent-ip 192.168.2.1 # sflow collector ip 192.168.1.10 port 6343 # sflow sampling-rate 1024 # sflow polling-interval 20 ``` -------------------------------- ### Build nfdump on Ubuntu 18.04 LTS with Clang Source: https://github.com/phaag/nfdump/blob/master/README.md Instructions for building nfdump on Ubuntu 18.04 LTS using Clang as the compiler. This involves installing Clang and then configuring the build process to use it. ```shell sudo apt-get install clang-10 CC=clang-10 ./configure ... ``` -------------------------------- ### Reading and Filtering Flow Records with nfdump Source: https://context7.com/phaag/nfdump/llms.txt Demonstrates how to read flow files, apply filters based on IP addresses, ports, and protocols, and format the output. Includes examples for reading single or multiple files, using time ranges, and exporting data in CSV and JSON formats. ```bash # Read single flow file nfdump -r /var/log/netflow/router1/nfcapd.202501011200 # Filter by source IP and destination port nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ 'src ip 192.0.2.1 and dst port 443' # Read multiple files in time range nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ 'proto tcp and port 22' # Complex filter with aggregation nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ 'net 10.0.0.0/8 and bytes > 1000000' -A srcip,dstip -n 20 # Output as CSV for further processing nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -o csv 'src ip 192.0.2.0/24' > flows.csv # Output as JSON nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -o json 'dst port 80 or dst port 443' > web_flows.json # Custom output format nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -o "fmt:%ts %sa:%sp -> %da:%dp %pr %byt %pkt" # Expected output format: # 2025-01-01 12:00:05 192.0.2.1:52345 -> 198.51.100.1:443 TCP 45678 234 # 2025-01-01 12:00:08 192.0.2.3:51234 -> 203.0.113.5:80 TCP 12345 89 ``` -------------------------------- ### Convert Flow-tools to nfdump Format Source: https://context7.com/phaag/nfdump/llms.txt Demonstrates how to convert flow data from the flow-tools format to the nfdump format using the ft2nfdump utility. Includes examples for single file conversion and batch processing of multiple files. ```bash # Convert flow-tools file to nfdump ft2nfdump -r ft-v05.2025-01-01.120000 \ -w /var/log/netflow/converted/nfcapd.202501011200 # Batch conversion for ftfile in /var/log/flow-tools/ft-*; do timestamp=$(echo $ftfile | sed 's/.*ft-v05\.\([0-9-]*\)\.\([0-9]*\)/\1\2/') ft2nfdump -r "$ftfile" -w "/var/log/netflow/converted/nfcapd.$timestamp" done ``` -------------------------------- ### Filter and Aggregate NetFlow Data Source: https://github.com/phaag/nfdump/blob/master/README.md Example of using nfdump to filter flows from a file based on source IP and destination port, and then aggregate the results by source IP and destination IP. ```shell nfdump -r flowfile 'src ip 192.0.2.1 and dst port 443' -A srcip,dstip ``` -------------------------------- ### Manage nfdump Log Expiration Source: https://context7.com/phaag/nfdump/llms.txt Commands to manage log expiration for nfdump, including setting combined time and size limits, and an example of how to automate this process using cron. It also includes a command to verify disk usage of the log directory. ```bash # Combined limits (30 days AND 100GB) nfexpire -l /var/log/netflow/router1 -e 30d -s 100G # Automatic expiration via cron # Add to crontab: # 0 2 * * * /usr/local/bin/nfexpire -l /var/log/netflow/router1 -e 30d -s 100G # Verify disk usage du -sh /var/log/netflow/router1 ``` -------------------------------- ### Start Multiple nfcapd Collectors on Different Ports Source: https://github.com/phaag/nfdump/blob/master/README.md Shows how to start two independent nfcapd collectors, each listening on a different UDP port (23456 and 23457). This is recommended for busy networks to distribute the load and store data in separate directories for different flow sources (router1 and router2). ```sh nfcapd -D -S 2 -B 1024000 -w /flow_base_dir/router1 -p 23456 nfcapd -D -S 2 -B 1024000 -w /flow_base_dir/router2 -p 23457 ``` -------------------------------- ### Enable On-the-Fly Compression with nfcapd Source: https://github.com/phaag/nfdump/blob/master/README.md Demonstrates how to enable data compression during collection using the nfcapd tool. The `-z` option allows specifying the compression algorithm, such as lz4 for efficient real-time compression. Other supported algorithms include LZO, ZSTD, and bzip2. ```sh nfcapd -z=lz4 ... ``` -------------------------------- ### Build nfdump on CentOS 7.x with SCL Source: https://github.com/phaag/nfdump/blob/master/README.md Instructions for building nfdump on CentOS 7.x using the Software Collections (SCL) utility to enable specific GCC versions for compilation. ```shell yum install centos-release-scl yum install devtoolset-8-gcc devtoolset-8-gcc-c++ scl enable devtoolset-8 -- bash ``` -------------------------------- ### nfdump Filter Syntax Reference Source: https://context7.com/phaag/nfdump/llms.txt Provides a comprehensive reference for the advanced filtering syntax used by nfdump. Covers filtering by IP addresses, networks, ports, protocols, traffic volume, time ranges, AS numbers, interfaces, and TCP flags. ```bash # IP address filters nfdump -r flowfile 'src ip 192.0.2.1' nfdump -r flowfile 'dst net 10.0.0.0/8' nfdump -r flowfile 'ip 2001:db8::1' nfdump -r flowfile 'src ip in [192.0.2.1 192.0.2.2 192.0.2.3]' # Port filters nfdump -r flowfile 'port 80' nfdump -r flowfile 'src port > 1024 and dst port 443' nfdump -r flowfile 'port in [80 443 8080 8443]' # Protocol filters nfdump -r flowfile 'proto tcp' nfdump -r flowfile 'proto udp and port 53' nfdump -r flowfile 'icmp' # Traffic volume filters nfdump -r flowfile 'bytes > 1000000' nfdump -r flowfile 'packets < 10' nfdump -r flowfile 'duration > 3600' # Time-based filters nfdump -r flowfile 'timestamp > 2025-01-01T00:00:00 and timestamp < 2025-01-01T23:59:59' # Complex logical expressions nfdump -r flowfile '(src net 10.0.0.0/8 or src net 172.16.0.0/12) and dst port 22 and bytes > 100000' # AS number filters nfdump -r flowfile 'src as 64500 and dst as 64501' # Interface filters nfdump -r flowfile 'in if 2 and out if 5' # TCP flag filters nfdump -r flowfile 'flags S and not flags A' # SYN without ACK nfdump -r flowfile 'flags RST or flags F' # RST or FIN ``` -------------------------------- ### MaxMind Geolocation API in C Source: https://context7.com/phaag/nfdump/llms.txt Illustrates how to use the MaxMind geolocation functions provided by the nfdump library in C applications. It covers loading the database, performing country and city lookups for IPv4 and IPv6 addresses, and retrieving Autonomous System (AS) information. ```c #include // Load MaxMind database if (LoadMaxMind("/var/lib/nfdump/geodb.nf") != 1) { fprintf(stderr, "Failed to load geolocation database\n"); return 1; } // Lookup IPv4 country uint32_t ipv4 = 0xC0000201; // 192.0.2.1 char country[4]; LookupV4Country(ipv4, country); printf("Country: %s\n", country); // Output: Country: US // Lookup IPv4 location (city) char location[128]; LookupV4Location(ipv4, location, sizeof(location)); printf("Location: %s\n", location); // Output: Location: New York, US // Lookup IPv6 uint64_t ipv6[2] = {0x20010db800000000ULL, 0x0000000000000001ULL}; LookupV6Country(ipv6, country); printf("Country: %s\n", country); // Get AS information uint32_t asn; char asOrg[256]; LookupV4AS(ipv4, &asn, asOrg, sizeof(asOrg)); printf("AS%u: %s\n", asn, asOrg); // Output: AS15169: Google LLC ``` -------------------------------- ### Split Collected Data Per Source with nfcapd Source: https://github.com/phaag/nfdump/blob/master/README.md Demonstrates how to use the `-n` option with nfcapd to split incoming NetFlow data based on the source router or IP address. Each source is assigned a name, IP, and a dedicated directory for storing its flow records, allowing for better organization. ```sh nfcapd -D -S 2 -n router1,172.16.17.18,/flow_base_dir/router1 \ -n router2,172.16.17.20,/flow_base_dir/router2 -p 23456 ``` -------------------------------- ### Collect All Sources into a Single nfcapd File Source: https://github.com/phaag/nfdump/blob/master/README.md Illustrates how to configure a single nfcapd collector to receive NetFlow data from all sources and consolidate it into a single output file. This approach is simpler but may be less organized for managing data from many sources. ```sh nfcapd -D -S 2 -w /flow_base_dir/routers -p 23456 ``` -------------------------------- ### Replay Captured NetFlow Data with nfreplay Source: https://context7.com/phaag/nfdump/llms.txt Replay captured NetFlow data to a specified remote collector using nfreplay. This tool supports replaying individual files, directories of files within a time range, and allows for rate limiting to control the replay speed. It can also replay data in different NetFlow versions. ```bash # Replay flow file to remote collector nfreplay -r /var/log/netflow/router1/nfcapd.202501011200 \ -H 192.168.1.20:9995 # Replay multiple files in sequence nfreplay -R /var/log/netflow/router1 \ -t 2025-01-01/00:00-2025-01-01/23:59 \ -H 192.168.1.20:9995 # Replay with rate limiting (10x speedup) nfreplay -r /var/log/netflow/router1/nfcapd.202501011200 \ -H 192.168.1.20:9995 -X 10 # Replay as NetFlow v5 format nfreplay -r /var/log/netflow/router1/nfcapd.202501011200 \ -H 192.168.1.20:9995 -v5 ``` -------------------------------- ### Filter NetFlow Data with nfdump Syntax Source: https://github.com/phaag/nfdump/blob/master/README.md Shows how to use `nfdump` to filter NetFlow records based on specific criteria, such as source IP address and destination port. The filter syntax is similar to tcpdump, enabling powerful and flexible analysis of network traffic. ```sh nfdump -r flowfile 'src ip 192.0.2.1 and dst port 443' ``` -------------------------------- ### Convert Packet Captures to NetFlow with nfpcapd Source: https://context7.com/phaag/nfdump/llms.txt Generate NetFlow records from live network traffic or existing pcap files using nfpcapd. This tool can capture from network interfaces, process pcap files, forward records to a remote collector, and capture from multiple interfaces simultaneously. It supports various compression options and NetFlow versions. ```bash # Capture from network interface and create NetFlow records nfpcapd -D -i eth0 -w /var/log/nfpcap/local -S 5 -z=lz4 # Process existing pcap file nfpcapd -r capture.pcap -w /var/log/nfpcap/converted # Capture and forward to remote collector nfpcapd -i eth0 -H 192.168.1.10:9995 # Capture from multiple interfaces nfpcapd -D -i eth0 -w /var/log/nfpcap/eth0 & nfpcapd -D -i eth1 -w /var/log/nfpcap/eth1 & # Read pcap and export as NetFlow v5 nfpcapd -r traffic.pcap -w /tmp/flows -v5 ``` -------------------------------- ### Read nfdump Files in C Source: https://context7.com/phaag/nfdump/llms.txt A C code snippet demonstrating how to open, read, and process records from an nfdump file using the nfdump library. It iterates through data blocks and extracts flow information such as IP addresses, ports, bytes, and packets. ```c #include #include #include // Open nfdump file for reading nffile_t *nffile = OpenFile("/var/log/netflow/router1/nfcapd.202501011200", NULL); if (!nffile) { fprintf(stderr, "Failed to open file\n"); return 1; } // Read data blocks dataBlock_t *dataBlock = NULL; int done = 0; while (!done) { int ret = ReadBlock(nffile, &dataBlock); if (ret == NF_EOF) { done = 1; break; } if (ret != NF_OK) { fprintf(stderr, "Read error\n"); break; } // Process each record record_header_t *record = dataBlock->data; uint32_t numRecords = dataBlock->NumRecords; for (uint32_t i = 0; i < numRecords; i++) { // Cast to common record structure recordHeaderV3_t *v3Record = (recordHeaderV3_t *)record; // Access flow fields uint32_t srcIP = v3Record->srcaddr; uint32_t dstIP = v3Record->dstaddr; uint16_t srcPort = v3Record->srcPort; uint16_t dstPort = v3Record->dstPort; uint64_t bytes = v3Record->inBytes; uint64_t packets = v3Record->inPackets; printf("Flow: %u.%u.%u.%u:%u -> %u.%u.%u.%u:%u %lu bytes, %lu packets\n", srcIP >> 24, (srcIP >> 16) & 0xFF, (srcIP >> 8) & 0xFF, srcIP & 0xFF, srcPort, dstIP >> 24, (dstIP >> 16) & 0xFF, (dstIP >> 8) & 0xFF, dstIP & 0xFF, dstPort, bytes, packets); // Move to next record record = (record_header_t *)((void *)record + record->size); } } CloseFile(nffile); DisposeFile(nffile); ``` -------------------------------- ### Configure Cisco Router NetFlow v9 with Sampling Source: https://github.com/phaag/nfdump/blob/master/README.md Sets up NetFlow version 9 export with advanced features like flow sampling and detailed configuration options. This includes interface-level sampling map configuration and various export template settings. ```cisco-ios interface Ethernet1/0 ip address 192.168.92.162 255.255.255.224 duplex half flow-sampler my-map ! ! flow-sampler-map my-map mode random one-out-of 5 ! ip flow-cache timeout inactive 60 ip flow-cache timeout active 1 ip flow-capture fragment-offset ip flow-capture packet-length ip flow-capture ttl ip flow-capture vlan-id ip flow-capture icmp ip flow-capture ip-id ip flow-capture mac-addresses ip flow-export version 9 ip flow-export template options export-stats ip flow-export template options sampler ip flow-export template options timeout-rate 1 ip flow-export template timeout-rate 1 ip flow-export destination 192.168.92.218 9995 ``` -------------------------------- ### Convert Legacy nfdump Files to New Format Source: https://github.com/phaag/nfdump/blob/master/README.md This command converts older nfdump flow files (version 1.6.x) to the newer format required by recent versions of nfdump. Ensure you specify the correct input and output file paths. ```shell ./nfdump -r old-flowfile -y -w new-flowfile ``` -------------------------------- ### Enrich Flow Data with GeoIP and ASN Information using geolookup Source: https://context7.com/phaag/nfdump/llms.txt Use the geolookup tool to create a geolocation database from MaxMind files, then use this database with nfdump to filter and display flows based on geographic location (country, city) and AS number. The tool requires paths to MaxMind databases and an output file for the generated geoDB. ```bash # Create geolocation database from MaxMind files geolookup -c /usr/share/GeoIP/GeoLite2-City.mmdb \ -a /usr/share/GeoIP/GeoLite2-ASN.mmdb \ -s /var/lib/nfdump/geodb.nf # Process flows with geolocation data nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -G /var/lib/nfdump/geodb.nf \ 'src country US and dst country CN' # Filter by city nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -G /var/lib/nfdump/geodb.nf \ 'src location "New York"' # Filter by AS number from geolocation nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -G /var/lib/nfdump/geodb.nf \ 'src as 15169' # Google # Output with geo information nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -G /var/lib/nfdump/geodb.nf \ -o "fmt:%ts %sa:%sp %sc %sl -> %da:%dp %dc %dl %byt" ``` -------------------------------- ### Process Collected NetFlow Data with nfdump Source: https://github.com/phaag/nfdump/blob/master/README.md Basic command to process a single NetFlow data file using nfdump. This is typically used for reading and analyzing data collected by nfcapd. ```shell nfdump -r /flow_base_dir/router1/nfcapd.202501011200 ``` -------------------------------- ### Configure Cisco Router NetFlow Interface Source: https://github.com/phaag/nfdump/blob/master/README.md Enables NetFlow collection on a specific interface of a Cisco router. This configuration sets up the IP address for the interface and enables flow caching for NetFlow. ```cisco-ios interface fastethernet 0/0 ip address 192.168.92.162 255.255.255.224 ip route-cache flow ``` -------------------------------- ### Aggregate NetFlow Data by AS Numbers with nfdump Source: https://context7.com/phaag/nfdump/llms.txt Aggregate NetFlow records by source and destination Autonomous System (AS) numbers, limiting the output to the top 20 flows by bytes. This is useful for understanding traffic patterns between different network providers. ```bash nfdump -R /var/log/netflow/router1 -t 2025-01-01/00:00-2025-01-01/23:59 \ -A srcas,dstas -n 20 -s bytes ``` -------------------------------- ### Detect Tor Exit Nodes with torlookup Source: https://context7.com/phaag/nfdump/llms.txt Identify Tor exit node traffic by downloading and creating a Tor exit node database using torlookup. This database can then be used with nfdump to filter flows associated with Tor exit nodes. The tool supports downloading new databases or updating existing ones. ```bash # Download and create Tor exit node database torlookup -u -s /var/lib/nfdump/tordb.nf # Update existing database torlookup -u -f /var/lib/nfdump/tordb.nf # Filter flows involving Tor exit nodes nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -T /var/lib/nfdump/tordb.nf \ 'src tor' # Find traffic to internal network from Tor nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -T /var/lib/nfdump/tordb.nf \ 'src tor and dst net 10.0.0.0/8' # Output with Tor indicator nfdump -r /var/log/netflow/router1/nfcapd.202501011200 \ -T /var/lib/nfdump/tordb.nf \ -o "fmt:%ts %sa:%sp [%stor] -> %da:%dp %byt %pkt" ``` -------------------------------- ### Configure Cisco Router NetFlow Export Source: https://github.com/phaag/nfdump/blob/master/README.md Configures the export of NetFlow data from a Cisco router to a specified destination and port. It also sets the NetFlow version and active timeout for the flow cache. ```cisco-ios ip flow-export 192.168.92.218 9995 ip flow-export version 5 ip flow-cache timeout active 5 ``` -------------------------------- ### Manage Flow Data Expiration with nfexpire Source: https://context7.com/phaag/nfdump/llms.txt Automatically expire old NetFlow data based on age or disk usage using nfexpire. This tool helps manage storage by removing flow files older than a specified number of days or by ensuring the total storage does not exceed a defined limit. It operates on a specified directory of flow files. ```bash # Expire files older than 30 days nfexpire -l /var/log/netflow/router1 -e 30d # Expire to maintain maximum 100GB storage nfexpire -l /var/log/netflow/router1 -s 100G ``` -------------------------------- ### Anonymize IP Addresses in Flow Files with nfanon Source: https://context7.com/phaag/nfdump/llms.txt Anonymize IP addresses in NetFlow files using the nfanon tool with a secret key for privacy compliance. This tool can process individual files or entire directories, preserving the network topology while obscuring actual IP addresses. Verification can be done by examining the output. ```bash # Anonymize flow file with CryptoPAn nfanon -K mySecretKey123 \ -r /var/log/netflow/router1/nfcapd.202501011200 \ -w /var/log/netflow/anon/nfcapd.202501011200 # Anonymize directory of files for file in /var/log/netflow/router1/nfcapd.*; do nfanon -K mySecretKey123 -r "$file" -w "/var/log/netflow/anon/$(basename $file)" done # Verify anonymization nfdump -r /var/log/netflow/anon/nfcapd.202501011200 | head # Expected: IP addresses are anonymized but topology preserved # Original: 192.0.2.1 -> 198.51.100.1 # Anonymized: 10.123.45.67 -> 172.89.12.34 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.