### Fio Example Command for Performance Testing Source: https://github.com/axboe/fio/blob/master/HOWTO.rst This command demonstrates a typical fio setup for testing read and write performance with specific I/O depths, sizes, rates, and block sizes. It's useful for generating the example output shown in the documentation. ```bash TZ=UTC fio --ioengine=null --iodepth=2 --size=100M --numjobs=2 \ --rate_process=poisson --io_limit=32M --name=read --bs=128k \ --rate=11M --name=write --rw=write --bs=2k --rate=700k ``` -------------------------------- ### Start Fio Server (All Interfaces, Custom Port) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on all available network interfaces on a specified port. ```bash fio --server=,4444 ``` -------------------------------- ### Start Fio Server (IPv6) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on IPv6 localhost and a specified port. ```bash fio --server=ip6:::1,4444 ``` -------------------------------- ### Start Fio Server Backend Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts the fio server backend. This is typically done on the machine that will be running the workload. ```bash server# fio --server ``` -------------------------------- ### Start Fio Server (Default) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on all interfaces on the default port (8765). ```bash fio --server ``` -------------------------------- ### Start Fio Server (Specific IP, Default Port) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on a specific IP address on the default port. ```bash fio --server=1.2.3.4 ``` -------------------------------- ### Start Fio Server (Unix Domain Socket) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on a local Unix domain socket. ```bash fio --server=sock:/tmp/fio.sock ``` -------------------------------- ### Command Line Equivalent for Basic Job File Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Shows the equivalent command-line arguments for the basic job file example. ```bash fio --name=global --rw=randread --size=128m --name=job1 --name=job2 ``` -------------------------------- ### Fio Block Size Configuration Examples Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Examples demonstrating how to specify block sizes for reads, writes, and trims using the 'bs' option. A single value applies to all, while comma-separated values allow per-type specification. ```shell bs=256k ``` ```shell bs=8k,32k ``` ```shell bs=8k,32k, ``` ```shell bs=,8k ``` ```shell bs=,8k, ``` -------------------------------- ### Start Fio Server (Specific IP and Port) Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Starts a Fio server listening on a specific IP address associated with a hostname and a custom port. ```bash fio --server=ip:hostname,4444 ``` -------------------------------- ### Fio Command for Generating Example Output Source: https://github.com/axboe/fio/blob/master/HOWTO.rst This command is used to generate example output for Fio, demonstrating various I/O operations and configurations. ```bash TZ=UTC fio --iodepth=8 --ioengine=null --size=100M --runtime=58m \ --time_based --rate=2512k --bs=256K --numjobs=10 \ --name=readers --rw=read --name=writers --rw=write ``` -------------------------------- ### Start Fio Server Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Starts a fio server listening on specified arguments. Arguments define the listening type (TCP/IP v4, v6, or Unix domain socket), hostname/IP, and port. ```bash $ fio --server=args ``` ```bash fio --server ``` ```bash fio --server=ip:hostname,4444 ``` ```bash fio --server=ip6:::1,4444 ``` ```bash fio --server=,4444 ``` ```bash fio --server=1.2.3.4 ``` ```bash fio --server=sock:/tmp/fio.sock ``` -------------------------------- ### Running fio with Command-Line Options Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Alternative to job files for specifying parameters directly on the command line. Each --name starts a new job. ```bash $ fio --name=myjob --iodepth=2 --filename=mydata.bin --rw=randrw ``` -------------------------------- ### Fio Client/Server Mode Setup Source: https://context7.com/axboe/fio/llms.txt Configure and run fio in client/server mode to distribute benchmarks across multiple machines. The server listens for commands, and the client directs the tests. ```bash # On the storage server (192.168.1.100): fio --server # On the client machine: fio --client=192.168.1.100 remote-test.fio # With multiple servers from host file: echo "192.168.1.100" > hosts.txt echo "192.168.1.101" >> hosts.txt fio --client=hosts.txt distributed-test.fio # Output aggregates results from all servers: # Run status group 0 (all jobs): # READ:bw=4.5GiB/s, iops=1.2M, run=60001-60002msec ``` -------------------------------- ### Fio Block Size Range Configuration Example Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Example of configuring a range of block sizes for I/O units using the 'bsrange' option. The delimiter can be a comma or a colon. ```shell bsrange=1k-4k,2k-8k ``` -------------------------------- ### Linux Disk Statistics Output Example Source: https://github.com/axboe/fio/blob/master/HOWTO.rst This example displays Linux-specific disk statistics as reported by fio, including I/O counts, sectors transferred, merge counts, disk busy ticks, time spent in queue, and overall disk utilization. ```text Disk stats (read/write): sda: ios=16398/16511, sectors=32321/65472, merge=30/162, ticks=6853/819634, in_queue=826487, util=100.00% ``` -------------------------------- ### Fio Buffer Pattern Examples Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Examples of how to specify buffer patterns in fio. Patterns can be strings, integers, or hexadecimal values, and can be combined. ```shell buffer_pattern='filename' ``` ```shell buffer_pattern="abcd" ``` ```shell buffer_pattern=-12 ``` ```shell buffer_pattern=0xdeadface ``` ```shell buffer_pattern=0xdeadface"abcd"-12'filename' ``` -------------------------------- ### Mount Cgroup Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Example command to mount a cgroup blkio filesystem. This is a prerequisite for using the `cgroup` option in Fio. ```bash # mount -t cgroup -o blkio none /cgroup ``` -------------------------------- ### Fio Group Statistics Output Example Source: https://github.com/axboe/fio/blob/master/HOWTO.rst This example shows the aggregated performance statistics for a group of fio jobs, including read and write bandwidth, total I/O performed, and the runtime range of the jobs within the group. ```text Run status group 0 (all jobs): READ: bw=20.9MiB/s (21.9MB/s), 10.4MiB/s-10.8MiB/s (10.9MB/s-11.3MB/s), io=64.0MiB (67.1MB), run=2973-3069msec WRITE: bw=1231KiB/s (1261kB/s), 616KiB/s-621KiB/s (630kB/s-636kB/s), io=64.0MiB (67.1MB), run=52747-53223msec ``` -------------------------------- ### Building Fio on 64-bit Windows with Cygwin Source: https://github.com/axboe/fio/blob/master/README.rst Steps to compile fio on 64-bit Windows using Cygwin. Ensure necessary mingw64 packages, including mingw64-x86_64-zlib for log compression, are installed. ```bash make clean && make -j ``` -------------------------------- ### Building Fio for 32-bit Windows Source: https://github.com/axboe/fio/blob/master/README.rst To build fio for 32-bit Windows, install the i686 versions of Cygwin packages and run configure with the --build-32bit-win flag before compiling. ```bash ./configure --build-32bit-win ``` -------------------------------- ### TAP Output Format Example Source: https://github.com/axboe/fio/blob/master/mock-tests/README.md Illustrates the Test Anything Protocol (TAP) output format, showing test success (ok) and failure (not ok) with descriptions. ```text TAP version 13 1..12 ok 1 - Microsecond latency: 123456000 == 123456000 ok 2 - Millisecond latency: 1234567890000 == 1234567890000 not ok 3 - Some failing test # All tests passed ``` -------------------------------- ### Fio Job File: Poisson Request Flow Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Example fio job file demonstrating how to simulate a Poisson process for request submission to achieve a fixed submission rate, regardless of device completion rates. ```ini [poisson-rate-submit] size=128m rw=randread ioengine=libaio iodepth=32 direct=1 # by setting the submit mode to offload, we can guarantee a fixed rate of # submission regardless of what the device completion rate is. io_submit_mode=offload rate_iops=50 # Real world random request flow follows Poisson process. To give better # insight on latency distribution, we simulate request flow under Poisson # process. rate_process=poisson ``` -------------------------------- ### Zipf Distribution Example Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Use the zipf distribution with a theta of 1.2. This skews random I/O to favor certain data blocks. ```shell random_distribution=zipf:1.2 ``` -------------------------------- ### Fio Keyword Arithmetic Example Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Demonstrates using arithmetic expressions with Fio's reserved keywords. Keywords are automatically substituted with current system values. ```default size=8*$mb_memory ``` -------------------------------- ### Command Line Invocation with Environment Variables Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Example of how to set environment variables before running a fio job file. ```bash $ SIZE=64m NUMJOBS=4 fio jobfile.fio ``` -------------------------------- ### Create a New Mock Test File Source: https://github.com/axboe/fio/blob/master/mock-tests/README.md Example C code for a new mock test file using the TAP library. It initializes TAP, declares the number of tests, asserts conditions, and returns the test status. ```c #include "../lib/tap.h" int main(void) { tap_init(); tap_plan(3); // Number of tests tap_ok(1 == 1, "Basic equality"); tap_ok(2 + 2 == 4, "Addition works"); tap_skip("Not implemented yet"); return tap_done(); } ``` -------------------------------- ### Job File for Sequential Read/Write Source: https://context7.com/axboe/fio/llms.txt Create a job file to run multiple phases with different workload characteristics using INI-style configuration. This example defines sequential read and write phases. ```ini ; seq-rw-test.fio - Sequential read/write benchmark [global] ioengine=libaio direct=1 bs=1m size=10G runtime=60 time_based group_reporting ; Sequential read phase [seq-read] rw=read iodepth=16 numjobs=4 ; Sequential write phase (runs after read completes) [seq-write] stonewall rw=write iodepth=16 numjobs=4 ``` ```bash # Run the job file fio seq-rw-test.fio # Output shows results for each job section: # seq-read: (groupid=0, jobs=4): err= 0 # read: IOPS=3850, BW=3850MiB/s ... # seq-write: (groupid=1, jobs=4): err= 0 # write: IOPS=2200, BW=2200MiB/s ... ``` -------------------------------- ### Read 4 files with aio at different depths Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Example of reading from multiple files concurrently using 'libaio' with varying I/O depths for each file. ```ini ; Read 4 files with aio at different depths [global] ioengine=libaio buffered=0 rw=randread bs=128k size=512m directory=/data1 [file1] iodepth=4 [file2] iodepth=32 [file3] iodepth=8 [file4] iodepth=16 ``` -------------------------------- ### Fio Command for IOPS Rate Example Source: https://github.com/axboe/fio/blob/master/HOWTO.rst This command demonstrates setting a specific IOPS rate for Fio operations, useful for performance testing. ```bash TZ=UTC fio --iodepth=16 --ioengine=posixaio --filename=/tmp/fiofile \ --direct=1 --size=100M --time_based --runtime=50s --rate_iops=89 \ --bs=7K --name=Client1 --rw=write ``` -------------------------------- ### Integrate Mock Tests into CI Source: https://github.com/axboe/fio/blob/master/mock-tests/README.md Shows how to integrate FIO mock tests into a CI script. The first example simply checks for test success, while the second uses 'prove -v' for more detailed TAP output parsing. ```bash # In CI script make mock-tests || exit 1 ``` ```bash prove -v mock-tests/build/* ``` -------------------------------- ### Fio JSON+ Output Example (Completion Latency Bins) Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md The JSON+ output format is identical to JSON output but includes a full dump of completion latency bins. Each bin shows the count of I/Os within a specific latency duration. ```json { "fio_version": 3, "jobname": "test", "groupid": 1, "error": 0, "read": { "kb": 1024, "bw": 10240, "iops": 2560, "runtime": 100, "slat": { "min": 100, "max": 1000, "mean": 500, "dev": 100 }, "clat": { "min": 1000, "max": 5000, "mean": 3000, "dev": 500 }, "tlat": { "min": 1100, "max": 6000, "mean": 3500, "dev": 600 }, "bw_min": 9000, "bw_max": 11000, "bw_agg_pct": 95, "bw_mean": 10240, "bw_dev": 500 }, "write": { "kb": 2048, "bw": 20480, "iops": 5120, "runtime": 200, "slat": { "min": 150, "max": 1200, "mean": 600, "dev": 150 }, "clat": { "min": 1500, "max": 6000, "mean": 4000, "dev": 700 }, "tlat": { "min": 1650, "max": 7200, "mean": 4600, "dev": 850 }, "bw_min": 18000, "bw_max": 23000, "bw_agg_pct": 98, "bw_mean": 20480, "bw_dev": 1000 }, "cpu": { "user": 10.5, "sys": 5.2, "csw": 100, "mjf": 0, "minf": 0 }, "iodepth": { "1": 10, "2": 20, "4": 30, "8": 40, "16": 50, "32": 60, "64": 70 }, "bins": { "87552": 1, "89600": 1, "94720": 1, "96768": 1, "97792": 1, "99840": 1, "100864": 2, "103936": 6, "104960": 534, "105984": 5995, "107008": 7529 }, "disk_name": "/dev/sda", "disk_read_iops": 3000, "disk_write_iops": 6000, "disk_read_merges": 50, "disk_write_merges": 100, "disk_read_ticks": 10000, "disk_write_ticks": 20000, "disk_queue_time": 500, "disk_util": "75.5%" } ``` -------------------------------- ### Merging and Scaling Blktrace Files with Fio Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Example command to merge and scale blktrace files. Scalars are applied index-wise to the input trace files to adjust their speed. Iterations can also be specified to match runtimes. ```bash $ fio --read_iolog=":" --merge_blktrace_file"" --merge_blktrace_scalars="50:100" --merge_blktrace_iters="2:1" ``` -------------------------------- ### Offset Modifier for Random I/O in Fio Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Appends a number to a random I/O pattern to specify the number of I/Os to perform before getting a new offset. For example, `rw=randread:8` generates a new random offset for every 8 I/Os. ```default rw=randread:8 ``` -------------------------------- ### Run Mock Tests from Directory Source: https://github.com/axboe/fio/blob/master/mock-tests/README.md Navigate to the 'mock-tests' directory and use 'make test' to run all tests, 'make test-tap' for TAP harness output, or 'make test-' for a specific test. ```bash cd mock-tests make test ``` ```bash make test-tap ``` ```bash make test-latency_precision ``` -------------------------------- ### Build Fio from Source Source: https://github.com/axboe/fio/blob/master/README.rst Standard build process for Fio. Requires GNU make. Ensure you are in the Fio source directory. ```bash $ ./configure $ make $ make install ``` -------------------------------- ### Start New Reporting Group Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Use 'new_group' to explicitly start a new reporting group. If not specified, all jobs in a file belong to the same group unless separated by 'stonewall'. ```text new_group ``` -------------------------------- ### Example Fio Runtime Output Source: https://github.com/axboe/fio/blob/master/HOWTO.rst An example of the status output displayed by fio while a job is running, showing job progress, read/write rates, IOPS, and estimated time remaining. ```text Jobs: 1 (f=1): [_(1),M(1)][24.8%][r=20.5MiB/s,w=23.5MiB/s][r=82,w=94 IOPS][eta 01m:31s] ``` -------------------------------- ### Example Terse Output (v2) Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md This is an example of fio's terse output format, version 2. It provides a single line of semicolon-separated values for scripted analysis. The job description, if provided, follows on a second line. ```default 2;card0;0;0;7139336;121836;60004;1;10109;27.932460;116.933948;220;126861;3495.446807;1085.368601;226;126864;3523.635629;1089.012448;24063;99944;50.275485%;59818.274627;5540.657370;7155060;122104;60004;1;8338;29.086342;117.839068;388;128077;5032.488518;1234.785715;391;128085;5061.839412;1236.909129;23436;100928;50.287926%;59964.832030;5644.844189;14.595833%;19.394167%;123706;0;7313;0.1%;0.1%;0.1%;0.1%;0.1%;0.1%;100.0%;0.00%;0.00%;0.00%;0.00%;0.00%;0.00%;0.01%;0.02%;0.05%;0.16%;6.04%;40.40%;52.68%;0.64%;0.01%;0.00%;0.01%;0.00%;0.00%;0.00%;0.00%;0.00% ``` -------------------------------- ### Build fio for ESX Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Enable building fio for ESX environments using the --esx switch with configure. ```bash ./configure --esx ``` -------------------------------- ### Libblkio Pre-Start Properties Configuration Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Configure additional libblkio properties to be set before starting. ```APIDOC ## libblkio_pre_start_props=str : [libblkio] A colon-separated list of additional libblkio properties to be set after connecting but before starting the libblkio instance. Each property must have the format `=`. Colons can be escaped as `\:`. These are set after the engine sets any other properties, so those can be overridden. Available properties depend on the libblkio version in use and are listed at [https://libblkio.gitlab.io/libblkio/blkio.html#properties](https://libblkio.gitlab.io/libblkio/blkio.html#properties) ``` -------------------------------- ### Merging Blktrace Files Only Example Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Command to create only the merged blktrace file without further processing. ```bash $ fio --merge-blktrace-only ``` -------------------------------- ### Run All Mock Tests with Make Source: https://github.com/axboe/fio/blob/master/mock-tests/README.md Execute all available mock tests using the 'make mock-tests' command. This is the primary way to run the entire suite. ```bash make mock-tests ``` -------------------------------- ### View available documentation formats Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md List all supported output formats for fio documentation generation. ```bash make -C doc help ``` -------------------------------- ### Fio Final Job Statistics Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md This is an example of the detailed statistics Fio displays for a job after it completes or is interrupted. It includes IOPS, bandwidth, latency, and more. ```text Client1: (groupid=0, jobs=1): err= 0: pid=16109: Sat Jun 24 12:07:54 2017 write: IOPS=88, BW=623KiB/s (638kB/s)(30.4MiB/50032msec) slat (nsec): min=500, max=145500, avg=8318.00, stdev=4781.50 clat (usec): min=170, max=78367, avg=4019.02, stdev=8293.31 lat (usec): min=174, max=78375, avg=4027.34, stdev=8291.79 clat percentiles (usec): | 1.00th=[ 302], 5.00th=[ 326], 10.00th=[ 343], 20.00th=[ 363], | 30.00th=[ 392], 40.00th=[ 404], 50.00th=[ 416], 60.00th=[ 445], | 70.00th=[ 816], 80.00th=[ 6718], 90.00th=[12911], 95.00th=[21627], | 99.00th=[43779], 99.50th=[51643], 99.90th=[68682], 99.95th=[72877], | 99.99th=[78119] bw ( KiB/s): min= 532, max= 686, per=0.10%, avg=622.87, stdev=24.82, samples= 100 iops : min= 76, max= 98, avg=88.98, stdev= 3.54, samples= 100 lat (usec) : 250=0.04%, 500=64.11%, 750=4.81%, 1000=2.79% ``` -------------------------------- ### Enable ACT Prep Phase Source: https://github.com/axboe/fio/blob/master/HOWTO.rst When set, this option runs the preparation phase of the ACT workload. ```bash prep ``` -------------------------------- ### io_uring Benchmark Configuration Source: https://context7.com/axboe/fio/llms.txt Configure fio to use the io_uring engine for high-performance asynchronous I/O. This setup is optimized for modern Linux systems. ```ini ; io_uring-test.fio - High-performance io_uring benchmark [global] ioengine=io_uring direct=1 bs=4k size=8G runtime=60 time_based group_reporting [io_uring-randread] rw=randread iodepth=128 numjobs=4 hipri=1 fixedbufs=1 registerfiles=1 sqthread_poll=1 filename=/dev/nvme0n1 ``` -------------------------------- ### Running Fio Source: https://github.com/axboe/fio/blob/master/README.rst Basic command to run fio. Provide job files and options as parameters. Multiple job files will be serialized. ```bash $ fio [options] [jobfile] ... ``` -------------------------------- ### Terse Output Format - Disk Utilization Section Example Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Indicates that for disk utilization, all disks used by fio will have a corresponding section in the terse output. ```default ``` -------------------------------- ### Steady-State Detection Configuration Source: https://context7.com/axboe/fio/llms.txt Configure fio to automatically detect when performance stabilizes (steady state) for accurate measurements. This example uses IOPS variance. ```ini ; steady-state.fio - Run until steady state is reached [global] ioengine=libaio direct=1 rw=randwrite bs=4k size=100G time_based runtime=3600 iodepth=32 numjobs=4 filename=/dev/nvme0n1 group_reporting ; Stop when IOPS variance drops below 5% steadystate=iops:5% steadystate_duration=60s steadystate_ramp_time=300s ``` -------------------------------- ### I/O Advice Hinting Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Utilizes `posix_fadvise(2)` to provide hints to the kernel about expected I/O patterns. ```APIDOC ## I/O Advice Hinting ### Description Uses `posix_fadvise(2)` to inform the kernel about anticipated I/O patterns, potentially optimizing performance. ### Option - **fadvise_hint=str** Specifies the type of advice to give the kernel. Accepted values: - **0**: No hint provided (backwards-compatible). - **1**: Advise based on fio workload type (backwards-compatible). Uses `FADV_RANDOM` for random workloads and `FADV_SEQUENTIAL` for sequential workloads. - **sequential**: Advise using `FADV_SEQUENTIAL`. - **random**: Advise using `FADV_RANDOM`. - **noreuse**: (Description incomplete in source text) ``` -------------------------------- ### Generating Fio Documentation Source: https://github.com/axboe/fio/blob/master/README.rst Commands to build Fio's documentation in HTML or man page format using Sphinx. Use 'make -C doc help' to see all supported output formats. ```bash make -C doc html ``` ```bash make -C doc man ``` ```bash make -C doc help ``` -------------------------------- ### Fio Client Connecting to Multiple Servers Source: https://github.com/axboe/fio/blob/master/HOWTO.rst Demonstrates how a Fio client can connect to and manage workloads on multiple Fio servers simultaneously. ```bash fio --client= --client= ``` -------------------------------- ### Build man page documentation Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Generate the manual page for fio. The man page will be located at doc/output/man/fio.1. ```bash make -C doc man ``` -------------------------------- ### Wait for Preceding Jobs Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Use `stonewall` or `wait_for_previous` to make the current job wait for all preceding jobs in the job file to exit before starting. This creates serialization points. ```text stonewall ``` ```text wait_for_previous ``` -------------------------------- ### Invalidate Cache Before I/O Source: https://github.com/axboe/fio/blob/master/doc/fio_doc.md Invalidates buffer/page cache parts of files before starting I/O, if supported by the platform and file type. Defaults to true. Ignored if pre_read is specified. ```default invalidate=bool ```