### Build and Install Libtpa Source: https://github.com/bytedance/libtpa/blob/main/doc/quick_start.rst These commands compile the libtpa library and then install it to the system. The 'make' command builds the project, and 'make install' places the compiled artifacts in the appropriate system directories. ```make make make install ``` -------------------------------- ### Install Dependencies for Debian Source: https://github.com/bytedance/libtpa/blob/main/doc/quick_start.rst This command installs the necessary dependencies for building libtpa on a Debian-based system. It assumes you are in the libtpa source directory. The `--with-meson` flag is used to enable meson build system support. ```text ./buildtools/install-dep.deb.sh --with-meson ``` -------------------------------- ### Techo Application Example Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Demonstrates how to run the 'techo' utility, a simple echo server for testing libtpa TCP connections. The example shows how to specify the listening port. ```text # techo 5678 EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'PA' EAL: Probe PCI driver: mlx5_pci (15b3:1018) device: 0000:00:05.0 (socket -1) mlx5_net: Default miss action is not supported. :: listening on port 5678 ... ``` -------------------------------- ### Listen and Accept Connection Example Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst This example demonstrates a basic listen and accept loop using libtpa. It first calls tpa_listen_on to start listening on a specified port, then enters a loop where it runs the worker, accepts new connections using tpa_accept_burst, and registers them. ```c int sid; if (tpa_listen_on(NULL, 80, NULL) < 0) { fprintf(stderr, "failed to listen on port 80: %s\n", strerror(errno)); return -1; } while (1) { /* explained later */ tpa_worker_run(worker); if (tpa_accept_burst(worker, &sid, 1) == 1) register_new_connection(sid); /* ... */ } ``` -------------------------------- ### Build and Install Libtpa Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Commands to build and install the Libtpa library. It also explains how Libtpa manages DPDK builds and how to perform a clean build or switch DPDK versions. ```shell make make install ``` ```shell make distclean make ``` -------------------------------- ### Install Mellanox OFED Dependencies (Debian) Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Installs necessary dependencies for Libtpa on Debian-based systems. It can also be run with an option to include a specific Meson version if needed. ```shell ./buildtools/install-dep.deb.sh ``` ```shell ./buildtools/install-dep.deb.sh --with-meson ``` -------------------------------- ### Libtpa Configuration Example Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst A configuration file snippet for a Libtpa application, showing network interface settings including name and IP address. ```text net { name = eth0 ip = 192.168.1.10 ``` -------------------------------- ### Run Swing Application with Libtpa Configuration Source: https://github.com/bytedance/libtpa/blob/main/doc/quick_start.rst This command demonstrates how to run the 'swing' application, a telnet-like tool, using libtpa. It includes a sample configuration string for network and DPDK settings and specifies the target IP address and port for the connection. ```text # tpa run swing 192.168.1.12 22 :: TPA_CFG='net { name=eth0; mac=fa:16:3e:30:4f:90; ip=192.168.1.10; mask=255.255.255.0; \ gw=192.168.1.1; ip6=fe80::f816:3eff:fe30:4f90/64; } dpdk { pci=0000:00:05.0; } ' :: cmd=swing 192.168.1.12 22 EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'PA' EAL: Probe PCI driver: mlx5_pci (15b3:1018) device: 0000:00:05.0 (socket -1) mlx5_net: Default miss action is not supported. :: connecting to 192.168.1.12:22 ... [connected] > < SSH-2.0-OpenSSH_9.0 ``` -------------------------------- ### Listening and Accepting Connections Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst APIs for setting up a listener and accepting incoming connections. ```APIDOC ## POST /bytedance/libtpa/listen ### Description Sets up a listener on a specified port. This function is used to initiate the process of accepting incoming network connections. ### Method POST ### Endpoint /bytedance/libtpa/listen ### Parameters #### Query Parameters - **port** (int) - Required - The port number to listen on. - **opts** (void*) - Optional - Options for the listener. ### Request Body None ### Response #### Success Response (200) - **return_value** (int) - Returns 0 on success, negative on error. #### Response Example ```json { "return_value": 0 } ``` ## POST /bytedance/libtpa/accept ### Description Accepts newly incoming connections and assigns them to a worker. This function retrieves a list of accepted sockets for the given worker. ### Method POST ### Endpoint /bytedance/libtpa/accept ### Parameters #### Path Parameters - **worker** (struct tpa_worker*) - Required - The worker to assign the accepted sockets to. #### Query Parameters - **sid** (int*) - Output - Pointer to an integer that will store the socket descriptor of the accepted connection. - **nr_sid** (int) - Required - The number of socket descriptors to retrieve. ### Request Body None ### Response #### Success Response (200) - **return_value** (int) - Returns the number of accepted sockets. #### Response Example ```json { "return_value": 1 } ``` ## GET /bytedance/libtpa/sock_info/{sid} ### Description Retrieves detailed information about a specific socket. ### Method GET ### Endpoint /bytedance/libtpa/sock_info/{sid} ### Parameters #### Path Parameters - **sid** (int) - Required - The socket descriptor for which to retrieve information. #### Query Parameters - **info** (struct tpa_sock_info*) - Output - Pointer to a structure that will be filled with socket information. ### Request Body None ### Response #### Success Response (200) - **return_value** (int) - Returns 0 on success, negative on error. #### Response Example ```json { "return_value": 0 } ``` ## DELETE /bytedance/libtpa/close/{sid} ### Description Closes the connection identified by the given socket descriptor. ### Method DELETE ### Endpoint /bytedance/libtpa/close/{sid} ### Parameters #### Path Parameters - **sid** (int) - Required - The socket descriptor to close. ### Request Body None ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### External Memory Registration Example (C) Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst A C example demonstrating how to register an external memory buffer using tpa_extmem_register. This is necessary for performing zero-copy writes with buffers residing in the registered memory region. Error handling for the registration process is included. ```c void *buf = aligned_alloc(4096, 4096); if (tpa_extmem_register(buf, 4096, NULL, 1, 4096) != 0) { fprintf(stderr, "failed to register external memory: %s\n", strerror(errno)); return; } ``` -------------------------------- ### Run Tperf Server for Loopback Mode Test Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Starts a tperf server instance using libtpa for loopback mode testing. It assigns a specific TPA_ID and CPU core for the server process. ```text # TPA_ID=server taskset -c 1 tperf -s -n 1 EAL: Detected 96 lcore(s) EAL: Detected 2 NUMA nodes EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:5e:00.1 (socket 0) mlx5_pci: Default miss action is not supported. ``` -------------------------------- ### Complete Echo Server Example with libtpa Source: https://context7.com/bytedance/libtpa/llms.txt This example demonstrates a full echo server implementation using libtpa. It includes network initialization, listening on a port, accepting connections, handling incoming data with zero-copy reads (tpa_zreadv), echoing data back (tpa_zwritev), and managing connection lifecycles. It utilizes libtpa's event polling mechanism for efficient I/O handling. ```c #include #include #include #include #include struct connection { int sid; }; static struct tpa_worker *worker; void close_connection(struct connection *conn) { tpa_close(conn->sid); free(conn); } void echo_handler(struct connection *conn) { struct tpa_iovec iov; ssize_t ret; ret = tpa_zreadv(conn->sid, &iov, 1); if (ret <= 0) { if (ret < 0 && errno == EAGAIN) return; close_connection(conn); return; } // Echo back: reuse read buffer for write if (tpa_zwritev(conn->sid, &iov, 1) != iov.iov_len) { iov.iov_read_done(iov.iov_base, iov.iov_param); close_connection(conn); return; } } void register_connection(int sid) { struct connection *conn = malloc(sizeof(struct connection)); struct tpa_event event; conn->sid = sid; event.events = TPA_EVENT_IN; event.data = conn; tpa_event_ctrl(sid, TPA_EVENT_CTRL_ADD, &event); } void poll_connections(void) { struct tpa_event events[32]; struct connection *conn; int nr_events, i; nr_events = tpa_event_poll(worker, events, 32); for (i = 0; i < nr_events; i++) { conn = events[i].data; if (events[i].events & (TPA_EVENT_IN | TPA_EVENT_ERR | TPA_EVENT_HUP)) echo_handler(conn); } } int main(int argc, char **argv) { uint16_t port = 5678; int sid; if (tpa_init(1) < 0) { perror("tpa_init"); return -1; } worker = tpa_worker_init(); if (!worker) { fprintf(stderr, "worker init failed: %s\n", strerror(errno)); return -1; } if (tpa_listen_on(NULL, port, NULL) < 0) { fprintf(stderr, "listen failed: %s\n", strerror(errno)); return -1; } printf("Echo server listening on port %u\n", port); while (1) { tpa_worker_run(worker); if (tpa_accept_burst(worker, &sid, 1) == 1) register_connection(sid); poll_connections(); } return 0; } ``` -------------------------------- ### Start Redis with Libtpa and Benchmark (Sock Trace Enabled) Source: https://github.com/bytedance/libtpa/blob/main/doc/redis.rst This command starts the Redis server accelerated by libtpa and then runs a benchmark test using redis-benchmark. The benchmark specifically tests the GET operation with multiple threads and clients. Sock trace is enabled by default. ```text taskset -c 1 tpa run ./redis/src/redis-server --protected-mode no # taskset -c 1-20 redis-benchmark -h 192.168.1.10 --threads 20 -c 100 -t get -n 10000000 ``` -------------------------------- ### tpa_zwritev Example with Zero Copy Disabled (C) Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst Demonstrates the tpa_zwritev function in C, specifically how setting iov_phys to 0 disables zero-copy writes and forces a fallback to the non-zero-copy version. Includes a callback function for freeing the write buffer. ```c static void free_write_buffer(void *iov_base, void *iov_param) { free(iov_base); } ssize_t tpa_zwrite_example(size_t size) { struct tpa_iovec iov; ssize_t ret; iov.iov_len = size; iov.iov_base = malloc(size); iov.iov_phys = 0; iov.iov_param = NULL; iov.iov_write_done = free_write_buffer; ret = tpa_zwritev(sid, &iov, 1); if (ret < 0) iov.iov_write_done(iov.iov_base, iov.iov_param); return ret; } ``` -------------------------------- ### tpa_zreadv Example for Data Processing Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst This example shows how to use tpa_zreadv to read data, process it, and then reclaim the buffer using the iov_read_done callback. It handles potential errors, EAGAIN for non-blocking operations, and EOF. ```c struct tpa_iovec iov; ssize_t ret; ret = tpa_zreadv(sid, &iov, 1); if (ret < 0) { if (errno == EAGAIN) { return 0; /* error happened; handle it here */ } if (ret == 0) { /* EOF reached; close it */ tpa_close(sid); } if (ret > 0) { /* process the read buffer at iov.iov_base */ process_data(iov.iov_base, iov.iov_len); /* free it when the process is done */ iov.iov_read_done(iov.iov_base, iov.iov_param); } ``` -------------------------------- ### Dump Detailed Version Information with tpa -vv Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst The 'tpa -vv' command displays detailed version information for both the installed and currently running versions of the tpa application. This includes the TPA ID, PID, program name, version, and uptime for client and server components. ```bash # tpa -vv installed: v1.0-rc0 running: -------- TPA_ID pid program version uptime client 2517834 tperf v1.0-rc0 2023-12-04 15:20:15, up 21s server 2517867 tperf v1.0-rc0 2023-12-04 15:20:28, up 9s ``` -------------------------------- ### Connection Management API Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst APIs for managing TCP connections, including initiating active connections and setting up listening sockets. ```APIDOC ## tpa_connect_to ### Description Starts a new active TCP connection to a specified server and port. ### Method `tpa_connect_to` ### Endpoint N/A (Function call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters for `tpa_connect_to` - **server** (const char *) - The server address (IPv4 or IPv6 string). - **port** (uint16_t) - The remote port to connect to. - **opts** (const struct tpa_sock_opts *) - Optional. Additional options for the connection. Can be NULL. The `local_port` option is relevant here. ### Request Example ```c int sid; struct tpa_sock_opts opts = { .local_port = 12345 }; sid = tpa_connect_to("::1", 80, &opts); if (sid < 0) { fprintf(stderr, "failed to connect: %s\n", strerror(errno)); return -1; } ``` ### Response #### Success Response (200) Returns a non-negative sock id (sid) on success. #### Response Example ```c // On success, sid will be >= 0 int sid = 1; ``` ### Error Handling Returns a negative value on error. --- ## tpa_listen_on ### Description Creates a socket that listens on the specified local address and port for incoming connections. ### Method `tpa_listen_on` ### Endpoint N/A (Function call) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters for `tpa_listen_on` - **local** (const char *) - The local address to listen on. Can be NULL for dual stack, or a specific IPv4/IPv6 address. - **port** (uint16_t) - The local port to listen on. - **opts** (const struct tpa_sock_opts *) - Optional. Additional options for the listen socket. The `listen_scaling` and `data` options are relevant here. ### Request Example ```c struct tpa_sock_opts opts = { .listen_scaling = 1 }; int listen_sid = tpa_listen_on(NULL, 8080, &opts); if (listen_sid < 0) { fprintf(stderr, "failed to listen: %s\n", strerror(errno)); return -1; } ``` ### Response #### Success Response (200) Returns a non-negative sock id (sid) on success. #### Response Example ```c // On success, listen_sid will be >= 0 int listen_sid = 2; ``` ### Error Handling Returns a negative value on error. ### `tpa_sock_opts` Structure ```c struct tpa_sock_opts { uint64_t listen_scaling:1; // For tpa_listen_on only uint64_t bits_reserved:63; void *data; // For tpa_listen_on only uint16_t local_port; // For tpa_connect_to only uint8_t reserved[128 - 18]; } __attribute__((packed)); ``` **Note**: By default, only the worker thread that calls `tpa_listen_on` will receive new passive connections. `listen_scaling = 1` distributes new connections to all workers. ``` -------------------------------- ### Build Redis with Libtpa Source: https://github.com/bytedance/libtpa/blob/main/doc/redis.rst This script builds the Redis server with libtpa acceleration. Ensure libtpa is installed before executing this script. It navigates to the redis demo directory and runs the build script. ```text cd demo/redis ./build.sh ``` -------------------------------- ### Switch DPDK Version for Libtpa Build Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Demonstrates how to specify a different DPDK version for building Libtpa by setting the DPDK_VERSION environment variable before running the make commands. ```shell export DPDK_VERSION=v22.11 make make install ``` -------------------------------- ### Data Read and Write Operations Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst APIs for reading data from and writing data to connections, with support for zero-copy. ```APIDOC ## POST /bytedance/libtpa/zreadv ### Description Performs a zero-copy read from a socket. Libtpa fills the iovec structure with data, and the application must invoke the `iov_read_done` callback after processing. ### Method POST ### Endpoint /bytedance/libtpa/zreadv ### Parameters #### Path Parameters - **sid** (int) - Required - The socket descriptor to read from. - **iov** (struct tpa_iovec*) - Output - A pointer to a `tpa_iovec` structure that will be filled with read data information. - **nr_iov** (int) - Required - The number of iovec structures to read. ### Request Body None ### Response #### Success Response (200) - **return_value** (ssize_t) - Returns the number of bytes read, 0 on EOF, or negative on error (e.g., EAGAIN). #### Response Example ```json { "return_value": 1024 } ``` ## POST /bytedance/libtpa/write ### Description Writes data to a socket using a standard non-zero-copy buffer. ### Method POST ### Endpoint /bytedance/libtpa/write ### Parameters #### Path Parameters - **sid** (int) - Required - The socket descriptor to write to. - **buf** (const void*) - Required - A pointer to the buffer containing the data to write. - **count** (size_t) - Required - The number of bytes to write from the buffer. ### Request Body None ### Response #### Success Response (200) - **return_value** (ssize_t) - Returns the number of bytes written, or negative on error. #### Response Example ```json { "return_value": 1024 } ``` ## POST /bytedance/libtpa/zwritev ### Description Performs a zero-copy write to a socket. The application must fill in `iov_phys`, `iov_write_done`, and `iov_param` fields of the `tpa_iovec` structure. The `iov_write_done` callback is invoked when the data transfer is complete. ### Method POST ### Endpoint /bytedance/libtpa/zwritev ### Parameters #### Path Parameters - **sid** (int) - Required - The socket descriptor to write to. - **iov** (const struct tpa_iovec*) - Required - An array of `tpa_iovec` structures containing the data to write. - **nr_iov** (int) - Required - The number of `tpa_iovec` structures in the array. ### Request Body None ### Response #### Success Response (200) - **return_value** (ssize_t) - Returns the number of bytes written, or negative on error. #### Response Example ```json { "return_value": 1024 } ``` ``` -------------------------------- ### Run Tperf Client for Loopback Mode Test Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Initiates a tperf client instance using libtpa to test loopback mode. It connects to a specified server address and port, using a dedicated TPA_ID and CPU core. The output shows performance statistics. ```text # TPA_ID=client taskset -c 2 tperf -c 127.0.0.1 -t rr -m 1 EAL: Detected 96 lcore(s) EAL: Detected 2 NUMA nodes EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:5e:00.1 (socket 0) mlx5_pci: Default miss action is not supported. 0 RR .0 min=4.10us avg=4.43us max=96.51us count=224809 1 RR .0 min=4.10us avg=4.38us max=79.36us count=228426 2 RR .0 min=4.10us avg=4.36us max=84.22us count=229371 3 RR .0 min=4.10us avg=4.36us max=135.17us count=229385 4 RR .0 min=4.10us avg=4.36us max=81.41us count=229366 5 RR .0 min=4.10us avg=4.36us max=77.31us count=229459 6 RR .0 min=4.10us avg=4.36us max=78.08us count=229349 7 RR .0 min=4.10us avg=4.36us max=105.47us count=229238 8 RR .0 min=4.10us avg=4.36us max=77.82us count=229565 9 RR .0 min=4.10us avg=4.36us max=87.04us count=229363 --- 0 nr_conn=1 nr_zero_io_conn=0 ``` -------------------------------- ### Libtpa Initialization and Worker Management (C) Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst Shows C functions for initializing the libtpa system with a specified number of workers and initializing individual workers. tpa_init must be called before any other libtpa functions, and tpa_worker_init should be called once per worker thread. ```c int tpa_init(int nr_worker); struct tpa_worker *tpa_worker_init(void); void tpa_worker_run(struct tpa_worker *worker); ``` -------------------------------- ### Swing Application Usage Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Shows the command-line usage for the 'swing' utility, a debug tool similar to telnet for testing libtpa networking. It includes available options, such as enabling zero copy writes. ```text # swing -h usage: swing [options] server port Supported options are: -z enable zero copy write ``` -------------------------------- ### List All Supported Libtpa Configuration Options Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst The 'tpa cfg list' command displays all configuration options supported by libtpa. These options are organized by section (e.g., log, net, tcp, dpdk) and include their current values or status (e.g., N/A). ```bash # tpa cfg list log.level 2 log.file N/A net.ip 192.168.1.10 net.mask 255.255.255.0 net.gw 192.168.1.1 net.ip6 :: net.gw6 :: net.mac fa:16:3e:30:4f:90 net.name eth0 net.bonding N/A trace.enable 1 trace.more_trace 0 trace.trace_size 8KB trace.nr_trace 2048 trace.no_wrap 0 tcp.nr_max_sock 32768 tcp.pkt_max_chain 45 tcp.usr_snd_mss 0 tcp.time_wait 1m tcp.keepalive 2m tcp.delayed_ack 1ms tcp.tso 1 tcp.rx_merge 1 tcp.opt_ts 1 tcp.opt_ws 1 tcp.opt_sack 1 tcp.retries 7 tcp.syn_retries 7 tcp.rcv_queue_size 2048 tcp.snd_queue_size 512 tcp.cwnd_init 16384 tcp.cwnd_max 1073741824 tcp.rcv_ooo_limit 2048 tcp.drop_ooo_threshold 33792 tcp.measure_latency 0 tcp.rto_min 100ms tcp.write_chunk_size 16KB tcp.local_port_range 41000 64000 shell.postinit_cmd N/A dpdk.socket-mem 1024 dpdk.pci 0000:00:05.0 dpdk.extra_args N/A dpdk.mbuf_cache_size 512 dpdk.mbuf_mem_size 0 dpdk.numa 0 dpdk.huge-unlink 1 offload.flow_mark 1 offload.sock_offload 0 offload.port_block_offload 1 pktfuzz.enable 0 pktfuzz.log N/A archive.enable 1 archive.flush_interval 60 ``` -------------------------------- ### Tperf Application Usage Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Details the command-line options for the 'tperf' benchmark tool, which is used for performance testing within the libtpa framework. It covers both client and server modes, along with various test and operational parameters. ```text # tperf -h usage: tperf [options] tperf -s [options] tperf -t test [options] Tperf, a libtpa performance benchmark. Client options: -c server run in client mode (the default mode) and specifies the server address (default: 127.0.0.1) -t test specifies the test mode, which is listed below -p port specifies the port to connect to (default: 4096) -d duration specifies the test duration (default: 10s) -m message_size specifies the message size (default: 1000) -n nr_thread specifies the thread count (default: 1) -i do integrity verification (default: off) -C nr_conn specifies the connection to be created for each thread (default: 1) -W 0|1 disable/enable zero copy write (default: on) -S start_cpu specifies the starting cpu to bind Server options: -s run in server mode -n nr_thread specifies the thread count (default: 1) -l addr specifies local address to listen on -p port specifies the port to listen on (default: 4096) -S start_cpu specifies the starting cpu to bind The supported test modes are: * read read data from the server end * write write data to the server end * rw test read and write simultaneously * rr send a request (with payload) to the server and expects a response will be returned from the server end * crr basically does the same thing like rr, except that a ``` -------------------------------- ### Getting Detailed Socket Information with tpa_sock_info_get Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst The tpa_sock_info_get function fetches detailed information about a socket, identified by its session ID (sid). It populates a tpa_sock_info structure with details like local and remote IPs, ports, and associated worker data. ```c struct tpa_sock_info { struct tpa_worker *worker; /* it's the tpa_sock_opts.data set by user */ void *data; struct tpa_ip local_ip; struct tpa_ip remote_ip; uint16_t local_port; uint16_t remote_port; uint8_t reserved[76]; }; int tpa_sock_info_get(int sid, struct tpa_sock_info *info); ``` -------------------------------- ### Redis Benchmark GET Operation Results (Text) Source: https://github.com/bytedance/libtpa/blob/main/doc/redis.rst This snippet shows the results of a redis-benchmark command for the GET operation. It details the number of requests completed, parallel clients, payload size, and host configurations. It also includes a comprehensive percentile distribution of latencies. ```text # taskset -c 1-20 redis-benchmark -h 192.168.1.10 --threads 20 -c 100 -t get -n 10000000 ====== GET ====== 10000000 requests completed in 47.44 seconds 100 parallel clients 3 bytes payload keep alive: 1 host configuration "save": 3600 1 300 100 60 10000 host configuration "appendonly": no multi-thread: yes threads: 20 Latency by percentile distribution: 0.000% <= 0.047 milliseconds (cumulative count 1) 50.000% <= 0.399 milliseconds (cumulative count 5397185) 75.000% <= 0.519 milliseconds (cumulative count 7518625) 87.500% <= 0.607 milliseconds (cumulative count 8779787) 93.750% <= 0.751 milliseconds (cumulative count 9384364) 96.875% <= 0.783 milliseconds (cumulative count 9748247) 98.438% <= 0.799 milliseconds (cumulative count 9852298) 99.219% <= 0.823 milliseconds (cumulative count 9928968) 99.609% <= 0.847 milliseconds (cumulative count 9963131) 99.805% <= 0.887 milliseconds (cumulative count 9982340) 99.902% <= 0.959 milliseconds (cumulative count 9990268) 99.951% <= 1.079 milliseconds (cumulative count 9995194) 99.976% <= 1.199 milliseconds (cumulative count 9997680) 99.988% <= 1.327 milliseconds (cumulative count 9998830) 99.994% <= 1.487 milliseconds (cumulative count 9999405) 99.997% <= 1.655 milliseconds (cumulative count 9999702) 99.998% <= 1.855 milliseconds (cumulative count 9999849) 99.999% <= 2.135 milliseconds (cumulative count 9999924) 100.000% <= 2.303 milliseconds (cumulative count 9999962) 100.000% <= 2.511 milliseconds (cumulative count 9999981) 100.000% <= 2.607 milliseconds (cumulative count 9999991) 100.000% <= 2.639 milliseconds (cumulative count 9999996) 100.000% <= 2.647 milliseconds (cumulative count 9999998) 100.000% <= 2.663 milliseconds (cumulative count 10000000) 100.000% <= 2.663 milliseconds (cumulative count 10000000) Cumulative distribution of latencies: 0.006% <= 0.103 milliseconds (cumulative count 593) 0.155% <= 0.207 milliseconds (cumulative count 15510) 1.130% <= 0.303 milliseconds (cumulative count 112989) ``` -------------------------------- ### List Active Sockets with 'tpa sk' Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Lists all currently active network sockets managed by libtpa. This command is useful for monitoring real-time connections and their associated details. ```bash # tpa sk sid=4 192.168.1.10:55569 192.168.1.10:4096 worker=0 established sid=5 192.168.1.10:55555 192.168.1.10:4096 worker=0 established sid=6 192.168.1.10:55589 192.168.1.10:4096 worker=0 established sid=7 192.168.1.10:55609 192.168.1.10:4096 worker=0 established ``` -------------------------------- ### Connect to a server using tpa_connect_to Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst Initiates an active TCP connection to a specified server and port. Supports optional socket options for customization, such as binding to a specific local port. The function is non-blocking, and connection establishment is indicated by an OUT event. ```c int tpa_connect_to(const char *server, uint16_t port, const struct tpa_sock_opts *opts); ``` ```c int sid; sid = tpa_connect_to("::1", 80, NULL); if (sid < 0) { fprintf(stderr, "failed to connect: %s\n", strerror(errno)); return -1; } ``` -------------------------------- ### Matrix Shell Test Case Generation (Text) Source: https://github.com/bytedance/libtpa/blob/main/doc/internals.rst Example of defining test parameters and generating test cases using the matrix shell. The 'test.ms' file specifies parameters and their possible values, which are then used to generate a comprehensive list of test combinations. ```text # cat test.ms params: test: [read, write, rw] message_size: [1, 4KB, 16KB] nr_thread: [1, 4] end echo "testing with params: nr_thread=$nr_thread test=$test message_size=$message_size" ``` ```text ms-list test.ms --short | nl 1 test/test=read__message_size=1__nr_thread=1 2 test/test=read__message_size=1__nr_thread=4 3 test/test=read__message_size=4KB__nr_thread=1 4 test/test=read__message_size=4KB__nr_thread=4 5 test/test=read__message_size=16KB__nr_thread=1 6 test/test=read__message_size=16KB__nr_thread=4 7 test/test=write__message_size=1__nr_thread=1 8 test/test=write__message_size=1__nr_thread=4 9 test/test=write__message_size=4KB__nr_thread=1 10 test/test=write__message_size=4KB__nr_thread=4 11 test/test=write__message_size=16KB__nr_thread=1 12 test/test=write__message_size=16KB__nr_thread=4 13 test/test=rw__message_size=1__nr_thread=1 14 test/test=rw__message_size=1__nr_thread=4 15 test/test=rw__message_size=4KB__nr_thread=1 16 test/test=rw__message_size=4KB__nr_thread=4 17 test/test=rw__message_size=16KB__nr_thread=1 18 test/test=rw__message_size=16KB__nr_thread=4 ``` -------------------------------- ### List Detailed Socket Info with 'tpa sk -v' and grep Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Outputs very detailed information for each socket, which can be filtered using grep to find specific metrics like read and write latencies. This is useful for in-depth performance analysis. ```bash # tpa sk -v | grep -e sid -e _lat sid=0 192.168.1.10:54157 192.168.1.10:4096 worker=0 established write_lat.submit(avg/max) : 0.0/16.1us write_lat.xmit(avg/max) : 0.1/52.5us write_lat.complete(avg/max) : 4.2/102.1us read_lat.submit(avg/max) : 0.1/16.1us ``` -------------------------------- ### libtpa Latency Metrics Explained Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Explains the latency metrics for read and write operations in libtpa. These metrics provide insights into the time taken for different stages of data transfer, such as submission, transmission, and completion. ```text read_lat.drain(avg/max) : 0.2/49.6us read_lat.complete(avg/max) : 0.2/49.7us read_lat.last_write(avg/max) : 4.8/102.8us write_lat.submit denotes the latency from stage 1 to stage 2. write_lat.xmit denotes the latency from stage 1 to stage 3. write_lat.complete denotes the latency from stage 1 to stage 4. read_lat.submit denotes the latency from stage 1 to stage 2. read_lat.drain denotes the latency from stage 1 to stage 3. read_lat.complete denotes the latency from stage 1 to stage 4. ``` -------------------------------- ### List All Sockets (Including Closed) with 'tpa sk -a' Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Displays all network sockets, including those that have been closed. This provides a comprehensive history of socket states, useful for debugging connection issues. ```bash # tpa sk -a sid=[0] 192.168.1.10:55588 192.168.1.10:4096 worker=0 closed sid=[1] 192.168.1.10:55586 192.168.1.10:4096 worker=0 closed sid=[2] 192.168.1.10:55607 192.168.1.10:4096 worker=0 closed sid=[3] 192.168.1.10:55614 192.168.1.10:4096 worker=0 closed sid=4 192.168.1.10:55569 192.168.1.10:4096 worker=0 established sid=5 192.168.1.10:55555 192.168.1.10:4096 worker=0 established sid=6 192.168.1.10:55589 192.168.1.10:4096 worker=0 established sid=7 192.168.1.10:55609 192.168.1.10:4096 worker=0 established ``` -------------------------------- ### View Libtpa Socktrace for Retransmissions Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Shows how libtpa automatically archives socktraces when recovery from abnormal events like retransmissions occurs. It also displays how to view recovery time details using the 'tpa st' command. ```bash # tpa st | grep rto | head /var/log/tpa/client/socktrace194 ... 2023-12-04.16:55:00.070575 ... rto-107.447ms /var/log/tpa/client/socktrace193 ... 2023-12-04.16:55:00.068062 ... rto-214.160ms /var/log/tpa/client/socktrace192 ... 2023-12-04.16:55:00.065471 ... rto-214.160ms /var/log/tpa/client/socktrace191 ... 2023-12-04.16:55:00.062957 ... rto-234.977ms /var/log/tpa/client/socktrace190 ... 2023-12-04.16:55:00.060359 ... rto-214.160ms /var/log/tpa/client/socktrace189 ... 2023-12-04.16:55:00.057774 ... rto-214.160ms /var/log/tpa/client/socktrace188 ... 2023-12-04.16:55:00.055150 ... rto-184.099ms /var/log/tpa/client/socktrace187 ... 2023-12-04.16:55:00.052640 ... rto-178.073ms /var/log/tpa/client/socktrace186 ... 2023-12-04.16:55:00.050103 ... rto-181.962ms /var/log/tpa/client/socktrace185 ... 2023-12-04.16:55:00.047533 ... rto-179.440ms ``` ```bash tpa st /var/log/tpa/client/socktrace194 ``` -------------------------------- ### Analyze TCP Handshake and Payload with Libtpa Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Demonstrates how libtpa can trace a typical TCP handshake, track TCP payload transmission, and calculate precise latency. It also highlights that the 'swing' debug tool introduces a 1ms delay per loop. ```text # Lines 1-3: Typical TCP handshake process # Line 4: 12 bytes of TCP payload ("hello world") sent # Line 5: Reply received from techo ``` -------------------------------- ### Define socket options for libtpa connections Source: https://github.com/bytedance/libtpa/blob/main/doc/prog_guide.rst Defines a structure for passing extra options to libtpa socket creation functions like tpa_connect_to and tpa_listen_on. It includes options for listen scaling, user-defined data, and specifying a local port for active connections. ```c /* * Provides extra options for socks going to be created by * tpa_connect_to and tpa_connect_to. */ struct tpa_sock_opts { /* * When @listen_scaling is set to * - 0: passive connections will be only distributed to the worker * where this listen sock has been bound to. * - 1: passive connections will be distributed to all workers. * * tpa_listen_on only. */ uint64_t listen_scaling:1; uint64_t bits_reserved:63; /* * A private data set by user for listen sock. It could be * retrieved by tpa_sock_info_get when a new sock is accepted. * * tpa_listen_on only. */ void *data; /* * Specifies a local port to bind to. * * tpa_connect_to only. */ uint16_t local_port; uint8_t reserved[128 - 18]; /* XXX: it's ugly */ } __attribute__((packed)); ``` -------------------------------- ### Set TPA_ID for Libtpa Instance Source: https://github.com/bytedance/libtpa/blob/main/doc/user_guide.rst Manually sets the TPA_ID for a libtpa instance, providing more control, especially when running multiple instances of the same application. This is recommended as many libtpa tools require TPA_ID. ```bash # TPA_ID=client tpa run swing .... ```