### Twemproxy Command Line Interface Examples Source: https://context7.com/twitter/twemproxy/llms.txt Demonstrates various command-line options for starting and configuring the twemproxy (nutcracker) server. Includes options for help, daemon mode, configuration testing, logging levels, memory management, and version information. ```bash # Show help and available options nutcracker --help # Start with default configuration nutcracker -c /etc/nutcracker/nutcracker.yml # Start as a daemon with custom settings nutcracker -d -c /etc/nutcracker.yml -p /var/run/nutcracker.pid -o /var/log/nutcracker.log # Test configuration file for syntax errors nutcracker -t -c /etc/nutcracker.yml # Start with custom stats port and verbose logging nutcracker -c /etc/nutcracker.yml -s 22223 -v 6 # Start with smaller mbuf size for handling many connections nutcracker -c /etc/nutcracker.yml -m 512 # Show version information nutcracker -V # Print stats description nutcracker --describe-stats ``` -------------------------------- ### Start and Manage Twemproxy Service Source: https://context7.com/twitter/twemproxy/llms.txt Commands to start twemproxy with production settings and manage its systemd service. This includes starting the daemon, configuring logging, and setting up the service to run on boot. ```bash # Start nutcracker for production with recommended settings nutcracker \ -c /etc/nutcracker/production.yml \ -p /var/run/nutcracker.pid \ -o /var/log/nutcracker/nutcracker.log \ -s 22222 \ -a 127.0.0.1 \ -i 30000 \ -v 6 \ -m 512 \ -d # Systemd service file example (/etc/systemd/system/nutcracker.service) # [Unit] # Description=twemproxy - A fast and lightweight proxy for memcached and redis # After=network.target # # [Service] # Type=forking # PIDFile=/var/run/nutcracker.pid # ExecStartPre=/usr/local/bin/nutcracker -t -c /etc/nutcracker/nutcracker.yml # ExecStart=/usr/local/bin/nutcracker -d -c /etc/nutcracker/nutcracker.yml -p /var/run/nutcracker.pid -o /var/log/nutcracker.log -v 6 -m 512 # ExecReload=/bin/kill -HUP $MAINPID # Restart=on-failure # LimitNOFILE=65535 # # [Install] # WantedBy=multi-user.target # Enable and start service sudo systemctl enable nutcracker sudo systemctl start nutcracker sudo systemctl status nutcracker ``` -------------------------------- ### Twemproxy Configuration Example (YAML) Source: https://github.com/twitter/twemproxy/blob/master/README.md An example of a Twemproxy configuration file written in YAML format. This snippet demonstrates how to define multiple server pools, each with specific settings for listening addresses, hashing, distribution, connection limits, server details, and protocol types. ```yaml alpha: listen: 127.0.0.1:22121 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true redis: true server_retry_timeout: 2000 server_failure_limit: 1 servers: - 127.0.0.1:6379:1 beta: listen: 127.0.0.1:22122 hash: fnv1a_64 hash_tag: "{}" ``` -------------------------------- ### Twemproxy Configuration Example Source: https://context7.com/twitter/twemproxy/llms.txt An example configuration for a Twemproxy pool named 'example_pool'. It demonstrates settings for listening address, hash function, distribution mode, timeouts, connection limits, server configurations, and protocol specifics for Redis. ```yaml example_pool: # Required: Listening address (ip:port, hostname:port, or unix socket path) listen: 127.0.0.1:22121 # listen: /var/run/nutcracker.sock 0666 # Unix socket with permissions # Hash function for key distribution # Options: one_at_a_time, md5, crc16, crc32, crc32a, fnv1_64, fnv1a_64 (default), # fnv1_32, fnv1a_32, hsieh, murmur, jenkins hash: fnv1a_64 # Two-character hash tag for partial key hashing # Example: "{}" means keys "user:{123}:name" and "user:{123}:email" hash same hash_tag: "{}" # Key distribution mode # Options: ketama (default, consistent hashing), modula, random distribution: ketama # Connection timeout in milliseconds (default: unlimited) timeout: 400 # TCP backlog for listening socket (default: 512) backlog: 1024 # Enable TCP keepalive for server connections (default: false) tcpkeepalive: true # Preconnect to all servers on startup (default: false) preconnect: true # Protocol: true for Redis, false for memcached (default: false) redis: true # Redis authentication password redis_auth: secretpassword # Redis database number (default: 0) redis_db: 0 # Maximum connections per server (default: 1) server_connections: 1 # Maximum client connections (default: unlimited) client_connections: 10000 # Auto-eject failed servers (default: false) auto_eject_hosts: true # Retry timeout for ejected servers in ms (default: 30000) server_retry_timeout: 30000 # Consecutive failures before ejection (default: 2) server_failure_limit: 3 # Server list: ip:port:weight or ip:port:weight name servers: - 127.0.0.1:6379:1 - 127.0.0.1:6380:1 server2 ``` -------------------------------- ### Twemproxy Configuration Example Source: https://context7.com/twitter/twemproxy/llms.txt An example configuration for a Redis pool named 'example_pool' in Twemproxy. ```APIDOC ## Twemproxy Configuration Example ### Description This section provides a detailed example of a Twemproxy configuration for a Redis pool, showcasing various options for listening addresses, hashing, distribution, timeouts, connection management, and server definitions. ### Configuration Snippet ```yaml example_pool: # Required: Listening address (ip:port, hostname:port, or unix socket path) listen: 127.0.0.1:22121 # listen: /var/run/nutcracker.sock 0666 # Unix socket with permissions # Hash function for key distribution # Options: one_at_a_time, md5, crc16, crc32, crc32a, fnv1_64, fnv1a_64 (default), # fnv1_32, fnv1a_32, hsieh, murmur, jenkins hash: fnv1a_64 # Two-character hash tag for partial key hashing # Example: "{}" means keys "user:{123}:name" and "user:{123}:email" hash same hash_tag: "{}" # Key distribution mode # Options: ketama (default, consistent hashing), modula, random distribution: ketama # Connection timeout in milliseconds (default: unlimited) timeout: 400 # TCP backlog for listening socket (default: 512) backlog: 1024 # Enable TCP keepalive for server connections (default: false) tcpkeepalive: true # Preconnect to all servers on startup (default: false) preconnect: true # Protocol: true for Redis, false for memcached (default: false) redis: true # Redis authentication password redis_auth: secretpassword # Redis database number (default: 0) redis_db: 0 # Maximum connections per server (default: 1) server_connections: 1 # Maximum client connections (default: unlimited) client_connections: 10000 # Auto-eject failed servers (default: false) auto_eject_hosts: true # Retry timeout for ejected servers in ms (default: 30000) server_retry_timeout: 30000 # Consecutive failures before ejection (default: 2) server_failure_limit: 3 # Server list: ip:port:weight or ip:port:weight name servers: - 127.0.0.1:6379:1 - 127.0.0.1:6380:1 server2 ``` ### Parameters * **listen** (string) - Required - The listening address (ip:port, hostname:port, or unix socket path). * **hash** (string) - Optional - Hash function for key distribution. Options include `one_at_a_time`, `md5`, `crc16`, `crc32`, `crc32a`, `fnv1_64`, `fnv1a_64` (default), `fnv1_32`, `fnv1a_32`, `hsieh`, `murmur`, `jenkins`. * **hash_tag** (string) - Optional - Two-character hash tag for partial key hashing. Example: `{}`. * **distribution** (string) - Optional - Key distribution mode. Options: `ketama` (default, consistent hashing), `modula`, `random`. * **timeout** (integer) - Optional - Connection timeout in milliseconds (default: unlimited). * **backlog** (integer) - Optional - TCP backlog for listening socket (default: 512). * **tcpkeepalive** (boolean) - Optional - Enable TCP keepalive for server connections (default: false). * **preconnect** (boolean) - Optional - Preconnect to all servers on startup (default: false). * **redis** (boolean) - Optional - Protocol type: true for Redis, false for memcached (default: false). * **redis_auth** (string) - Optional - Redis authentication password. * **redis_db** (integer) - Optional - Redis database number (default: 0). * **server_connections** (integer) - Optional - Maximum connections per server (default: 1). * **client_connections** (integer) - Optional - Maximum client connections (default: unlimited). * **auto_eject_hosts** (boolean) - Optional - Auto-eject failed servers (default: false). * **server_retry_timeout** (integer) - Optional - Retry timeout for ejected servers in ms (default: 30000). * **server_failure_limit** (integer) - Optional - Consecutive failures before ejection (default: 2). * **servers** (array of strings) - Required - List of servers in the format `ip:port:weight` or `ip:port:weight name`. ### Request Example ```yaml # Configuration file content ``` ### Response This section describes configuration parameters, not a typical API response. #### Success Response (N/A) N/A #### Response Example N/A ``` -------------------------------- ### C Function Documentation Example Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Example of documenting a C function, including its purpose, arguments, return values, and potential caveats. It highlights the importance of clear explanations for non-standard argument usage or unexpected return conditions. ```c /* * Try to acquire a physical address lock while a pmap is locked. If we * fail to trylock we unlock and lock the pmap directly and cache the * locked pa in *locked. The caller should then restart their loop in case * the virtual to physical mapping has changed. * * Returns 0 on success and -1 on failure. */ int vm_page_pa_tryrelock(pmap_t pmap, vm_paddr_t pa, vm_paddr_t *locked) { ... } ``` -------------------------------- ### Twemproxy YAML Configuration Examples Source: https://context7.com/twitter/twemproxy/llms.txt Illustrates various configurations for twemproxy server pools using YAML. Covers Redis and Memcached protocols, hashing algorithms, distribution methods, connection pooling, and high-availability settings. ```yaml # Basic Redis pool configuration alpha: listen: 127.0.0.1:22121 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true redis: true server_retry_timeout: 2000 server_failure_limit: 1 servers: - 127.0.0.1:6379:1 # Redis pool with hash tags for key grouping beta: listen: 127.0.0.1:22122 hash: fnv1a_64 hash_tag: "{}" distribution: ketama auto_eject_hosts: false timeout: 400 redis: true servers: - 127.0.0.1:6380:1 server1 - 127.0.0.1:6381:1 server2 - 127.0.0.1:6382:1 server3 - 127.0.0.1:6383:1 server4 # Memcached pool with preconnect and backlog gamma: listen: 127.0.0.1:22123 hash: fnv1a_64 distribution: ketama timeout: 400 backlog: 1024 preconnect: true auto_eject_hosts: true server_retry_timeout: 2000 server_failure_limit: 3 servers: - 127.0.0.1:11212:1 - 127.0.0.1:11213:1 # Unix socket listener with weighted servers omega: listen: /tmp/nutcracker.sock 0666 hash: hsieh distribution: ketama auto_eject_hosts: false servers: - 127.0.0.1:11214:100000 - 127.0.0.1:11215:1 # Redis pool with authentication secure_pool: listen: 127.0.0.1:22125 hash: fnv1a_64 distribution: ketama redis: true redis_auth: mypassword redis_db: 0 timeout: 400 servers: - 127.0.0.1:6379:1 # High-availability pool with liveness settings resilient_pool: listen: 127.0.0.1:22126 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true server_retry_timeout: 30000 server_failure_limit: 3 timeout: 400 tcpkeepalive: true redis: true servers: - 127.0.0.1:6379:1 - 127.0.0.1:6380:1 - 127.0.0.1:6381:1 ``` -------------------------------- ### C Null Pointer Check Example Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Demonstrates the correct way to test pointers against NULL in C. It emphasizes using `p == NULL` instead of the less explicit `!(p)`. ```c if (p == NULL) { // pointer is NULL } ``` ```c if (!(p)) { // This is not the preferred way to check for NULL } ``` -------------------------------- ### Build Twemproxy from Source Source: https://context7.com/twitter/twemproxy/llms.txt This section provides instructions for building Twemproxy from its source code, including cloning the repository, installing build dependencies for Debian/Ubuntu and RHEL/CentOS, and configuring build options for release, debug, and optimized versions. ```bash # Clone the repository git clone git@github.com:twitter/twemproxy.git cd twemproxy # Install build dependencies (Debian/Ubuntu) sudo apt-get install automake libtool autoconf # Install build dependencies (RHEL/CentOS) sudo yum install automake libtool autoconf # Generate configure script autoreconf -fvi # Standard release build ./configure make sudo make install # Debug build with full logging CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full make # Optimized build CFLAGS="-O3 -fno-strict-aliasing" ./configure make ``` -------------------------------- ### Build Twemproxy from Source Source: https://context7.com/twitter/twemproxy/llms.txt Instructions for compiling twemproxy from source code. This includes options for custom optimization levels and installation to a specified prefix. It also covers building from a release tarball, which simplifies the process by avoiding the need for autoreconf. ```bash # Build with specific optimization level CFLAGS="-O1" ./configure make # Run the built binary src/nutcracker -h # Run unit tests make check # Install to custom prefix ./configure --prefix=/opt/nutcracker make sudo make install # Build from release tarball (simpler, no autoreconf needed) wget https://github.com/twitter/twemproxy/releases/download/0.5.0/twemproxy-0.5.0.tar.gz tar xzf twemproxy-0.5.0.tar.gz cd twemproxy-0.5.0 ./configure make sudo make install ``` -------------------------------- ### C Conditional Compilation Example Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Demonstrates preferred conditional compilation using an if statement for configuration options known at build time. This approach allows the compiler to perform more thorough checks on all code paths compared to traditional preprocessor directives. ```c if (HAS_FOO) ... else ... ``` -------------------------------- ### Conditional Compilation Example (C/C++) Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Demonstrates the use of preprocessor directives for conditional compilation in C/C++. This pattern allows code sections to be included or excluded based on defined macros like HAS_FOO. Compilers like GCC can optimize this effectively. ```c #ifdef HAS_FOO ... #else ... #endif ``` -------------------------------- ### Twemproxy Debug Log Example Source: https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md Illustrates the log output when Twemproxy is run with debug logging enabled at LOG_INFO level. It shows connection acceptance, server connection, request timeouts, and server ejection events. ```log [Thu Aug 2 00:03:09 2012] nc_proxy.c:336 accepted c 7 on p 6 from '127.0.0.1:54009' [Thu Aug 2 00:03:09 2012] nc_server.c:528 connected on s 8 to server '127.0.0.1:11211:1' [Thu Aug 2 00:03:09 2012] nc_core.c:270 req 1 on s 8 timedout [Thu Aug 2 00:03:09 2012] nc_core.c:207 close s 8 '127.0.0.1:11211' on event 0004 eof 0 done 0 rb 0 sb 20: Connection timed out [Thu Aug 2 00:03:09 2012] nc_server.c:406 close s 8 schedule error for req 1 len 20 type 5 from c 7: Connection timed out [Thu Aug 2 00:03:09 2012] nc_server.c:281 update pool 0 'alpha' to delete server '127.0.0.1:11211:1' for next 2 secs [Thu Aug 2 00:03:10 2012] nc_connection.c:314 recv on sd 7 eof rb 20 sb 35 [Thu Aug 2 00:03:10 2012] nc_request.c:334 c 7 is done [Thu Aug 2 00:03:10 2012] nc_core.c:207 close c 7 '127.0.0.1:54009' on event 0001 eof 1 done 1 rb 20 sb 35 [Thu Aug 2 00:03:11 2012] nc_proxy.c:336 accepted c 7 on p 6 from '127.0.0.1:54011' [Thu Aug 2 00:03:11 2012] nc_server.c:528 connected on s 8 to server '127.0.0.1:11212:1' [Thu Aug 2 00:03:12 2012] nc_connection.c:314 recv on sd 7 eof rb 20 sb 8 [Thu Aug 2 00:03:12 2012] nc_request.c:334 c 7 is done [Thu Aug 2 00:03:12 2012] nc_core.c:207 close c 7 '127.0.0.1:54011' on event 0001 eof 1 done 1 rb 20 sb 8 ``` -------------------------------- ### C Macro for Parameterized Addition Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Example of a parameterized C macro for adding 1 to a value. It demonstrates the best practice of enclosing macro parameters in parentheses to avoid unexpected behavior with operator precedence. ```c #define ADD_1(_x) ((_x) + 1) ``` -------------------------------- ### Manage Twemproxy Signals for Logging and Shutdown Source: https://context7.com/twitter/twemproxy/llms.txt This section details how to use Unix signals to control Twemproxy's log level, rotate log files, and perform graceful shutdowns. It includes commands to find the process ID and examples for logrotate configuration. ```bash # Get nutcracker process ID pgrep nutcracker # 12345 # Or from pid file cat /var/run/nutcracker.pid # 12345 # Increase log verbosity (SIGTTIN) kill -SIGTTIN 12345 # Log level increased by 1 # Decrease log verbosity (SIGTTOU) kill -SIGTTOU 12345 # Log level decreased by 1 # Reopen log files for rotation (SIGHUP) kill -SIGHUP 12345 # Example logrotate configuration for nutcracker # /etc/logrotate.d/nutcracker # /var/log/nutcracker.log { # daily # rotate 7 # compress # delaycompress # missingok # notifempty # postrotate # kill -SIGHUP $(cat /var/run/nutcracker.pid) 2>/dev/null || true # endscript # } # Graceful shutdown kill -SIGTERM 12345 # Log level reference: # 0 - EMERG # 1 - ALERT # 2 - CRIT # 3 - ERR # 4 - WARN # 5 - NOTICE (default) # 6 - INFO (recommended for production) # 7 - DEBUG # 8-11 - VERBOSE DEBUG levels ``` -------------------------------- ### SETRANGE Source: https://github.com/twitter/twemproxy/blob/master/notes/redis.md Overwrites part of a string value stored at a key, replacing the part of the string starting at the specified offset. ```APIDOC ## SETRANGE ### Description Overwrites part of a string value stored at a key, replacing the part of the string starting at the specified offset. ### Method POST ### Endpoint /twitter/twemproxy ### Parameters #### Query Parameters - **command** (string) - Required - The command to execute, which is 'SETRANGE'. - **key** (string) - Required - The key to modify. - **offset** (integer) - Required - The offset at which to start overwriting. - **value** (string) - Required - The string value to use for overwriting. ### Request Example ```json { "command": "SETRANGE", "key": "mykey", "offset": 5, "value": "new_part" } ``` ### Response #### Success Response (200) - **length** (integer) - The length of the string after it was modified. #### Response Example ```json { "length": 15 } ``` ``` -------------------------------- ### C Function Definition Style Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Demonstrates the preferred C style for function definitions, where the function name starts in the first column for easy identification. ```c static char * concat(char *s1, char *s2) { body of the function } ``` -------------------------------- ### C Boolean Test Example Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Illustrates the proper use of the '!' operator for boolean tests in C. It shows that '!' should only be used for boolean conditions, not for checking character or pointer termination. ```c if (*p == '\0') { // Correct way to check for null terminator } ``` ```c if (!*p) { // Avoid this for non-boolean checks } ``` -------------------------------- ### C Assignment in Condition Avoidance Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Provides an example of how to avoid using assignments within conditional statements (if/while) in C. It shows the preferred method of assigning and then checking for NULL. ```c struct foo *foo; foo = malloc(sizeof(*foo)); if (foo == NULL) { return -1; } ``` ```c struct foo *foo; if ((foo = malloc(sizeof(*foo))) == NULL) { return -1; } ``` -------------------------------- ### Twemproxy Performance Benchmarking Source: https://github.com/twitter/twemproxy/blob/master/notes/redis.md Details on setting up and running performance benchmarks for Twemproxy against Redis. ```APIDOC ## Twemproxy Performance Benchmarking ### Description This section outlines the setup and results for benchmarking Twemproxy's performance when proxying Redis commands. ### Setup - **Redis Server**: Running on machine A. - **Twemproxy (nutcracker)**: Running on machine A as a local proxy to the Redis server. - **Redis Benchmark Tool**: `redis-benchmark` running on machine B. - **Network Configuration**: Machine A is different from Machine B. - **Twemproxy Build**: Compiled with `--enable-debug=no`. - **Twemproxy Runtime**: Running with `mbuf-size` of 512 (specified with `-m 512`). - **Redis Server Version**: Built from the Redis 2.6 branch. ### Redis Benchmark Against Redis Server **Command**: `redis-benchmark -h -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379` **Results**: - `SET`: 89285.71 requests per second - `GET`: 92592.59 requests per second - `INCR`: 89285.71 requests per second - `LPUSH`: 90090.09 requests per second - `LPOP`: 90090.09 requests per second - `SADD`: 90090.09 requests per second - `SPOP`: 93457.95 requests per second - `LPUSH` (needed to benchmark LRANGE): 89285.71 requests per second - `LRANGE_100` (first 100 elements): 36496.35 requests per second - `LRANGE_300` (first 300 elements): 15748.03 requests per second - `LRANGE_500` (first 450 elements): 11135.86 requests per second - `LRANGE_600` (first 600 elements): 8650.52 requests per second ``` -------------------------------- ### Debug Twemproxy Configuration and Runtime Source: https://context7.com/twitter/twemproxy/llms.txt Tools and commands for debugging twemproxy issues. This includes syntax checking configuration files, running with increased verbosity, tracing system calls, detecting memory leaks with valgrind, and enabling core dumps. ```bash # Test configuration file syntax nutcracker -t -c /etc/nutcracker/nutcracker.yml # nutcracker: configuration file 'nutcracker.yml' syntax is ok # Run with maximum verbosity for debugging nutcracker -c nutcracker.yml -v 11 -o /tmp/nutcracker-debug.log # Trace system calls strace -o strace.txt -ttT -s 1024 -p $(pgrep nutcracker) # Memory leak detection with valgrind valgrind --tool=memcheck --leak-check=yes ./src/nutcracker -c nutcracker.yml # Enable core dumps for crash analysis ulimit -c unlimited # Core dumps will be written to current directory or /var/crash ``` -------------------------------- ### C Header File Guard Example Source: https://github.com/twitter/twemproxy/blob/master/notes/c-styleguide.txt Illustrates the standard C preprocessor directives used as header guards to prevent multiple inclusions of a header file. This is crucial for avoiding redefinition errors and circular dependencies. ```c #ifndef _FOO_H_ #define _FOO_H_ ... #endif /* _FOO_H_ */ ``` -------------------------------- ### Memcached Protocol Operations with Twemproxy Source: https://context7.com/twitter/twemproxy/llms.txt Demonstrates Memcached ASCII protocol commands executed through Twemproxy. Covers storage, retrieval, delete, arithmetic, CAS, touch, and version operations. ```bash # Connect to twemproxy memcached pool using netcat or telnet nc 127.0.0.1 22123 # Storage commands set mykey 0 3600 11 Hello World # STORED add newkey 0 3600 5 value # STORED replace mykey 0 3600 7 Updated # STORED append mykey 0 3600 5 -tail # STORED prepend mykey 0 3600 5 head- # STORED # Compare-and-swap (CAS) operation gets mykey # VALUE mykey 0 17 12345 # head-Updated-tail # END cas mykey 0 3600 10 12345 new-value! # STORED # Retrieval commands get mykey # VALUE mykey 0 10 # new-value! # END get mykey otherkey thirdkey # VALUE mykey 0 10 # new-value! # VALUE otherkey 0 5 # value # END gets mykey # VALUE mykey 0 10 12346 # new-value! # END # Delete command delete mykey # DELETED delete nonexistent # NOT_FOUND # Arithmetic commands set counter 0 3600 1 0 # STORED incr counter 1 # 1 incr counter 10 # 11 decr counter 5 # 6 # Touch command (update expiration) touch mykey 7200 # TOUCHED # Version command version # VERSION 1.6.9 # Quit connection quit ``` -------------------------------- ### Twemproxy Server Commands Source: https://github.com/twitter/twemproxy/blob/master/notes/redis.md This section details the server commands supported by Twemproxy. Note that all listed commands are marked as 'No' for supported status in the provided documentation. ```APIDOC ## Twemproxy Server Commands ### Description This section details the server commands supported by Twemproxy. All commands listed are marked as not supported in the provided documentation. ### Method N/A (These are server commands, not typical HTTP API endpoints) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ## Supported Commands: - **BGREWRITEAOF** (Not Supported) - **BGSAVE** (Not Supported) - **CLIENT KILL** (Not Supported) - **CLIENT LIST** (Not Supported) - **CONFIG GET** (Not Supported) - **CONFIG SET** (Not Supported) - **CONFIG RESETSTAT** (Not Supported) - **DBSIZE** (Not Supported) - **DEBUG OBJECT** (Not Supported) - **DEBUG SEGFAULT** (Not Supported) - **FLUSHALL** (Not Supported) - **FLUSHDB** (Not Supported) - **INFO** (Not Supported) - **LASTSAVE** (Not Supported) ### Command Format Examples (as per documentation, though not supported): - **CLIENT KILL**: `CLIENT KILL ip:port` - **CONFIG GET**: `CONFIG GET parameter` - **CONFIG SET**: `CONFIG SET parameter value` - **DEBUG OBJECT**: `DEBUG OBJECT key` ``` -------------------------------- ### Redis Protocol Support Source: https://context7.com/twitter/twemproxy/llms.txt Demonstrates how to interact with Twemproxy using Redis commands via `redis-cli`. ```APIDOC ## Redis Protocol Support ### Description Twemproxy supports a comprehensive set of Redis commands for various data structures including strings, hashes, lists, sets, sorted sets, and more. This section provides examples of common Redis operations performed through Twemproxy. ### Method `redis-cli` (or any Redis client) ### Endpoint `127.0.0.1:22121` (or the configured listen address) ### Parameters No specific parameters for the protocol itself, but Redis commands have their own parameters. ### Request Example ```bash # Connect to twemproxy Redis pool using redis-cli redis-cli -h 127.0.0.1 -p 22121 # String operations SET mykey "Hello World" GET mykey INCR counter APPEND mykey " - Extended" # Multi-key operations (keys are automatically fragmented) MSET key1 "value1" key2 "value2" key3 "value3" MGET key1 key2 key3 DEL key1 key2 key3 # Hash operations HSET user:1000 name "John" email "john@example.com" HGET user:1000 name HGETALL user:1000 HMGET user:1000 name email # List operations LPUSH mylist "world" "hello" LRANGE mylist 0 -1 RPOP mylist # Set operations SADD myset "member1" "member2" "member3" SMEMBERS myset SISMEMBER myset "member1" # Sorted set operations ZADD leaderboard 100 "player1" 200 "player2" 150 "player3" ZRANGE leaderboard 0 -1 WITHSCORES ZRANK leaderboard "player2" # Key expiration SET session:abc123 "user_data" EX 3600 TTL session:abc123 EXPIRE mykey 300 PTTL mykey # Geo operations GEOADD locations 13.361389 52.519444 "Berlin" -122.419416 37.774929 "San Francisco" GEODIST locations "Berlin" "San Francisco" km GEOPOS locations "Berlin" # HyperLogLog for cardinality estimation PFADD visitors "user1" "user2" "user3" PFCOUNT visitors # Lua scripting (requires keys to hash to same server) EVAL "return redis.call('GET', KEYS[1])" 1 mykey ``` ### Response Redis command responses (e.g., `OK`, `(integer) 1`, `"Hello World"`, etc.). #### Success Response (200) Standard Redis responses. #### Response Example ``` OK (integer) 1 "Hello World" ``` ``` -------------------------------- ### Twemproxy Hash Commands Source: https://github.com/twitter/twemproxy/blob/master/notes/redis.md This section details the various Hash commands supported by Twemproxy, including HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HINCRBYFLOAT, HKEYS, HLEN, HMGET, HMSET, HSET, HSETNX, HVALS, and HSCAN. ```APIDOC ## Twemproxy Hash Commands ### Description This section details the various Hash commands supported by Twemproxy, including HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HINCRBYFLOAT, HKEYS, HLEN, HMGET, HMSET, HSET, HSETNX, HVALS, and HSCAN. ### Commands #### HDEL * **Description**: Delete one or more fields from a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HDEL key field [field ...]` #### HEXISTS * **Description**: Determine if a hash contains a specified field. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HEXISTS key field` #### HGET * **Description**: Get the value associated with a field in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HGET key field` #### HGETALL * **Description**: Get all fields and values in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HGETALL key` #### HINCRBY * **Description**: Increment the value of a field in a hash by a specified integer. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HINCRBY key field increment` #### HINCRBYFLOAT * **Description**: Increment the value of a field in a hash by a specified float. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HINCRBYFLOAT key field increment` #### HKEYS * **Description**: Get all the fields in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HKEYS key` #### HLEN * **Description**: Get the number of fields in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HLEN key` #### HMGET * **Description**: Get the values associated with multiple fields in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HMGET key field [field ...]` #### HMSET * **Description**: Set multiple fields to multiple values in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HMSET key field value [field value ...]` #### HSET * **Description**: Set the value of a field in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HSET key field value` #### HSETNX * **Description**: Set the value of a field in a hash, only if the field does not exist. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HSETNX key field value` #### HVALS * **Description**: Get all the values in a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HVALS key` #### HSCAN * **Description**: Incrementally iterate over fields and values of a hash. * **Method**: Not applicable (command-line interface) * **Endpoint**: Not applicable * **Parameters**: * **Path Parameters**: None * **Query Parameters**: None * **Request Body**: None * **Format**: `HSCAN key cursor [MATCH pattern] [COUNT count]` ``` -------------------------------- ### Pub/Sub Commands Source: https://github.com/twitter/twemproxy/blob/master/notes/redis.md This section details the Pub/Sub commands available in Twemproxy. All commands are currently unsupported. ```APIDOC ## Pub/Sub Commands ### Description This section details the Pub/Sub commands available in Twemproxy. All commands are currently unsupported. ### Method N/A ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response (N/A) N/A #### Response Example N/A ## PSUBSCRIBE ### Description Subscribes the client to the given patterns. Currently not supported. ### Method N/A ### Endpoint N/A ## PUBLISH ### Description Posts a message to the given channel. Currently not supported. ### Method N/A ### Endpoint N/A ## PUBSUB ### Description Describes the currently active channels or patterns. Currently not supported. ### Method N/A ### Endpoint N/A ## PUNSUBSCRIBE ### Description Unsubscribes the client from the given patterns. Currently not supported. ### Method N/A ### Endpoint N/A ## SUBSCRIBE ### Description Subscribes the client to the given channels. Currently not supported. ### Method N/A ### Endpoint N/A ## UNSUBSCRIBE ### Description Unsubscribes the client from the given channels. Currently not supported. ### Method N/A ### Endpoint N/A ``` -------------------------------- ### Production Deployment Configuration for Redis and Memcached Source: https://context7.com/twitter/twemproxy/llms.txt Recommended YAML configurations for deploying twemproxy in production environments. This includes settings for both Redis and Memcached clusters, focusing on high availability and optimal performance, such as auto-ejection, retry timeouts, and connection pooling. ```yaml # Production Redis cluster configuration production_redis: listen: 0.0.0.0:22121 hash: fnv1a_64 hash_tag: "{}" distribution: ketama # Enable auto-ejection for resilience auto_eject_hosts: true server_retry_timeout: 30000 server_failure_limit: 3 # Set reasonable timeout timeout: 400 # Enable TCP keepalive tcpkeepalive: true # Preconnect for faster first requests preconnect: true redis: true redis_auth: your_secure_password # Use named servers for easier maintenance servers: - 10.0.1.10:6379:1 redis-primary-1 - 10.0.1.11:6379:1 redis-primary-2 - 10.0.1.12:6379:1 redis-primary-3 - 10.0.1.13:6379:1 redis-primary-4 # Production memcached cluster configuration production_memcached: listen: 0.0.0.0:11211 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true server_retry_timeout: 30000 server_failure_limit: 3 timeout: 200 backlog: 2048 preconnect: true tcpkeepalive: true servers: - 10.0.2.10:11211:1 memcached-1 - 10.0.2.11:11211:1 memcached-2 - 10.0.2.12:11211:1 memcached-3 - 10.0.2.13:11211:1 memcached-4 ``` -------------------------------- ### Get Twemproxy Statistics via Socat Source: https://github.com/twitter/twemproxy/blob/master/notes/debug.txt This command retrieves statistics from a twemproxy instance running on localhost:22222. It sends an empty string to the server via socat and pipes the output to both the console (using tee) and a file named stats.txt. ```bash printf "" | socat - TCP:localhost:22222 | tee stats.txt ```