### Running SkylerRedis Master Server Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md This command starts a SkylerRedis master server on a specified port. Ensure Go 1.24+ is installed and the program is executable. ```shell ./your_program.sh --port=6380 ``` -------------------------------- ### Start SkylerRedis Slave Instance (Docker) Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/09_replication_en.md This command starts a SkylerRedis instance as a slave, replicating from a specified master. It requires the `--replicaof` flag to point to the master's IP and port, and the slave must be run on a different port than the master. The `` can be 'host.docker.internal' when using Docker Desktop. ```sh # The can be 'host.docker.internal' on Docker Desktop docker run -d -p 6380:6380 --name skyler-slave1 skyler-redis --port 6380 --replicaof 6379 ``` -------------------------------- ### CONFIG GET Commands Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `CONFIG GET` command for retrieving configuration parameters like `dir` and `dbfilename`. Verifies that the server returns the correct configuration values. ```APIDOC ## CONFIG GET Commands ### Description Tests the `CONFIG GET` command for retrieving configuration parameters like `dir` and `dbfilename`. Verifies that the server returns the correct configuration values. ### Method GET ### Endpoint `/config` ### Parameters #### Query Parameters - **command** (string) - Required - The configuration command, e.g., `CONFIG`. - **parameter** (string) - Required - The configuration parameter to get, e.g., `dir` or `dbfilename`. ### Request Example (Get dir) ``` CONFIG GET dir ``` ### Response #### Success Response (Get dir) - **parameter_name** (string) - The name of the configuration parameter. - **parameter_value** (string) - The value of the configuration parameter. #### Response Example (Get dir) ```json { "parameter_name": "dir", "parameter_value": "/path/to/redis/data" } ``` ### Request Example (Get dbfilename) ``` CONFIG GET dbfilename ``` ### Response #### Success Response (Get dbfilename) - **parameter_name** (string) - The name of the configuration parameter. - **parameter_value** (string) - The value of the configuration parameter. #### Response Example (Get dbfilename) ```json { "parameter_name": "dbfilename", "parameter_value": "dump.rdb" } ``` ``` -------------------------------- ### Running SkylerRedis Replica Server Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md This command starts a SkylerRedis replica server, connecting it to a master server at the specified host and port. Requires Go 1.24+ and an executable program. ```shell ./your_program.sh --port=6381 --replicaof="127.0.0.1 6380" ``` -------------------------------- ### Start SkylerRedis Slave Instances Source: https://github.com/manhhodinh/skylerredis/blob/main/skyler_redis_summary_en.md These commands start SkylerRedis slave instances using Docker. Each slave is configured to replicate from a master instance. The `--replicaof` flag points to the master's IP and port. Different host ports (6380, 6381) are mapped for each slave, and their internal ports are also specified. `host.docker.internal` is used as a special DNS name to refer to the host machine from within Docker containers on Docker Desktop. ```shell # Start the first slave, listening on port 6380, replicating from the master on 6379 docker run -d -p 6380:6380 --name skyler-slave1 skyler-redis --port 6380 --replicaof host.docker.internal 6379 # Start the second slave, listening on port 6381 docker run -d -p 6381:6381 --name skyler-slave2 skyler-redis --port 6381 --replicaof host.docker.internal 6379 ``` -------------------------------- ### GET After Expiry Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `GET` command's behavior after a key has expired based on its `PX` setting. It verifies that expired keys return a null reply. ```APIDOC ## GET After Expiry ### Description Tests the `GET` command's behavior after a key has expired based on its `PX` setting. It verifies that expired keys return a null reply. ### Method GET ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - The key to retrieve. ### Request Example (Key Expired) ``` # At t=0ms: SET k v PX 50 # At t=60ms: GET k ``` ### Response #### Success Response (Key Expired) - **value** (string) - Null reply, represented as `$-1`. #### Response Example (Key Expired) ``` $-1 ``` ### Request Example (Key Still Alive) ``` # At t=0ms: SET k v PX 100 # At t=50ms: GET k ``` ### Response #### Success Response (Key Still Alive) - **value** (string) - The value associated with the key. #### Response Example (Key Still Alive) ``` $1 v ``` ``` -------------------------------- ### Run SkylerRedis TCP Server Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/01_tcp_server_en.md Executes the Go program to start the basic TCP server. This command compiles and runs the main application located in the `./cmd/skyler-redis` directory. ```sh go run ./cmd/skyler-redis ``` -------------------------------- ### Start SkylerRedis Master Instance (Docker) Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/09_replication_en.md This command starts a SkylerRedis instance in master mode using Docker. It exposes the default Redis port (6379) and assigns a container name for easy management. No specific replication flags are needed as master is the default mode. ```sh docker run -d -p 6379:6379 --name skyler-master skyler-redis ``` -------------------------------- ### Running SkylerRedis Server with RDB Persistence Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md Starts a SkylerRedis server on a specific port, loading initial data from an RDB file located in the current directory. Requires Go 1.24+. ```shell ./your_program.sh --dir "." --dbfilename "dump.rdb" --port=6380 ``` -------------------------------- ### Docker Command for SkylerRedis with Sharding Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/08_sharding_en.md Demonstrates how to start a SkylerRedis server using Docker with sharding enabled. This command specifies the number of shards and the maximum memory per shard, illustrating practical deployment configuration. ```sh # Start with 4 shards, each with a maxmemory of 1000 keys docker run -d -p 6379:6379 --name skyler-redis-server skyler-redis --numshards 4 --maxmemory 1000 ``` -------------------------------- ### GET Command - Retrieve String Values (Go) Source: https://context7.com/manhhodinh/skylerredis/llms.txt Retrieves the string value associated with a given key. It automatically handles expiration checks and updates the LRU clock upon access. Returns null if the key does not exist or has expired. This function implements the Redis GET command. ```go // internal/command/get.go func (Get) Handle(Conn net.Conn, args []string, isMaster bool, masterReplID string, masterReplOffset int, connectedSlaves int, shard *memory.Shard) { key := args[1] shard.Mu.Lock() defer shard.Mu.Unlock() entryPtr, ok := shard.Store[key] if !ok || (entryPtr.ExpiryTime != (time.Time{}) && time.Now().After(entryPtr.ExpiryTime)) { delete(shard.Store, key) utils.WriteNull(Conn) return } // Update LRU clock on access entryPtr.LRU = shard.LruClock shard.LruClock++ shard.Store[key] = entryPtr utils.WriteBulkString(Conn, entryPtr.Value) } // Usage with redis-cli: // GET mykey // Response: $11\r\nHello World\r\n // GET nonexistent // Response: $-1\r\n (null) ``` -------------------------------- ### Redis Hashes HGETALL Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Gets all fields and values in a hash. Returns an empty array if the hash does not exist. ```redis HGETALL h ``` -------------------------------- ### RDB Key-Value Load Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the loading of simple key-value pairs from an RDB file after a server restart. Verifies that GET commands return the correct values for keys persisted in RDB. ```APIDOC ## RDB Key-Value Load ### Description Tests the loading of simple key-value pairs from an RDB file after a server restart. Verifies that `GET` commands return the correct values for keys persisted in RDB. ### Method GET ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - The key to retrieve. ### Request Example ``` GET key ``` ### Response #### Success Response (200) - **value** (string) - The value associated with the key. #### Response Example ```json { "value": "some_value" } ``` ``` -------------------------------- ### Get Server Information (INFO) - Go Source: https://context7.com/manhhodinh/skylerredis/llms.txt Retrieves server information, specifically the replication status. It returns the role (master/slave), replication ID, offset, and connected slave count if the section is 'REPLICATION' and the server is a master. Otherwise, it indicates the role as slave or returns an error for unknown sections. This function uses server state variables. ```Go import ( "fmt" "net" "strings" "github.com/manhhodinh/skylerredis/internal/memory" "github.com/manhhodinh/skylerredis/internal/utils" ) // INFO command handler func (INFO) Handle(Conn net.Conn, args []string, isMaster bool, masterReplID string, masterReplOffset int, connectedSlaves int, shard *memory.Shard) { switch strings.ToUpper(args[1]) { case "REPLICATION": if isMaster { utils.WriteBulkString(Conn, fmt.Sprintf("role:master\nmaster_replid:%s\nmaster_repl_offset:%d\nconnected_slaves:%d", masterReplID, masterReplOffset, connectedSlaves)) } else { utils.WriteBulkString(Conn, "role:slave") } default: utils.WriteError(Conn, fmt.Sprintf("unknown INFO section '%s'", args[1])) } } ``` -------------------------------- ### Text-based JSON Format Example for Key-Value Persistence Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/13_persistence_rdb_en.md This snippet demonstrates a human-readable JSON structure for saving key-value pairs along with their expiration times. It is simple to generate and parse but results in large files and slow I/O, and is incompatible with Redis. ```json [ {"key": "mykey", "value": "myvalue", "expires_at": 1672531199000}, {"key": "anotherkey", "value": "anothervalue", "expires_at": null} ] ``` -------------------------------- ### Server Initialization and Command Routing in Go Source: https://context7.com/manhhodinh/skylerredis/llms.txt Sets up the main server, parses command-line flags, and initializes the event loop for handling network connections. It supports master/replica configurations and routes commands to appropriate shards based on the key. Background goroutines are spawned for shard-specific eviction tasks. ```go package main import ( "bufio" "flag" "log" "net" "os" "time" "github.com/your_org/your_repo/app/eventloop" "github.com/your_org/your_repo/app/server" "github.com/your_org/your_repo/app/utils" "github.com/your_org/your_repo/internal/command" "github.com/your_org/your_repo/internal/memory" ) var ( port = flag.String("port", "6379", "Port to listen on") hostname = flag.String("hostname", "127.0.0.1", "Hostname to listen on") numShards = flag.Int("numshards", 1, "Number of shards for sharded architecture") maxmemory = flag.Int("maxmemory", 0, "Max memory in bytes for each shard") replicaof = flag.String("replicaof", "", "Master host and port for replica configuration") isMaster = flag.Bool("ismaster", true, "Whether the server is a master") masterReplID = flag.String("replid", "", "Replica ID for replication") masterReplOffset = flag.Int64("re offset", 0, "Replica offset for replication") connectedSlaves = flag.Int("slaves", 0, "Number of connected slaves") dir = flag.String("dir", ".", "Directory for RDB persistence") dbfilename = flag.String("dbfilename", "dump.rdb", "RDB filename") ) func main() { flag.Parse() // Ensure RDB persistence directory exists if _, err := os.Stat(*dir); os.IsNotExist(err) { log.Fatalf("Directory '%s' does not exist: %v", *dir, err) } var replicaOfStr string if *replicaof != "" { if len(flag.Args()) > 0 { replicaOfStr = *replicaof + " " + flag.Args()[0] } } memory.InitShards(*numShards, *maxmemory) // Placeholder for server instance initialization var serverInstance interface{} // Replace with actual server type if *replicaof != "" { // Initialize as replica } else { // Initialize as master serverInstance = server.NewMaster(*hostname, *port, *masterReplID, *masterReplOffset, *connectedSlaves) } readCallback := func(conn net.Conn) error { reader := bufio.NewReader(conn) args, err := utils.ParseArgs(conn, reader) if err != nil { return err } var key string if len(args) > 1 { key = args[1] } shard := memory.GetShardForKey(key) command.HandleCommand(conn, args, *isMaster, *masterReplID, *masterReplOffset, *connectedSlaves, shard) if *isMaster && utils.IsModifyCommand(args) { if master, ok := serverInstance.(*server.Master); ok { go master.PropagateCommand(args) } } shard.EvictKeysByLRU() return nil } el, _ := eventloop.New(readCallback) go el.Start() // Background expiration and LRU eviction for each shard for i := 0; i < *numShards; i++ { shard := memory.Shards[i] go func(s *memory.Shard) { ticker := time.NewTicker(100 * time.Millisecond) for range ticker.C { s.EvictRandomKeys() s.EvictKeysByLRU() } }(shard) } l, err := net.Listen("tcp", *hostname+":"+*port) if err != nil { log.Fatalf("Failed to listen on %s:%s: %v", *hostname, *port, err) } log.Printf("SkylerRedis server started on %s:%s", *hostname, *port) for { conn, err := l.Accept() if err != nil { log.Printf("Error accepting connection: %v", err) continue } el.Add(conn) } } // Placeholder for intServer function if it exists elsewhere func intServer(replicaOfStr *string, port *string) interface{} { // Dummy implementation return nil } ``` -------------------------------- ### Redis Streams XRANGE – Start Exclusive Boundary Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Retrieves entries from a Redis Stream starting strictly after a given ID. The '(' prefix denotes an exclusive start boundary. ```redis XRANGE s (0-1 + ``` -------------------------------- ### Verify Replication with redis-cli Source: https://github.com/manhhodinh/skylerredis/blob/main/skyler_redis_summary_en.md This section demonstrates how to verify master-slave replication using the `redis-cli` tool. It involves writing a key-value pair to the master and then reading that key from each slave to confirm that the data has been successfully replicated. ```shell # Write to Master redis-cli -p 6379 SET message "hello from master" # Read from Slaves # Check slave 1 redis-cli -p 6380 GET message # Expected output: "hello from master" # Check slave 2 redis-cli -p 6381 GET message # Expected output: "hello from master" ``` -------------------------------- ### Initialize Go Module Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/01_tcp_server_en.md Initializes a new Go module for the SkylerRedis project. This command creates a `go.mod` file, which defines the project's module path and dependencies, essential for managing Go packages. ```sh go mod init github.com/ManhHoDinh/SkylerRedis ``` -------------------------------- ### Redis Hashes HGET Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Gets the value of a hash field. If the field or hash does not exist, nil is returned. ```redis HGET h f ``` -------------------------------- ### Inefficient Goroutine-per-Connection Model in Go Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/02_io_multiplexing_en.md This Go code snippet illustrates a common but inefficient approach to handling multiple client connections by spawning a new goroutine for each incoming connection. This model suffers from high memory usage and scheduler overhead, making it unsuitable for high-performance applications. ```go // Inefficient model for { conn, _ := ln.Accept() go handleConnection(conn) // New goroutine for every client } ``` -------------------------------- ### Redis Streams XREAD - Basic Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Reads entries from one or more Redis Streams starting from a specific ID. This command retrieves available entries immediately. ```redis XREAD STREAMS mystream 0-0 ``` -------------------------------- ### Redis Streams XREAD – Multiple Streams Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Reads entries from multiple Redis Streams concurrently, specifying starting IDs for each stream. This allows coordinated consumption. ```redis XREAD STREAMS s1 s2 0-0 0-0 ``` -------------------------------- ### Test TCP Server Connection Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/01_tcp_server_en.md Tests the basic TCP server's ability to accept connections. This command uses `telnet` to connect to the server running on `localhost` at port `6379`. The server is expected to accept and immediately close the connection. ```sh telnet localhost 6379 ``` -------------------------------- ### Connecting to SkylerRedis with redis-cli Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md Demonstrates how to connect to a running SkylerRedis instance using the standard redis-cli tool. Assumes the server is running on localhost:6380. ```shell redis-cli -h 127.0.0.1 -p 6380 ``` -------------------------------- ### Corrupted RDB Handling Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the server's behavior when encountering a corrupted or truncated RDB file. Verifies that the server refuses to start or handles the error gracefully. ```APIDOC ## Corrupted RDB Handling ### Description Tests the server's behavior when encountering a corrupted or truncated RDB file. Verifies that the server refuses to start or handles the error gracefully. ### Method Server Startup ### Endpoint N/A (Server startup) ### Parameters N/A ### Request Example N/A ### Response #### Error Response - Expected: Server refuses to start or RDB load aborted. ``` -------------------------------- ### Untitled Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md No description -------------------------------- ### Go - Basic String SET command Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md Illustrates setting a string key-value pair in SkylerRedis. This is a fundamental operation for string storage. The value can be a number or a word. ```go // Example of SET command usage in Go client (conceptual): // client.Do("SET", "mykey", "myvalue") ``` -------------------------------- ### Redis Streams XGROUP CREATE – Create Group Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Creates a new consumer group for a Redis Stream, starting consumption from a specific stream ID (e.g., '0-0' for the beginning). ```redis XGROUP CREATE s g 0-0 ``` -------------------------------- ### Redis Streams XREAD – Simple Read from Specific ID Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Reads entries from a specified Redis Stream starting from ID '0-0'. This is a basic retrieval of available stream data. ```redis XREAD STREAMS s 0-0 ``` -------------------------------- ### Multiple Keys Load Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the RDB loading mechanism when the RDB file contains a large number of keys. Verifies that all keys are loaded correctly and are accessible. ```APIDOC ## Multiple Keys Load ### Description Tests the RDB loading mechanism when the RDB file contains a large number of keys. Verifies that all keys are loaded correctly and are accessible. ### Method Internal RDB Loading / GET ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - Any key expected to be in the RDB. ### Request Example ``` GET any_key_from_rdb ``` ### Response #### Success Response (200) - **value** (string) - The value associated with the key. #### Response Example ```json { "value": "value" } ``` ``` -------------------------------- ### Redis Replication Handshake Commands Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Demonstrates the initial commands exchanged during the Redis replication handshake between a replica and a master. This includes the PING command, REPLCONF commands for sending the listening port and capabilities, and the PSYNC command for full resynchronization. ```redis-cli PING ``` ```redis-cli REPLCONF listening-port 6380 ``` ```redis-cli REPLCONF capa eof ``` ```redis-cli PSYNC ? -1 ``` -------------------------------- ### Go Master-Replica Replication for Command Propagation Source: https://context7.com/manhhodinh/skylerredis/llms.txt Implements master-slave replication where the master server propagates data-modifying commands to connected replicas. The master generates a unique replication ID and tracks the replication offset. It manages a list of connected slaves and ensures commands are sent to them. Dependencies include 'sync' and custom 'entity' and 'utils' packages. It accepts a network connection and port for registering replicas, and propagates string slices representing commands. ```go // internal/server/master.go type Master struct { *entity.BaseServer MasterReplID string MasterReplOffset int Slaves []entity.BaseServer SlavesMu sync.Mutex } func (m *Master) RegisterSlave(conn net.Conn, port string) error { remoteHost, _, _ := net.SplitHostPort(conn.RemoteAddr().String()) slaveAddr := utils.FormatAddr(remoteHost, port) m.SlavesMu.Lock() defer m.SlavesMu.Unlock() slave := entity.BaseServer{ Conn: conn, Addr: slaveAddr, } m.Slaves = append(m.Slaves, slave) return nil } func (m *Master) PropagateCommand(args []string) { resp := utils.ArgsToRESP(args) respBytes := []byte(resp) m.SlavesMu.Lock() defer m.SlavesMu.Unlock() for i := len(m.Slaves) - 1; i >= 0; i-- { slave := m.Slaves[i] n, err := slave.Conn.Write(respBytes) if err != nil { m.Slaves = append(m.Slaves[:i], m.Slaves[i+1:]...) continue } m.MasterReplOffset += n } } // Usage: // Master: ./your_program.sh --port=6380 // Replica: ./your_program.sh --port=6381 --replicaof="127.0.0.1 6380" ``` -------------------------------- ### Redis Replication INFO Command Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Validates the 'INFO replication' command to determine the role of a Redis server. This snippet shows the expected output for a master and how a slave's role is indicated when started with the '--replicaof' option. ```redis-cli *2 $4 INFO $11 replication ``` -------------------------------- ### LPUSH Command - Prepend Values to List (Go) Source: https://context7.com/manhhodinh/skylerredis/llms.txt Adds one or more values to the head (left side) of a list associated with a key. If the list does not exist, it is created. This function also handles waking up any clients blocked on BLPOP operations for this list. It implements the Redis LPUSH command. ```go // internal/command/lpush.go func (LPush) Handle(Conn net.Conn, args []string, isMaster bool, masterReplID string, masterReplOffset int, connectedSlaves int, shard *memory.Shard) { key := args[1] shard.Mu.Lock() defer shard.Mu.Unlock() for i := 2; i < len(args); i++ { shard.RPush[key] = append([]string{args[i]}, shard.RPush[key]...) } utils.WriteInteger(Conn, len(shard.RPush[key])) wakeUpFirstBlocking(key) } // Usage with redis-cli: // LPUSH mylist "world" // LPUSH mylist "hello" // Response: :2\r\n // List now contains: ["hello", "world"] ``` -------------------------------- ### Redis Lists LPUSH / LPOP Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Demonstrates pushing elements to the head of a list (LPUSH) and popping elements from the head (LPOP). ```redis LPUSH l a ``` -------------------------------- ### SET PX Expiry Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `SET` command with the `PX` option to set a key with a specified expiry time in milliseconds. Includes happy path and invalid value scenarios. ```APIDOC ## SET PX Expiry ### Description Tests the `SET` command with the `PX` option to set a key with a specified expiry time in milliseconds. Includes happy path and invalid value scenarios. ### Method POST ### Endpoint `/set` ### Parameters #### Request Body - **key** (string) - Required - The key to set. - **value** (string) - Required - The value to set. - **PX** (integer) - Optional - Expiry time in milliseconds. - **invalid_px** (integer) - Optional - For testing invalid PX values. ### Request Example (Happy Path) ```json { "command": "SET", "key": "k", "value": "v", "PX": 100 } ``` ### Response #### Success Response (200) - **status** (string) - Indicates success, typically '+OK'. #### Response Example (Happy Path) ``` +OK ``` #### Error Response (Invalid PX Value) - **error** (string) - Error message for invalid PX value. #### Response Example (Invalid PX Value) ``` -ERR PX value is not an integer or out of range ``` ``` -------------------------------- ### RDB String Key Load Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the loading of RDB files containing string keys and values. Verifies that these keys are available after a server restart. ```APIDOC ## RDB String Key Load ### Description Tests the loading of RDB files containing string keys and values. Verifies that these keys are available after a server restart. ### Method Internal RDB Loading / GET ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - The key to retrieve. ### Request Example ``` GET key ``` ### Response #### Success Response (200) - **value** (string) - The value associated with the string key. #### Response Example ```json { "value": "value" } ``` ``` -------------------------------- ### Naive Goroutine-per-Connection Model in Go Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/05_concurrency_model_en.md Illustrates the straightforward but inefficient 'goroutine-per-connection' approach for network servers in Go. This model spawns a new goroutine for every incoming client connection, leading to high memory consumption and scheduler overhead at scale. ```go // The old, naive model for { conn, _ := listener.Accept() go handleConnection(conn) // New goroutine for every client } ``` -------------------------------- ### Go - Basic List RPUSH command Source: https://github.com/manhhodinh/skylerredis/blob/main/README.md Demonstrates pushing an element to the right of a list in SkylerRedis. This operation is part of the List data storage functionality. ```go // Example of RPUSH command usage in Go client (conceptual): // client.Do("RPUSH", "mylist", "element1") ``` -------------------------------- ### Redis Transactions Queue Commands Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Demonstrates queuing commands within a Redis transaction. Each command executed after MULTI and before EXEC will be queued. ```redis SET a 1 ``` -------------------------------- ### PSYNC Handshake - Full Resync Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `PSYNC ? -1` command, initiating a full resynchronization between master and replica. Expects a `+FULLRESYNC` reply with replication ID and offset. ```APIDOC ## PSYNC Handshake - Full Resync ### Description Tests the `PSYNC ? -1` command, initiating a full resynchronization between master and replica. Expects a `+FULLRESYNC` reply with replication ID and offset. ### Method POST ### Endpoint `/psync` ### Parameters #### Request Body - **replicationid** (string) - Required - `?` for initial sync. - **offset** (integer) - Required - `-1` for initial sync. ### Request Example ``` PSYNC ? -1 ``` ### Response #### Success Response (200) - **reply** (string) - `+FULLRESYNC 0`. #### Response Example ``` +FULLRESYNC 0 ``` ``` -------------------------------- ### Test Connection (PING) - Go Source: https://context7.com/manhhodinh/skylerredis/llms.txt Checks server connectivity and responsiveness. It echoes back any provided argument or returns 'PONG' if no argument is given. This command is useful for keep-alive checks and latency measurements. It handles potential argument count errors. ```Go import ( "net" "github.com/manhhodinh/skylerredis/internal/memory" "github.com/manhhodinh/skylerredis/internal/utils" ) // Ping command handler func (Ping) Handle(Conn net.Conn, args []string, isMaster bool, masterReplID string, masterReplOffset int, connectedSlaves int, shard *memory.Shard) { if len(args) > 2 { utils.WriteError(Conn, "wrong number of arguments for 'ping' command") return } if len(args) == 2 { utils.WriteSimpleString(Conn, args[1]) } else { utils.WriteSimpleString(Conn, "PONG") } } ``` -------------------------------- ### Redis RDB Persistence and Key-Value Operations Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests related to loading and verifying data stored in Redis's RDB (Redis Database) format. This includes simple key-value pairs, expiry times in seconds and milliseconds, database selection, and handling of end-of-file markers and corrupted files. It also covers persistence end-to-end testing, ensuring data is restored after server restarts. ```redis-cli GET key ``` ```redis-cli SET a b SHUTDOWN RESTART GET a ``` ```redis-cli *5 $3 SET $1 k $1 v $2 PX $3 100 ``` ```redis-cli SET k v PX -1 ``` ```redis-cli SET k v PX 50 ``` ```redis-cli SET k v PX 100 ``` ```redis-cli *3 $6 CONFIG $3 GET $3 dir ``` ```redis-cli CONFIG GET dbfilename ``` -------------------------------- ### Persistence End-to-End Restore Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the end-to-end persistence and restore mechanism. It involves setting a key, shutting down the server, restarting it, and then retrieving the key to verify it was restored correctly. ```APIDOC ## Persistence End-to-End Restore ### Description Tests the end-to-end persistence and restore mechanism. It involves setting a key, shutting down the server, restarting it, and then retrieving the key to verify it was restored correctly. ### Method Various (SET, SHUTDOWN, RESTART, GET) ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - The key to set and retrieve. - **value** (string) - Required - The value to set for the key. ### Request Example ``` SET a b SHUTDOWN RESTART GET a ``` ### Response #### Success Response (GET a) - **value** (string) - The value associated with the key 'a', expected to be 'b'. #### Response Example ``` $1 b ``` ``` -------------------------------- ### RDB SELECTDB Opcode Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `0xFE` opcode for switching databases during RDB loading. Verifies that subsequent keys are loaded into the correctly selected database. ```APIDOC ## RDB SELECTDB Opcode ### Description Tests the `0xFE` opcode for switching databases during RDB loading. Verifies that subsequent keys are loaded into the correctly selected database. ### Method Various (Internal RDB loading) ### Endpoint N/A (Internal RDB parsing) ### Parameters N/A ### Request Example N/A ### Response N/A (Internal RDB parsing behavior) #### Notes - Opcode `0xFE`: Switch DB index. ``` -------------------------------- ### Replica Handshake - REPLCONF listening-port Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `REPLCONF listening-port` command used by a replica to inform the master of its listening port during the handshake. ```APIDOC ## Replica Handshake - REPLCONF listening-port ### Description Tests the `REPLCONF listening-port` command used by a replica to inform the master of its listening port during the handshake. ### Method POST ### Endpoint `/replconf` ### Parameters #### Request Body - **command** (string) - Required - `REPLCONF`. - **option** (string) - Required - `listening-port`. - **port** (integer) - Required - The replica's listening port. ### Request Example ``` REPLCONF listening-port 6380 ``` ### Response #### Success Response (200) - **reply** (string) - Expected to be `+OK`. #### Response Example ``` +OK ``` ``` -------------------------------- ### Go Event Loop with Epoll for I/O Multiplexing Source: https://context7.com/manhhodinh/skylerredis/llms.txt Manages concurrent network connections using Linux's epoll interface for efficient I/O multiplexing. It registers connections in edge-triggered mode and processes events via callbacks when data is available. Dependencies include the 'net' and 'golang.org/x/sys/unix' packages. It takes a read callback function as input and returns an EventLoop instance. ```go // internal/eventloop/eventloop.go func New(readCallback func(conn net.Conn) error) (*EventLoop, error) { epfd, err := unix.EpollCreate1(0) if err != nil { return nil, err } return &EventLoop{ epfd: epfd, connections: make(map[int]net.Conn), ReadCallback: readCallback, }, nil } func (el *EventLoop) Add(conn net.Conn) error { fd, err := connToFileDescriptor(conn) if err != nil { return err } if err := unix.SetNonblock(fd, true); err != nil { return err } el.connections[fd] = conn event := &unix.EpollEvent{ Events: unix.EPOLLIN | unix.EPOLLET, Fd: int32(fd), } return unix.EpollCtl(el.epfd, unix.EPOLL_CTL_ADD, fd, event) } func (el *EventLoop) Start() { events := make([]unix.EpollEvent, 1024) for { n, err := unix.EpollWait(el.epfd, events, -1) if err != nil { if err == unix.EINTR { continue } continue } for i := 0; i < n; i++ { fd := int(events[i].Fd) conn, ok := el.connections[fd] if !ok { continue } if events[i].Events&(unix.EPOLLIN|unix.EPOLLHUP|unix.EPOLLERR) != 0 { if el.ReadCallback != nil { if err := el.ReadCallback(conn); err != nil { el.Remove(conn) } } } } } } // Usage in main.go: // readCallback := func(conn net.Conn) error { // reader := bufio.NewReader(conn) // args, err := utils.ParseArgs(conn, reader) // if err != nil { // return err // } // command.HandleCommand(conn, args, isMaster, masterReplID, masterReplOffset, connectedSlaves, shard) // return nil // } // el, _ := eventloop.New(readCallback) // go el.Start() ``` -------------------------------- ### Redis Replication Error Handling and Resync Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Covers various aspects of Redis replication, including handling partial resync rejections when a full resync is required, simulating replica disconnections and reconnections, and managing multiple replicas connected to a single master. ```redis-cli PSYNC replid 100 ``` -------------------------------- ### Count-Min Sketch Query Command (Go) Source: https://github.com/manhhodinh/skylerredis/blob/main/docs/06_probabilistic_ds_en.md Queries the estimated count for an item in a Count-Min Sketch. Returns the estimated count or 0 if the sketch or item does not exist. Implemented internally within SkylerRedis. ```Go package memory // Sketch represents a Count-Min Sketch data structure. type Sketch struct { depth uint64 width uint64 table [][]uint64 } var CountMinSketches = make(map[string]*Sketch) // CMSQUERY returns the estimated count for item in the Count-Min Sketch named key. // Returns 0 if the sketch or item does not exist. func CMSQUERY(key string, item []byte) uint64 { s, exists := CountMinSketches[key] if !exists { return 0 } return s.Query(item) } // Query estimates the frequency of the given item. func (s *Sketch) Query(item []byte) uint64 { hashes := s.getHashes(item) minCount := ^uint64(0) // Initialize with max uint64 value for i, h := range hashes { count := s.table[i][h%s.width] if count < minCount { minCount = count } } return minCount } // Placeholder for actual hash generation logic using hash/fnv func (s *Sketch) getHashes(item []byte) []uint64 { // This is a simplified representation. Actual implementation would use hash/fnv // to generate 's.depth' different hash values. hashes := make([]uint64, s.depth) for i := uint64(0); i < s.depth; i++ { hashes[i] = uint64(i) + 1 // Dummy hash for example } return hashes } ``` -------------------------------- ### WAIT Command Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the `WAIT` command, which makes the master wait until a specified number of replicas acknowledge a certain replication offset. Includes happy path and timeout scenarios. ```APIDOC ## WAIT Command ### Description Tests the `WAIT` command, which makes the master wait until a specified number of replicas acknowledge a certain replication offset. Includes happy path and timeout scenarios. ### Method GET ### Endpoint `/wait` ### Parameters #### Query Parameters - **numreplicas** (integer) - Required - The number of replicas to wait for. - **timeout** (integer) - Required - The maximum time to wait in milliseconds. ### Request Example (Happy Path) ``` WAIT 1 1000 ``` ### Response #### Success Response (Happy Path) - **acknowledged_replicas** (integer) - The number of replicas that acknowledged the offset, expected to be `1` in this case. #### Response Example (Happy Path) ``` :1 ``` ### Request Example (Timeout) ``` WAIT 2 500 # Assuming only 1 replica is available or slow ``` ### Response #### Success Response (Timeout) - **acknowledged_replicas** (integer) - The number of replicas that acknowledged the offset before the timeout, expected to be `0` or `1` if timeout occurs. #### Response Example (Timeout) ``` :0 ``` ``` -------------------------------- ### RDB Expiry (Seconds) Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests RDB persistence with expiry times set in Unix seconds. Verifies that keys expire correctly based on the second precision. ```APIDOC ## RDB Expiry (Seconds) ### Description Tests RDB persistence with expiry times set in Unix seconds. Verifies that keys expire correctly based on the second precision. ### Method Internal RDB Loading / GET ### Endpoint `/key` ### Parameters #### Path Parameters - **key** (string) - Required - The key to retrieve. ### Request Example ``` # Key set with expiry in seconds GET key ``` ### Response #### Success Response (200) - **value** (string) - The value if the key has not expired. #### Success Response (Expired) - **value** (string) - Null reply (`$-1`) if the key has expired. #### Response Example (Not Expired) ```json { "value": "value" } ``` #### Response Example (Expired) ``` $-1 ``` ``` -------------------------------- ### Replication Flow Diagram Source: https://github.com/manhhodinh/skylerredis/blob/main/design.md Details the process of replication in SkylerRedis, showing the interaction between replicas and the master for synchronization and command propagation. ```text Replica → PSYNC → Master Master → RDB snapshot → Replica Master → Command propagation → Replica ``` -------------------------------- ### RDB Opcode SELECTDB Handling Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the handling of the RDB SELECTDB opcode (`0xFE`). Verifies that subsequent keys are correctly loaded into the specified database. ```APIDOC ## RDB Opcode SELECTDB Handling ### Description Tests the handling of the RDB SELECTDB opcode (`0xFE`). Verifies that subsequent keys are correctly loaded into the specified database. ### Method Internal RDB Loading ### Endpoint N/A (Internal RDB parsing) ### Parameters N/A ### Request Example N/A ### Response N/A (Internal RDB parsing behavior) #### Notes - Opcode `0xFE`: Switch DB index. Subsequent keys are loaded into the selected DB. ``` -------------------------------- ### Corrupted RDB File Handling Source: https://github.com/manhhodinh/skylerredis/blob/main/codecrafters_redis_testcases.md Tests the server's response to a truncated RDB file. The server should fail to load the RDB safely, preventing startup with corrupted data. ```APIDOC ## Corrupted RDB File Handling ### Description Tests the server's response to a truncated RDB file. The server should fail to load the RDB safely, preventing startup with corrupted data. ### Method Server Startup ### Endpoint N/A (Server startup) ### Parameters N/A ### Request Example N/A ### Response #### Error Response - Expected: Server fails to load RDB safely. ```