How Redis Executes a Command

How Redis Executes a Command

Here we analyze how Redis executes a command. Compared with PostgreSQL, Redis’s command execution flow is much simpler. After the Redis server starts, it creates an event loop that waits for command requests. Before waiting for those requests, there is actually a step where a client establishes a connection: the client connects and registers a read handler, and when a command request arrives, the corresponding handler is invoked. The handler executes the command directly—acting as the executor—with no optimizer and no query-plan generation step.

Main Flow

The main Redis flow is as follows: after startup it does some initialization work, creates the listener, creates epoll, registers listener events, then enters the event loop to wait for clients to initiate connections.

main(int argc, char **argv)
--> spt_init(argc, argv);       // initialize the process title, i.e. the name shown by the ps command
--> zmalloc_set_oom_handler(redisOutOfMemoryHandler); // set the custom handler for out-of-memory allocation failures
--> initServerConfig();  // set Redis global default configuration parameters
--> ACLInit();   // initialize the ACL (Access Control List) subsystem
--> moduleInitModulesSystem();  // initialize the module system
--> initServer();       // server-side initialization
    --> createSharedObjects();
    --> adjustOpenFilesLimit();
    --> aeCreateEventLoop(server.maxclients+CONFIG_FDSET_INCR);  // initialize the event loop
        --> aeApiCreate(eventLoop)
            --> epoll_create(1024);   // create the epoll instance
    --> listenToPort(server.port,&server.ipfd)  // create the listening socket
        --> anetTcpServer(server.neterr,port,addr,server.tcp_backlog);
            --> _anetTcpServer(err, port, bindaddr, AF_INET, backlog)
                --> socket(p->ai_family,p->ai_socktype,p->ai_protocol)
            --> anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog,0)
                --> bind(s,sa,len)
                --> listen(s, backlog)
        --> anetNonBlock(NULL,sfd->fd[sfd->count])
    --> initialize the default 16 databases
    --> aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL)  // create a periodic task handling many background jobs, e.g. expiring keys, writing the AOF log, etc.
    --> createSocketAcceptHandler(&server.ipfd, acceptTcpHandler) // set the handler for new client connections
    --> aeCreateFileEvent(server.el, server.module_blocked_pipe[0], AE_READABLE,
        moduleBlockedClientPipeReadable,NULL)
--> InitServerLast();
--> redisSetCpuAffinity(server.server_cpulist);
--> setOOMScoreAdj(-1);
--> aeMain(server.el);
    while (!eventLoop->stop)
        aeProcessEvents(eventLoop, AE_ALL_EVENTS | AE_CALL_BEFORE_SLEEP | AE_CALL_AFTER_SLEEP);
        --> eventLoop->beforesleep(eventLoop);  // called before entering the event loop
        --> aeApiPoll(eventLoop, tvp)
        --> eventLoop->aftersleep(eventLoop);  // called after entering the event loop
--> aeDeleteEventLoop(server.el);

The Redis server calls the aeApiPoll function to wait for events. When an event occurs, aeApiPoll is invoked; it calls epoll_wait, which blocks the current thread until an event happens. The call stack is as follows:

aeApiPoll(aeEventLoop * eventLoop, struct timeval * tvp) (redis\src\ae_epoll.c:93)
aeProcessEvents(int flags, aeEventLoop * eventLoop) (redis\src\ae.c:398)
aeMain(aeEventLoop * eventLoop) (redis\src\ae.c:495)
main(int argc, char ** argv) (redis\src\server.c:7553)

When a new connection arrives, the acceptTcpHandler function is called. It creates a new client connection and a client object client that holds all information related to that client—connection info, client ID, the current database, the received command buffer, the send buffer, and so on. Basically everything about the client lives in the client object. It then registers a read handler, which is invoked when a command request arrives. The call stack is as follows:

acceptTcpHandler(aeEventLoop *el, int fd, void *privdata, int mask)
--> anetTcpAccept(server.neterr, fd, cip, sizeof(cip), &cport)
    --> anetGenericAccept(err,s,(struct sockaddr*)&sa,&salen)
        --> accept(s,sa,len)
--> acceptCommonHandler(connCreateAcceptedSocket(cfd),0,cip);
    --> createClient(conn)
        --> connNonBlock(conn);
        --> connSetReadHandler(conn, readQueryFromClient);  // sets the read handler
        --> connSetPrivateData(conn, c);
        --> selectDb(c,0);    // use database 0 by default

Once the connection with the client is established, when the client sends a command to Redis—i.e., when there is readable data—the readQueryFromClient function is called: it first reads the command (a string) sent by the client, then parses it according to the RESP protocol, and finally invokes the corresponding command handler based on the parsed command. Because the RESP protocol is so simple—usually just “command + arguments” with predefined delimiters—the parsing is very straightforward, unlike PostgreSQL which needs full lexical and syntactic analysis, flex, bison, and so on.

readQueryFromClient(connection *conn)
--> connRead(c->conn, c->querybuf+qblen, readlen)  // read data from the socket into the buffer
    --> conn->type->read(conn, buf, buf_len);
--> processInputBuffer(c);    // parse the redis protocol, store the command in the client's argv array
    --> processInlineBuffer(c)   // handle inline commands and create argument objects
        --> sdsfreesplitres(argv,argc);
    --> processMultibulkBuffer(c) // convert the protocol content in c->querybuf into argument objects in c->argv
    --> processCommandAndResetClient(c)        // execute the command and return the result
        --> processCommand(c)
            --> call(c,CMD_CALL_FULL);
                --> c->cmd->proc(c);  // execute the specific command

For example, the set command:

setCommand(client *c)
--> setGenericCommand(c,flags,c->argv[1],c->argv[2],expire,unit,NULL,NULL)
    --> genericSetKey(c,c->db,key, val,flags & OBJ_KEEPTTL,1);
        --> dbAdd(db,key,val);
            --> dictAdd(db->dict, copy, val); // add an element to the target hash table
                --> dictAddRaw(d,key,NULL);
                --> dictSetVal(d, entry, val);

Understanding how set is executed also requires understanding Redis’s macro-level framework. You can refer to this excellent article Redis Data Structures from a Macro Perspective.

The RESP Protocol

Compared with PostgreSQL, the interaction between a client and Redis is command-based. Compared with SQL syntax, Redis command syntax is quite simple—just “command + arguments”. The core reason is that Redis is a key-value database, not a relational one. Redis’s most fundamental requirement is speed, so the design must be simplified. The RESP protocol was therefore designed to be simple and fast to parse—so simple that no lexical or syntactic analysis is needed, just plain string parsing. Another consideration is the AOF persistence mechanism: AOF needs to write commands to a file, so the command format must be simple and easy to parse.

All client input commands in Redis can basically be thought of as an array of strings. For example, set name zhangsan is parsed by Redis into: *3\r\n$3\r\nset\r\n$4\r\nname\r\n$8\r\nzhangsan\r\n. This design is easy to parse and easy to understand.

Let us briefly describe the RESP protocol. The interaction between a client and Redis typically uses RESP as a request-response protocol as follows:

• The client sends the command to the Redis server as an array containing only bulk strings. The first (and sometimes second) bulk string in the array is the command name. The remaining elements of the array are the command’s arguments.
• The server replies with a RESP type. The reply type is determined by the command’s implementation and possibly the client’s protocol version.

Details:

The first byte identifies the type, and the following bytes constitute the content of that type.

Data Type	First Byte
Simple string	+
Simple error	-
Integer	:
Bulk string	$
Array	*

\r\n (CRLF) is the terminator of the protocol; it always separates the parts of the protocol.

A simple string is encoded as a plus sign (+), followed by a string. The string must not contain a CR (\r) or LF (\n) character, and it terminates with CRLF (i.e., \r\n). For example, Redis’s reply for a successful command is: +OK\r\n.

An integer is encoded as a colon (:), followed by an integer, in the format: :[<+|->]<value>\r\n. The integer must be a signed 64-bit decimal integer, with an optional plus (+) or minus (-) sign. For example: :1000\r\n.

The bulk string is the most frequently used encoding, representing a single binary string, in the format: $<length>\r\n<data>\r\n. It starts with $, followed by an integer indicating the string length, then the delimiter \r\n, then the string, and finally terminated with \r\n. For example: $6\r\nfoobar\r\n.

An array is encoded as an asterisk (*), in the format: *<number-of-elements>\r\n<element-1>...<element-n>. It is followed by an integer indicating the number of elements in the array, then the delimiter \r\n, then the array’s elements, each of which is itself RESP-encoded. For example: *2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n.

For other data types, refer to the official documentation Redis Serialization Protocol Specification; we will not go into them here.

As we know, after a client and server establish a connection, there is often authentication or protocol negotiation. In Redis, this handshake and negotiation can be done with the HELLO command.

HELLO <protocol-version> optional-arguments

See HELLO for details.

Summary

Redis pursues speed, so many of its designs are built around “fast”. For example, Redis’s command execution flow has no optimizer and no query-plan generation step—it executes the command directly. This makes Redis’s command execution flow very simple and highly efficient. Most importantly, Redis’s design core revolves around memory: all data is kept in memory, which greatly simplifies Redis’s design and is the core of its high performance. A central piece of Redis is its data structures—what data structures are fast enough while also saving memory. PostgreSQL, by contrast, is designed around disk: how to store and query data on disk more efficiently. As hardware evolves—CXL, persistent memory, and so on—future database designs will change along with the underlying hardware infrastructure.