Redis Persistence — The RDB Chapter
We know that the full dataset of a Redis database is stored in memory. If Redis is shut down, the in-memory data is lost. To solve this, Redis provides two persistence mechanisms: RDB and AOF. We previously analyzed the AOF chapter of Redis persistence; in this article we continue with the other mechanism, RDB.
How Redis Persistence Differs from PostgreSQL
When considering persistence for Redis, one key point is that persistence must have the lowest possible impact on Redis performance. One of Redis’s core use cases is caching, and caching demands high performance — otherwise you might as well access the relational database directly without adding a cache layer. Therefore, a major design goal of Redis’s persistence mechanism is to minimize performance impact; the persistence process should ideally not block the execution of client commands.
This differs from the persistence design of relational databases such as PostgreSQL. PostgreSQL uses WAL (Write-Ahead Logging) and a bgwriter background process to periodically flush pages from the buffer pool to disk for persistence. In PostgreSQL the full dataset lives on disk; during a query, if the page is not found in the ShareBuffer memory cache, it must be read from disk into ShareBuffer. Meanwhile, when there is not enough space in the buffer, eviction algorithms must flush ShareBuffer pages back to disk. In other words, PostgreSQL inevitably faces a large amount of disk IO during operation, because its data volume can be much larger than memory.
Redis, by contrast, keeps its full dataset in memory and does not need to read data from disk while executing commands. The purpose of its persistence is to recover data quickly on startup, thereby avoiding data loss (note that RDB cannot fully prevent data loss — “avoiding data loss” here means the data before the RDB snapshot can be preserved). So Redis persistence stores the full in-memory dataset to disk using different encodings per data type, and on startup decodes the disk data back into Redis’s various data structures.
RDB Implementation Principle
As we analyzed earlier, Redis persistence should preferably not block command execution, so it is natural to fork a child process to do the persistence work. RDB persistence is done via snapshots: when certain conditions are met, Redis automatically generates a copy of all in-memory data and stores it on disk. A snapshot is taken in the following situations:
- Executing the
saveorbgsavecommand (saveperforms the snapshot synchronously and blocks all client requests during execution;bgsaveperforms the snapshot asynchronously in the background and can continue serving client requests). - Automatic snapshot based on configured rules.
- Executing the
flushallcommand. - During replication.
The snapshot process:
- Redis uses the
forkfunction to duplicate a copy of the current process. - The parent process continues to receive and process commands from clients, while the child process begins writing the in-memory data into a temporary file on disk.
- Once the child has written all data, it replaces the old RDB file with the temporary file, completing one snapshot operation.
The snapshot principle:
When fork is executed, the operating system uses the copy-on-write (COW) strategy — at the moment fork happens, the parent and child share the same memory data. When the parent wants to modify a piece of data (e.g. executing a write command), the OS copies that piece of data to keep the child’s data unaffected. Therefore, the new RDB file stores the memory data as of the moment fork was executed.
After Redis starts, it reads the RDB snapshot file and loads the data from disk into memory. With RDB persistence, if Redis exits abnormally, all data changed after the last snapshot is lost.
Source Code Analysis
Let’s look at the RDB-related source code to see how writing an RDB file and recovering data by reading an RDB file are actually implemented.
The RDB Write Process
An RDB file can be generated via the BGSAVE command or via configured rules. Both BGSAVE and the configured rules call the rdbSaveBackground function to generate the RDB file.
127.0.0.1:6379> bgsave # command approach
Background saving startedCall chain:
bgsaveCommand(client *c) // execute the BGSAVE command
--> rdbSaveBackground(server.rdb_filename,rsiptr)
--> if ((childpid = redisFork(CHILD_TYPE_RDB)) == 0) // fork a child process
{
rdbSave(filename,rsi);
}An RDB file can also be triggered by configured rules, which are set in redis.conf with the following format:
# save <seconds> <changes>
save 3600 1 # take a snapshot if at least 1 key is modified within 3600 seconds
save 300 100 # take a snapshot if at least 100 keys are modified within 300 seconds
save 60 10000 # take a snapshot if at least 10000 keys are modified within 60 secondsA periodic task checks whether the configured rules are satisfied, and if so calls rdbSaveBackground to generate the RDB file.
Call chain:
serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData)
--> rdbSaveBackground(server.rdb_filename,rsiptr); // fork a child to generate the RDB file
--> rdbSave(filename,rsi); // perform the RDB file write
--> rdbSaveRio(&rdb,&error,RDBFLAGS_NONE,rsi)
--> rdbSaveKeyValuePair(rdb,&key,o,expire) // write the key-value pair to the RDB file
--> rdbSaveObjectType(rdb,val) // write type, key, value
--> rdbSaveStringObject(rdb,key)
--> rdbSaveObject(rdb,val,key) // process data according to the data typeLet’s look at the implementation of the rdbSave function. When writing the RDB file, a temporary file temp-pid.rdb must be created first — we cannot overwrite the original RDB file directly. If this run fails, it would corrupt the previously good RDB file, which is unacceptable. So we create a temporary file, write the data into it, and once the write completes, replace the original RDB file with the temporary one.
int rdbSave(char *filename, rdbSaveInfo *rsi)
{
char tmpfile[256];
snprintf(tmpfile,256,"temp-%d.rdb", (int) getpid()); // create a temporary file
FILE *fp = fopen(tmpfile,"w"); // open the file in write mode
rioInitWithFile(&rdb,fp); // initialize the rio struct; rio is Redis's wrapper for file operations, enabling convenient read/write
startSaving(RDBFLAGS_NONE);
// serialize the in-memory data and write it into the RDB file
if (rdbSaveRio(&rdb,&error,RDBFLAGS_NONE,rsi) == C_ERR) {
errno = error;
goto werr;
}
// force flush to disk
if (fflush(fp)) goto werr;
if (fsync(fileno(fp))) goto werr;
if (fclose(fp)) { fp = NULL; goto werr; }
rename(tmpfile,filename) == -1; // replace the original RDB file with the temporary file
}int rdbSaveRio(rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi)
{
char magic[10];
snprintf(magic,sizeof(magic),"REDIS%04d",RDB_VERSION);
if (rdbWriteRaw(rdb,magic,9) == -1) goto werr; // write REDIS and the RDB version RDB_VERSION, marking this as an RDB file and its version
// iterate over all databases and write their data into the RDB file
for (j = 0; j < server.dbnum; j++) {
redisDb *db = server.db+j;
dict *d = db->dict;
if (dictSize(d) == 0) continue; // skip empty databases
// get an iterator over the key-value pairs in the database; this actually traverses the hash table
// traversing the hash table means walking the hash-table array and the linked lists built to resolve collisions
dictIterator *di = dictGetSafeIterator(d);
// write the SELECT-DB opcode and the database number, equivalent to "select db"
if (rdbSaveType(rdb,RDB_OPCODE_SELECTDB) == -1) goto werr;
if (rdbSaveLen(rdb,j) == -1) goto werr;
// write the RESIZE-DB opcode and the database size, adjusting the hash table size in the dict so no further resizing is needed
uint64_t db_size, expires_size;
db_size = dictSize(db->dict);
expires_size = dictSize(db->expires);
if (rdbSaveType(rdb,RDB_OPCODE_RESIZEDB) == -1) goto werr;
if (rdbSaveLen(rdb,db_size) == -1) goto werr;
if (rdbSaveLen(rdb,expires_size) == -1) goto werr;
// iterate over all key-value pairs in the database and write them into the RDB file
while((de = dictNext(di)) != NULL) {
sds keystr = dictGetKey(de);
robj key, *o = dictGetVal(de);
long long expire;
initStaticStringObject(key,keystr);
expire = getExpire(db,&key); // get the key's expiration time
// write the key-value pair and its metadata (expiration time, LRU/LFU info)
// data is processed according to the different data types
if (rdbSaveKeyValuePair(rdb,&key,o,expire) == -1) goto werr;
// mixed persistence
/* When this RDB is produced as part of an AOF rewrite, move
* accumulated diff from parent to child while rewriting in
* order to have a smaller final write. */
if (rdbflags & RDBFLAGS_AOF_PREAMBLE &&
rdb->processed_bytes > processed+AOF_READ_DIFF_INTERVAL_BYTES)
{
processed = rdb->processed_bytes;
aofReadDiffFromParent();
}
/* Update child info every 1 second (approximately).
* in order to avoid calling mstime() on each iteration, we will
* check the diff every 1024 keys */
if ((key_count++ & 1023) == 0) {
long long now = mstime();
if (now - info_updated_time >= 1000) {
sendChildInfo(CHILD_INFO_TYPE_CURRENT_INFO, key_count, pname);
info_updated_time = now;
}
}
}
dictReleaseIterator(di);
di = NULL;
}
// write the EOF end marker
if (rdbSaveType(rdb,RDB_OPCODE_EOF) == -1) goto werr;
// compute the final CRC64 checksum and write it
cksum = rdb->cksum;
memrev64ifbe(&cksum);
if (rioWrite(rdb,&cksum,8) == 0) goto werr;
return C_OK;
}As you can see, the RDB file follows certain encoding rules — otherwise how would it be parsed when read? Let’s look at the RDB file structure:
+-------+-------------+-----------+-----------------+-----+-----------+
| REDIS | RDB-VERSION | SELECT-DB | KEY-VALUE-PAIRS | EOF | CHECK-SUM |
+-------+-------------+-----------+-----------------+-----+-----------+
|<-------- DB-DATA ---------->|Encoding notes:
REDISidentifies an RDB file; it begins withREDIS.RDB_VERSIONis the RDB version number. RDB files of different versions are incompatible; when reading an RDB file you must choose the appropriate loading method based on the version.SELECT-DBindicates which database the followingKEY-VALUE_PAIRSbelong to; when restoring data from RDB, this value is used to switch databases.KEY-VALUE-PAIRSholds the key-value information, including the key’s expiration time, LRU/LFU info, and the key-value data (data type, key, value). The value portion is encoded differently depending on the data type.EOFmarks the end of the file.CHECK-SUMis the checksum, used to verify whether the file is corrupted.
The key-value pair portion is encoded as | type | key | value |.
Different data types use different encodings. Taking the hash and set types as examples:
For the set type, the OBJ_ENCODING_HT encoding structure is:
+----------+-----------+-----------+-----+-----------+
| SET-SIZE | ELEMENT-1 | ELEMENT-2 | ... | ELEMENT-N |
+----------+-----------+-----------+-----+-----------+For the hash type, the OBJ_ENCODING_HT encoding structure is:
+-----------+-------+---------+-------+---------+-----+-------+---------+
| HASH-SIZE | KEY-1 | VALUE-1 | KEY-2 | VALUE-2 | ... | KEY-N | VALUE-N |
+-----------+-------+---------+-------+---------+-----+-------+---------+The specific code is shown below. Since it is lengthy, only the key parts are listed:
ssize_t rdbSaveObject(rio *rdb, robj *o, robj *key) {
ssize_t n = 0, nwritten = 0;
if (o->type == OBJ_STRING) { // string type
/* Save a string value */
if ((n = rdbSaveStringObject(rdb,o)) == -1) return -1;
nwritten += n;
} else if (o->type == OBJ_LIST) { // list type
/* Save a list value */
if (o->encoding == OBJ_ENCODING_QUICKLIST) {
// ...
}
} else if (o->type == OBJ_SET) { // set type
/* Save a set value */
if (o->encoding == OBJ_ENCODING_HT) {
dict *set = o->ptr;
dictIterator *di = dictGetIterator(set);
dictEntry *de;
if ((n = rdbSaveLen(rdb,dictSize(set))) == -1) {
dictReleaseIterator(di);
return -1;
}
nwritten += n;
while((de = dictNext(di)) != NULL) {
sds ele = dictGetKey(de);
if ((n = rdbSaveRawString(rdb,(unsigned char*)ele,sdslen(ele)))
== -1)
{
dictReleaseIterator(di);
return -1;
}
nwritten += n;
}
dictReleaseIterator(di);
} else if (o->encoding == OBJ_ENCODING_INTSET) {
size_t l = intsetBlobLen((intset*)o->ptr);
if ((n = rdbSaveRawString(rdb,o->ptr,l)) == -1) return -1;
nwritten += n;
}
} else if (o->type == OBJ_ZSET) { // sorted set type
/* Save a sorted set value */
if (o->encoding == OBJ_ENCODING_ZIPLIST) {
size_t l = ziplistBlobLen((unsigned char*)o->ptr);
if ((n = rdbSaveRawString(rdb,o->ptr,l)) == -1) return -1;
nwritten += n;
} else if (o->encoding == OBJ_ENCODING_SKIPLIST) { // skip list
// ...
}
} else if (o->type == OBJ_HASH) { // hash type
/* Save a hash value */
if (o->encoding == OBJ_ENCODING_ZIPLIST) {
size_t l = ziplistBlobLen((unsigned char*)o->ptr);
if ((n = rdbSaveRawString(rdb,o->ptr,l)) == -1) return -1;
nwritten += n;
} else if (o->encoding == OBJ_ENCODING_HT) { // hash table
dictIterator *di = dictGetIterator(o->ptr);
dictEntry *de;
if ((n = rdbSaveLen(rdb,dictSize((dict*)o->ptr))) == -1) {
dictReleaseIterator(di);
return -1;
}
nwritten += n;
while((de = dictNext(di)) != NULL) {
sds field = dictGetKey(de);
sds value = dictGetVal(de);
if ((n = rdbSaveRawString(rdb,(unsigned char*)field,
sdslen(field))) == -1)
{
dictReleaseIterator(di);
return -1;
}
nwritten += n;
if ((n = rdbSaveRawString(rdb,(unsigned char*)value,
sdslen(value))) == -1)
{
dictReleaseIterator(di);
return -1;
}
nwritten += n;
}
dictReleaseIterator(di);
}
} else if (o->type == OBJ_STREAM) {
// ...
} else if (o->type == OBJ_MODULE) {
// ...
}
return nwritten;
}Recovering Data from the RDB File
Recovering data is the reverse of writing — reconstructing the original data structures and context information.
main(int argc, char **argv)
--> initServer(); // initialize the server, create databases, etc.
--> loadDataFromDisk(); // load (RDB, AOF file) data
{
if (server.aof_state == AOF_ON)
loadAppendOnlyFile(server.aof_filename); // replay the AOF log file to recover data
else
rdbLoad(server.rdb_filename,&rsi,RDBFLAGS_NONE); // load the RDB file to recover data
}
--> aeMain(server.el); // event loop, handling client requestsLet’s look specifically at the rdbLoad function:
int rdbLoad(char *filename, rdbSaveInfo *rsi, int rdbflags) {
FILE *fp;
rio rdb;
int retval;
// open the RDB file
if ((fp = fopen(filename,"r")) == NULL) return C_ERR;
startLoadingFile(fp, filename,rdbflags);
rioInitWithFile(&rdb,fp);
retval = rdbLoadRio(&rdb,rdbflags,rsi); // recover data
fclose(fp);
stopLoading(retval==C_OK);
return retval;
}
int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi) {
uint64_t dbid;
int type, rdbver;
redisDb *db = server.db+0;
char buf[1024];
int error;
long long empty_keys_skipped = 0, expired_keys_skipped = 0, keys_loaded = 0;
rdb->update_cksum = rdbLoadProgressCallback;
rdb->max_processing_chunk = server.loading_process_events_interval_bytes;
// read the file header, check the RDB version and the REDIS identifier
if (rioRead(rdb,buf,9) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
serverLog(LL_WARNING,"Wrong signature trying to load DB from file");
errno = EINVAL;
return C_ERR;
}
rdbver = atoi(buf+5);
if (rdbver < 1 || rdbver > RDB_VERSION) {
serverLog(LL_WARNING,"Can't handle RDB format version %d",rdbver);
errno = EINVAL;
return C_ERR;
}
/* Key-specific attributes, set by opcodes before the key type. */
long long lru_idle = -1, lfu_freq = -1, expiretime = -1, now = mstime();
long long lru_clock = LRU_CLOCK();
while(1) { // process opcodes and recover data
sds key;
robj *val;
/* Read type. */ // read the opcode or the object type
if ((type = rdbLoadType(rdb)) == -1) goto eoferr;
/* Handle special types. */ // handle special opcodes
if (type == RDB_OPCODE_EXPIRETIME) { // read expiration time
/* EXPIRETIME: load an expire associated with the next key
* to load. Note that after loading an expire we need to
* load the actual type, and continue. */
expiretime = rdbLoadTime(rdb);
expiretime *= 1000;
if (rioGetReadError(rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_EXPIRETIME_MS) { // read millisecond-precision expiration time
/* EXPIRETIME_MS: milliseconds precision expire times introduced
* with RDB v3. Like EXPIRETIME but no with more precision. */
expiretime = rdbLoadMillisecondTime(rdb,rdbver);
if (rioGetReadError(rdb)) goto eoferr;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_FREQ) { // read LFU frequency
/* FREQ: LFU frequency. */
uint8_t byte;
if (rioRead(rdb,&byte,1) == 0) goto eoferr;
lfu_freq = byte;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_IDLE) { // read LRU idle time
/* IDLE: LRU idle time. */
uint64_t qword;
if ((qword = rdbLoadLen(rdb,NULL)) == RDB_LENERR) goto eoferr;
lru_idle = qword;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_EOF) { // end of file
/* EOF: End of file, exit the main loop. */
break;
} else if (type == RDB_OPCODE_SELECTDB) { // select database
/* SELECTDB: Select the specified database. */
if ((dbid = rdbLoadLen(rdb,NULL)) == RDB_LENERR) goto eoferr;
if (dbid >= (unsigned)server.dbnum) {
serverLog(LL_WARNING,
"FATAL: Data file was created with a Redis "
"server configured to handle more than %d "
"databases. Exiting\n", server.dbnum);
exit(1);
}
db = server.db+dbid; // select the database, equivalent to "select db"
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_RESIZEDB) { // resize the hash tables in the dict
/* RESIZEDB: Hint about the size of the keys in the currently
* selected data base, in order to avoid useless rehashing. */
uint64_t db_size, expires_size;
if ((db_size = rdbLoadLen(rdb,NULL)) == RDB_LENERR)
goto eoferr;
if ((expires_size = rdbLoadLen(rdb,NULL)) == RDB_LENERR)
goto eoferr;
dictExpand(db->dict,db_size);
dictExpand(db->expires,expires_size);
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_AUX) {
// handle auxiliary info, parse AUX info such as memory usage
continue; /* Read type again. */
} else if (type == RDB_OPCODE_MODULE_AUX) {
// load special data for custom modules
}
/* Read key */ // read the key
if ((key = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL)) == NULL)
goto eoferr;
/* Read value */ // read the value, reconstructing the data structure for each data type
val = rdbLoadObject(type,rdb,key,&error);
/* Set the expire time if needed */
if (expiretime != -1) { // set the expiration time
setExpire(NULL,db,&keyobj,expiretime);
}
/* Set usage information (for eviction). */ // set usage info (for eviction)
objectSetLRUOrLFU(val,lfu_freq,lru_idle,lru_clock,1000);
// ...
}
return C_OK;
}Recovering the data structures for different data types is done via the rdbLoadObject function. This function is very long, so only a portion is excerpted here:
robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int *error) {
if (rdbtype == RDB_TYPE_STRING) {
/* Read string value */
if ((o = rdbLoadEncodedStringObject(rdb)) == NULL) return NULL;
o = tryObjectEncoding(o);
} else if (rdbtype == RDB_TYPE_LIST) {
/* Read list value */
// create a list object
// insert data
// ...
} else if (rdbtype == RDB_TYPE_SET) {
/* Read Set value */
// ...
} else if (rdbtype == RDB_TYPE_HASH) {
o = createHashObject(); // create a hash object
// choose ziplist or hash table based on the number of elements
/* Load every field and value into the ziplist */
while (o->encoding == OBJ_ENCODING_ZIPLIST && len > 0) {
// ...
}
/* Load remaining fields and values into the hash table */
while (o->encoding == OBJ_ENCODING_HT && len > 0) {
len--;
/* Load encoded strings */
if ((field = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL)) == NULL) {
decrRefCount(o);
return NULL;
}
if ((value = rdbGenericLoadStringObject(rdb,RDB_LOAD_SDS,NULL)) == NULL) {
sdsfree(field);
decrRefCount(o);
return NULL;
}
/* Add pair to hash table */ // add <field,value> into the hash table
ret = dictAdd((dict*)o->ptr, field, value);
if (ret == DICT_ERR) {
rdbReportCorruptRDB("Duplicate hash fields detected");
sdsfree(value);
sdsfree(field);
decrRefCount(o);
return NULL;
}
}
} else if // other types ...
}At this point, the in-memory data structures have been recovered from the RDB file, and Redis can now respond to client requests.
Summary
Redis provides two persistence mechanisms: RDB and AOF, which are often used together, and in cluster replication scenarios RDB and AOF are also used for node data synchronization. Users can choose the appropriate persistence method for their specific scenario. In addition to these two methods, hybrid persistence is also an option — combining the strengths of both RDB and AOF. During AOF rewrite, a child process first forks and writes the current in-memory snapshot in RDB format as the upper half of the AOF file, while the parent records all AOF commands after the rewrite begins. Once the RDB-format data is written into the upper half of the AOF file, the parent’s recorded AOF commands are appended to the lower half, and finally the old AOF file is replaced, completing the AOF rewrite.
References: