I recently created a Redis clone in C++ as part of the CodeCrafters Redis challenge. You can find my implementation here. This post is a hodgepodge of random notes and lessons I compiled together while working through the project.

Redis is an “in-memory data structure store.” This means that Redis stores data in RAM as opposed to SSD. RAM is faster than disk because it uses volatile memory, whereas SSDs need to persist data even without power, so they use flash memory, which is slower. This makes read-and-write operations much faster on Redis (which uses RAM) compared to other traditional databases/storage companies (that store on disk).

Redis utilizes RAM through its process memory. For example, in my project, running the executable spins up an OS process and all data is allocated into the process’s address space, which is backed up by RAM. This also exposes the dangers in RAM storage. Imagine if the program was accidentally stopped or my computer was turned off. Then all of the client’s data would be lost. To mitigate this, Redis’s strategy is to save copies of the data on disk. This happens through both RDB (Redis database) and AOF (append-only) files.

RDB files are snapshots of data that occur periodically. AOF files are a log of all write operations that have been sent - this allows the data to be built back up if deleted.

In addition, companies can set up multiple Redis servers under leader-follower replication. In this architecture, one server is set up as the master server. The client sends write requests to the master server, which then propagates these commands to “replica” servers. However, read requests can be handled directly from the replica servers. This provides data redundancy and can also improve read operation performance.

To start the project, I needed to learn socket programming in C++. I had some knowledge of full-stack dev in Python & Go prior to this, and in my experience with those languages, I had focused on endpoints and sending/receiving data through HTTP. So, I was a bit confused on why Redis was interacting with sockets and not just using HTTP, which surely was simpler. It turns out that Redis actually has a custom protocol called RESP! You can find the details here. But since HTTP is just a layer on top of IP/TCP that packages data in a specific way, you can technically send data in whatever way you choose. Redis does so through RESP. For a database application, it’s much more light-weight and efficient than HTTP.

Sockets in C and C++ are programmed with the client-server model. On the client side, you must: 1) create a socket using socket(); 2) connect the socket to the server’s address using connect(); 3) send and receive data. On the server side, you must: 1) create a socket using socket(); 2) bind the socket to an address using bind(); 3) listen for connections; 4) accept a connection; 5) send and receive data.

In terms of program architecture, I created separate classes for storage (database), server-state, and client-state. In main.cpp, a server socket is created with storage and server-state class instances. A thread is spun up every time a client connects with these instances passed in. One thread per client isn’t efficient at scale vs. an event loop, but it is simple and works for a toy clone. Inside the client handler code, a class instance of the client state is created. This keeps client state isolated to the thread and allows for simple auth, while client actions that impact the server state or database will be accessible to all connected clients. Since there are potential race conditions, I included a mutex within the storage instance.

Redis is open-source. The company raised a Series G in 2021 at a $2B+ valuation led by Tiger. The team has built an excellent developer ecosystem with strong adoption. The two commercial drivers are Redis Cloud and Redis Enterprise. Redis Cloud is a service where Redis manages server deployment, scaling, etc on top of AWS or other cloud providers. Redis Enterprise is for on-prem customers who need additional features/support beyond the open-source version (open-core model). I believe infra software is more resilient to AI fears and it looks like Redis is doing fine. They have started adding features to become a data layer for AI agents.