Yes, you do need to have the entire data set in memory. Fortunately, for our needs this is easily possible, we are only at 1.5 GB worth of data (about 3.5 million keys).

We do not currently use clustering or sharding. We’ve got one server for Redis, and we’ll be bringing up a slave this week as a backup. Because we were fighting high loads and needed to get a solution up ASAP, we opted to bring just a master up at the time and then the slave later on.

We’re using the hash type primarily for stats. Since everything we do is based on tags, we can create a hash for a tag with fields for each of the stats we track for that tag, then in real-time we can do HINCRBY for the items that we want to adjust the stats on. When we want to get the stats, it’s just one HGETALL call (via a function that turns it into a native array) and we can have all of the stats.

The beauty of this is twofold, first since redis is so fast, there doesn’t appear to be any additional overhead vs a standard string GET, and second, we’ve reduced latency in the interface because in just one call we’ve got all of the stats.