hero-image

An introduction to Redis

February 10, 2020
By John Hammink

Imagine you’re implementing a leaderboard (top 10 players list) in one part of your game’s UI, and you need a solution that smoothly handles user-visible updates on the fly. You also need to write a user session cache to handle the “sticky” login state of users connecting to different authentication services. In yet another part of your Microservices Architecture, you need to temporarily store player-level state.

This requires a simple data store that’s extremely fast, lightweight, relatively format-agnostic, and which leverages RAM.

As a developer, you’re no doubt already familiar with NoSQL data stores and the flexibility they introduce when compared to an RDBMS. ACID-compliant database solutions in the cloud, best suited for transactional data, ensure record completeness and isolation, but impose a few different costs:

  • schema inflexibility (meaning your events must adhere to the same schema as the database, or else throw an error);
  • write-time, partition, replication, and latency costs.

This simply won’t scale to a large-scale game or social network where each of several million users needs a list or unique session data updated essentially in real-time.

Think of those constantly-updating lists, counters and views on popular social media, games, shared computing and collaboration platforms; consider the constantly-changing data on Facebook statuses, top Twitter trends and LinkedIn Profile View counts.

Or consider the results counters on search engines: would something like Google results counters really work almost instantaneously for billions of users if rigid transactional mechanisms wrote and read ACID- consistent data over latent network connections for each and every user?

Fortunately, there is an open-source solution purpose-build for precisely such use cases - Redis.

In this post, we’ll look at Redis: how an in-memory key-value store (and more) handles data, the use cases it supports and how it supports them via the available Redis data types. Finally, we’ll get you started with Aiven for Redis from both the Aiven console and CLI.

What is Redis?

Redis, or Remote Dictionary Server, is an in-memory, single threaded, open-source NoSQL datastore. Written in C by Salvatore Sanfilipo and launched in 2009, its high performance makes it a favorite as a fast data store, cache, and even lightweight message broker. As of February 2020, Redis is the most popular key-value datastore, according to DB-Engines ranking.

Roughly speaking, Redis falls in the middle of a data continuum from relatively unstructured to structured data. Think of a key-value store where the key (the thing you search by) can be any binary sequence, from a string to a JPEG file’s contents.

Redis on the continuum from structured to unstructured data

Redubbed “the data structure server” by the Redis folks themselves, the value part of the key-value pair can hold strings, lists, sets, sorted sets, hashes, bitmaps, and hyperloglogs, which we’ll cover below. It’s the range of operations that these data types support that makes Redis a flexible and easy choice to spin up on-the-fly solutions for which a relational datastore would be overkill.

With these features, Redis can accommodate a range of problems that map almost directly onto Redis’ data types themselves. But you don’t need to switch to Redis; most use it as a complement to other data stores in their architectures.

As Salvatore Sanfilipo himself says, “In order to take advantage of Redis in your production environment you don’t need to switch to Redis. You can just use it in order to do new things that were not possible before, or in order to fix old problems.”

What Redis does (and doesn’t do)

You can think of Redis as an out-of-the-box cache where you can store and recall data on the fly. As it turns out, Redis is able to do more than 100k SETs (writes) and 81k GETs (reads) per second. And with frequent — and configurable — disk writes on the off-cycles, your persisted data will be available when you reinitialize.

Some programming language libraries - think of Ruby’s resque and sidekiq libraries - even use Redis lists under the hood to implement background jobs for fast data sorts.

Unlike SQL databases, Redis doesn’t implement ACID-supporting database schemas such as tables and columns. Instead, as mentioned earlier, the storage format is {key:value}, where the stored values (searched by key) can be any of the data types on which we’ll elaborate upon below. Being memory-resident, single threaded, and written in a low level language as C, it’s an excellent choice for fast changing stores and caches.

So how do Redis’s supported data types enable this super-fast execution? Redis users create data structures on the fly and run operations directly on them.

The tet data type in Redis header image for data types

To that end, Redis supports the following data types:

Keys are items which are used to uniquely identify and search for the items (values) associated with them. Since keys are binary safe, they can be human-readable text or even binary strings, just as JPEG files.

Redis value types include the following:

Strings are the simplest values one can associate with a key. Strings are good for storing anything you might think of as plaintext, like code fragments, complete HTML pages, or IP addresses. Redis allows up to a 512MB size for strings.

Sorted by order of insertion, lists are collections of strings or linked lists. Lists can contain duplicate values, and you can add items to either the beginning or end.

Sets are like lists, except they cannot contain duplicate values, and are unsorted. Sets can make use of unions, intersections and subtractions. Because they only accept unique, distinct values, they are the go-to for managing cardinality. On the other hand, sorted sets are sets where every string is ordered by a float score. In sorted sets, scores can repeat but their associated values cannot.

Hashes are maps of value-associated fields representing distinct objects. Hashes can hold up to 4 billion fields. As such, hashes offer an efficient way for a single Redis instance to hold a very large number of objects and operate on them.

Redis handles bit arrays and bitmaps the same way it handles strings: you can set and clear individual bits, find the first set (or unset) bit, or count all bits set to a specific binary value, among other things.

One data type unique to Redis is the HyperLogLog or HLL: “…a probabilistic data structure used to estimate the cardinality of a set.” HLLs themselves contain only state information and employ algorithms to consume a constant amount of memory (12k at worst) instead of an amount of memory proportional to the numbers of items counted. HLLs read and write their input data from strings, so you’d use string operations for adding, retrieving and counting items from their cardinality calculations.

Redis offers streams with a type of pub/sub pattern, which can be used as part of a mechanism to channel non-critical items to consumers or groups. And it’s possible to implement new data types or support external modules to enhance Redis’ functionality.

So how do these data types map to specific use-cases?

Based on strings

Session caches or full-page caches can store HTML fragments or even full pages and serve them based on session data. These are also an original use of memcache, and what many such users actually do in Redis. Real-time metering for utilization-based pricing can be created, based on Redis strings, lists, or even using pub/sub instances as building blocks.

Based on lists

Most recent items lists can use LPUSH or LPOP commands to populate lists, and LTRIM to limit the list sizes. Many social media sites use Redis lists for “top trending” content item feeds. News feeds, aggregated from RSS feeds, can push latest updates to restricted lists using the same methods.

Based on sets

Developers can implement sets with operations like unions, intersections and subtractions (as in Venn diagrams) to search product categories or analyze preferences by demographic like age. Content filters implemented in Redis filter user input against discrete sets of words, added to sets via SADD commands. Counters use sets with INCRBY commands to increment counts, GETSET to clear the counters and EXPIRE and/or TTL commands to delete obsolete counters. Since the SADD command can only add unique members to sets, these are also useful for unique N items computations. These latter applications are useful for logging unique users or IP addresses.

Based on sorted sets

Forums like StackOverflow or Quora use sorted sets to rank answers by number of votes. Gaming app scoreboards use sorted sets to track N highest scores; these are similar to leaderboards with top N player lists. Task queues rank tasks by priority, while expiring item lists use unix time as the key and index current time and time-to-live; the sorted set is queried with ZRANGE against scores.

Based on hashes

Since hashes store multiple fields, they are perfectly suited for objects like user profiles, user posts, and multivariate metrics like product and research and development.

Based on pub/sub

Redis’s own pub/sub can be used as a building block to construct ephemeral, non-critical channels for some types of message routing and data collection.

Some popular applications that Redis supports may use a combination of data types: for example, sticky sessions that keep users logged-in between services during authentication sessions; storing random data that many services in an MSA need access to is another example.

But that’s just the beginning! The use cases for Redis are truly limitless. And there are many good reasons why a hosted, managed Redis solution is the best way to go: Do you need automatic setup and maintenance? How about the ability to provision your Redis instance at the click of a button? 24/7 support?

The Aiven for Redis solution

Aiven for Redis is a fully-managed and hosted Redis solution that is offered globally in Google Cloud platform, Amazon Web Services, Microsoft Azure, DigitalOcean and UpCloud. Aiven for Redis builds on the premise of Redis by providing automatic setup and maintenance. You can even provision your database at the click of a button.

Getting started

Aiven for Redis is easy to set up, either on the Aiven Console

creating an aiven for redis service in the aiven console

Directly from the command line:

creating an aiven for redis service in the aiven cli

or using our REST API.

Wrapping up

In this post, we’ve looked at Redis: how it handles data, the use cases it supports, and how it supports them. And we’ve gotten started with Aiven for Redis from both Aiven Console and Aiven CLI.

If you’d like try out, risk-free hosted and managed Aiven for Redis yourself, check it out with our no commitment, 30-day trial, or read all about it on the product page! In the meantime, subscribe to our blog and changelog RSS feeds, or follow us on Twitter and LinkedIn to be in the know.

References

Redis in Action

https://redis.io/topics/data-types-intro

http://oldblog.antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html

http://highscalability.com/blog/2011/7/6/11-common-web-use-cases-solved-in-redis.html

Start your free 30 day trial today

Test the whole platform for 30 days with no ifs, ands, or buts.