A quick mention of Redis and its use cases, with a tutorial to using Redis to cache data when using Node.js
Chris Gwilliams
|RSS FeedSolution Architect at Aiven
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
Redis is a database designed for fast access to data. Blazingly fast. Although it is a database, it is not exactly like the Postgres and MySQL relational databases that you may be more familiar with. Redis is an in-memory database that stores its contents in the RAM of your Server/PC/Container/Raspberry Pi; with many persistence options such as occasionally dumping all of its data to disk or appending to a log.
Getting used to Redis to start with may be a bit of a shock when coming from a more traditional DB background but its features and potential use cases are massive and, in many ways, from zero to getting started is minutes instead of hours or days. Take a look at our Introduction to Redis for a more in-depth look at Redis and how it works under the hood.
In this post, you will have a quickstart guide for using Redis as a cache for your Node.js Express app. We will build two (beyond trivial) apps to serve a single API endpoint. Then, we will run them both, one using Redis as a cache, and the other to send data without a caching layer; to show a very simple (but convincing) benchmark of the benefits Redis can provide with an almost zero implementation cost.
Because this is 2020, we have to do something so we can use the word FinTech, so we are going to provide an API that lists info about the health of Bitcoin... we will use the API of Alpha Vantage because their API is super simple, clean and free to use (with an email required for an API key). It may seem odd that I am providing free advertising to a company I know little about but it will all become clear when you hear what we will call our app....Cache in Hand — yeah, I knew you'd like it. 🐶
Express is a great framework for setting up a Node.js backend quickly but well structured and with room to scale. You will need npm installed but then you can open a command line and paste:
Loading code...
Loading code...
Here, we use express, the Redis client, and the response-time middleware to send that information in the response headers.
Let's write the app, we can do all of this in a single file and we only need 2 endpoints: one with a cache and one without.
Create a file and call it app.js, and it we will add to it now:
Loading code...
Loading code...
Create a file and call it config.json, inside we will create a JSON object that contains the port we will use, the API key we have and our Redis connection info.
Loading code...
uncached routeLoading code...
Now, you can run the server with node app.js and a simple curl command will show you the response text but, since we added a response time to the header, we want more info than that.
Loading code...
Using the -v flag to get a verbose output shows us the headers coming back and a response of ~435ms
You can, of course, install Redis for your OS from Redis and it is a perfectly good solution for testing or local development. However, when you want to deploy your product, you will need a more reliable and available solution. At Aiven, we provide a managed Redis service (along with $300 of credits for testing) that you can deploy on popular cloud providers, such as Google Cloud Platform, Amazon Web Services or Microsoft Azure.
All you need to do is:
cache-in-hand works for this example)startup plan will be more than enough here but you can upgrade later with zero downtimeHost, Port and a Password. This is all we need for this example but different clients may need different information (You can use the Service URI to connect from the CLI).Add the information to your config.json file and let's create a new route that adds the caching functionality to your server.
When adding to Redis, you will use a Key and a Value, for those that are used to Frontend web frameworks, this will be very similar to using a store in things like React or Vue.
The key can be extremely useful as it can be used to identify a particular client, or connection, or it can be used to identify remote content that is requested often.
In our example, we will create a Universally Unique Idetification Number or UUID when the server starts. This means that each client will see the cached information from Redis once it has been cached. We could set the key to a user's ID, for example, so only they see the cached information and other users would get newer responses.
Loading code...
Let's break it down.
/cache-in-hand and we check our Redis database for dataThat is it! Simple.
We could use curl to test this and read the response time header from the results but this sounds like a very manual process that could be automated, so we will!
My tool of choice is Bombardier for this kind of thing. It is built with Go but you can grab the binary if you don't want yet another language on your machine.
bombardier -c 200 -d 10s -l localhost:6666/cache-in-hand
This runs our benchmark over 10 seconds with 200 connections; the -l flag gives us more detail about the latency with a latency distribution. Let's see what happens when I run this on my machine:
/go-for-broke
Locally, my node server is able to handle around 300 requests per second comfortably, but still responds with 2284. For our needs, though, the latency distribution is most important. We see an average of 622ms and
/cache-in-hand
Whoa, a pretty big difference, right? Adding Redis as a cache not only reduces the latency by a factor of ~13.8 but it also allows the fourteen times the number of requests per second.
gzip (although this will also make a minimal difference in this case due to the size of the response)Wrapping this Redis check in each of your requests may be fine for a small project but, when you scale up to a few dozen routes, you will quickly want a different solution. The middleware support in Express is amazing and there are a number of caching middlewares that support Redis in npm.
The sample code can be found in our Examples Repository along with a number of other code samples for the services we offer. If you run into any issues, you can always reach out to our Support team and check out our help articles here.
mkdir cache-in-hand && cd cache-in-hand && npm init -ynpm install --save-dev express redis response-timeconst config = require('./config.json');
const express = require('express');
const responseTime = require('response-time')
const redis = require('redis');
const uuid = require('uuid/v4');
const axios = require('axios');var app = express();
app.use(responseTime());
app.listen(config.PORT, function() {
console.log('Cache-in-hand is running!');
});{
"API_KEY": "",
"API_URL": "https://www.alphavantage.co/query?function=CRYPTO_RATING&symbol=BTC&apikey=",
"REDIS_HOST": "",
"REDIS_PORT": ,
"REDIS_PWD": "",
"PORT": 6666
}app.get('/go-for-broke', async(req,res) => {
axios.get(config.API_URL + config.API_KEY)
.then(result => {
res.send(JSON.stringify(result.data));
}).catch(err => {
console.error(err);
});
});curl -v localhost:6666/go-for-broke
* Trying 127.0.0.1:6666...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 6666 (#0)
> GET /go-for-broke HTTP/1.1
> Host: localhost:6666
> User-Agent: curl/7.66.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 263
< ETag: W/"107-0WhO2PvF4fl39YQOl4rWIQttsjI"
< X-Response-Time: 434.951ms
< Date: Tue, 17 Mar 2020 08:27:39 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
{"Crypto Rating (FCAS)":{"1. symbol":"BTC","2. name":"Bitcoin","3. fcas rating":"Attractive","4. fcas score":"891","5. developer score":"847","6. market maturity score":"826","7. utility score":"958","8. last refreshed":"2020-03-16 00:00:00","9. timezone":"UTC"}}⏎
const REDIS_KEY = uuid.v4();
app.get('/cache-in-hand', function(req,res) {
try {
client.get(REDIS_KEY, function(err, result) {
if (result ) {
res.send(result);
} else {
axios.get(config.API_URL + config.API_KEY)
.then(apiResult => {
const str = JSON.stringify(apiResult.data)
client.set(REDIS_KEY, str);
res.send(str)
}).catch(err => {
console.error(err);
});
}
});
} catch (error) {
console.error(error)
res.send('Something went wrong!!!');
}
});