Caching in NodeJS with Redis

From the title of the post first thing we need to know is what is Caching.

Below are some of the definations how we can defing Caching

• Caching is basically a layer of abstraction that involves an intermediary storage mechanism in conjunction with a backend system (in our case Node.js) and, usually, a traditional database. The point is to enable an efficient data retrieval process, and caching systems are optimized for that particular purpose.
• Caching refers to the process of storing data in a temporary location so that the data can be accessed with minimal resources. Caching aims to reduce the cost of bandwidth of data sent over the network and the application’s response time. Applications that implement caching are faster and user friendly.
• Caching is a technique used for improving the performance of servers and a lot of devices used in day-to-day life.
  In caching, the result of certain computation or task is stored somewhere, and if the same computation is required again, it is served from that location itself rather than doing the computation again. This reduces computation and improves performance, thereby providing a better user experience.

Above are some of the definitions we get when we search for meaning of caching. But in simple words caching means fetching data and storing in some location so whenever next time the same data is asked we do not have to do database operation again for and send the data that we have stored in the temporary location.

For Node.js apps in particular, caching strategies should address the following concerns:

• Update or invalidate stale data from a cache: A very common problem in web development is handling cache expiration logic for maintaining an up-to-date cache
• Cache data that is frequently accessed: Caching makes a lot of sense here as we can process the data once, store it in a cache, and then retrieve it directly later, without doing any expensive or time-consuming operations. We would then need to periodically update the cache so users can see the latest information

The goal of caching is to save your server the stress of having to do the same thing over and over again that results in the same output. Let’s take for example, you have a page that generates a report of a companies inventory for the previous day for accountability. Now, let’s imagine that the report page will be viewed at least a thousand times daily by different auditors. Generating a single report wouldn’t be a problem, but if the same report is going to be generated every single time a thousand times, then you need to look for a better way to do it and that’s where caching comes in.

Following 2 images shows the difference between server using caching and server not using caching.

There are various ways how we can cache data in nodejs but in my post I'll be discussing about caching using Redis

Redis stands for REmote DIctionary Server that has the ability to store and manipulate high-level data types.

Redis is an in-memory database, its data access operations are faster than any other disk-based database, which makes Redis the perfect choice for caching.

Its key-value data storage system is another plus because it makes storage and retrieval much simpler. Using Redis, we can store and retrieve data in the cache using the SET and GET methods, respectively.

To get started using redis for caching application, install the redis node client by running
npm install -save redis

We also need to have a redis server running on our local machine
You can head over here to find out how.

When you are sure your redis server is installed and working properly, the next thing to do is to import the necessary modules and create your redis client:

    // index.js
    [...]

    const redis = require('redis')
    const client = redis.createClient();

    [...]

The process of caching with Redis is quite simple. When we receive a user request to a route that has caching enabled, we first check whether the requested data is already stored in the cache. If it is, we can quickly retrieve the data from the Redis cache and send the response back.

If the data is not stored in the cache, however — which we call a “cache miss” — we have to first retrieve the data from the database or from an external API call and send it to the client. We also make sure to store the retrieved data in the cache so that the next time the same request is made, we can simply send the cached data back to the user.

We can store data in Redis as key-value pair where key can be any custom string and value is the data we want to store that is retrived from database and processed.

Following is sample code for how we check for data in redis and send data if available in redis cache else first we retrieve data store it in cache and then return it to client.

const logger = require('../../../logger')
const CONFIG = require('../../../../config/config')
const { redis: redisClient } = require('../../../redis-client')
const axios = require('axios')

const mediaPageCollection = async (_, args, ctx) => {
  try {
    let { data: { slug, query } } = args

    //creating unique key based on slug
    let redisName = `MEDIA_PAGE_COLLECTION-${slug}`

    let redisData = await redisClient.get(redisName)

    if (redisData) {
      let data = JSON.parse(redisData)
      return {
        data
      }
    } else {
      let url = `${CONFIG.contentful.baseUrl}/spaces/${CONFIG.contentful.spaceId}/environments/${CONFIG.contentful.environment}`

      let data = await axios({
        url,
        method: 'POST',
        headers: { 'Authorization': `Bearer ${CONFIG.contentful.accessToken}` },
        data: {
          query
        }
      })
      data = data.data.data.mediaPageCollection

      await redisClient.setex(redisName, 43200, JSON.stringify(data))

      return {
        data
      }
    }
  } catch (error) {
    logger.error('ERROR WHILE FETCHING data >>>', error)
    return error
  }
}

Here in the above code we have created a custom key where MEDIA_PAGE_COLLECTION is static and slug value will be dynamic that we are getting from request object. First we are checking if any data exist in redis with the specified key return data from there else fetch data, store it in redis against the custom key and return the data.

While storing the data we have provided 3 parameters
- First parameter is key name.
- Second parameter is time for which we want to keep data in cache. This time is usually in seconds and data is cached for that particulat time after that the key is expired and removed from redis cache.
- And our last parameter is the data that we are willing to cache.

In case there are data changes when some data is updated or deleted and we want that next time whan the api id called it should return updated data and not what is already cached then we can delete key manually as follows.

redis.keys(key).then((keys) => {
    let pipeline = redis.pipeline()
    keys.forEach((k) => {
      pipeline.del(k)
    })
    return pipeline.exec();
  })

Here key is the custom string that we generated and now we want to delete it.

Conclusion:
Caching comes in handy a lot of times, but you need to know when to use it. it may be an overkill especially when the requests made to the application aren’t frequent. Also, POST, PUT and DELETE methods should never be cached because you want unique resources to be affected each time a request is made.
Caching is a near-mandatory operation for any data-intensive application. It improves app response time and even reduces costs associated with bandwidth and data volumes. It helps to minimize expensive database operations, third-party API calls, and server-to-server requests by saving a copy of previous request results locally on the server.

Reference Links:
- https://www.section.io/engineering-education/implementing-caching-in-nodejs-using-redis/
- https://medium.com/swlh/caching-in-node-js-using-redis-3b5400f41699
- https://livecodestream.dev/post/beginners-guide-to-redis-and-caching-with-nodejs/
- https://www.npmjs.com/package/redis

Tags