Loading Data Into Memory With MongoDB

Check out the technique in the video MongoDB: Touch (Load Data Into Memory). This technique can be useful in a variety of environments - especially environments where we need to store a part of our data in memory since the data are more frequently accessed. As we see, this happens on the collection level and we can specify the data and index. We do not have to specify both. People familiar with their environment may study which collections would be best served being in memory (in rare cases, all).

Check out the highest-rated Automating ETL course on Udemy, if you're interested in data.

For situations where we may want to use a caching layer with an API, MongoDB provides us with a useful tool for this. In the past, several of my clients have used MongoDB for their cached JSON API calls with another back-end underneath it. This gives users quick access to data without querying the database on each call. This can be overused in memory-restricted environments, so we want to do the appropriate analysis on what data we put into memory.

One regular point I like to remind my audience, especially as of recent: we can often solve the same problem with a variety of techniques. While we look at one or two ways to solve the problem, these aren't the only ways we can approach this problem. The most appropriate solution to a problem is one in which you can troubleshoot quickly in the future and one in which you understand. Be careful about applying solutions that solve a problem, but introduce new problems in the future.

A great re-occuring example of this is my discussion on alerting best practices. There are a variety of alerts everywhere for everything in today's development environments, but many of these alerts cause more issues than they solve (one of which is creating a distraction). Be careful about "easy" solutions which introduce bigger problems. Know your environment and where to apply the best solution. Of course, all these being written, it is helpful to learn new techniques to solve problems because they may be useful in the future.