Create a Scheduling Framework to Automate All The Things

From querying a market feed twice a minute to remembering all your friends' birthdays with a timely remark about their most recent social media post so much of life can benefit from a framework for automation.

The power, flexibility and accessibility of JavaScript are amazing for this. In this tutorial we'll see that any JavaScript function — or even an entire application — can be set to run on your own custom schedule in just a few simple steps.

Repository

https://github.com/tdreid/lifebot

What Will I Learn?

This tutorial explains how to build a slim, configurable JavaScript framework you can use to automate any number of scriptable tasks each with its own recurring schedule.

This includes how to:

• define recurring schedules elegantly with the node-schedule package using either date objects or the conventions of cron;
• use the dotenv module to load environment variables from an .env file;
• configure an environment variable to associate as many independent JavaScript based jobs with schedules as you may ever want;
• setup and use the very flexible log4js package to send nicely formatted feedback to stdout.

Requirements

All you will need for this are:

• a favorite text editor;
• a general familiarity with JavaScript, node and npm(these should already be installed in your environment.)

It's a great idea to use an Integrated Development Environment (IDE) and git, though these will not be covered in detail. If you don't have a favorite editor/IDE already I have been very satisfied with Cloud9.

Difficulty

• Intermediate

Tutorial Contents

Here is an overview of the steps:

1. Create a new package in a new folder.
2. Install the dependencies dotenv, log4js and node-schedule.
3. Create the main script: lifebot.js.
4. Create the jobs and specify the schedules.
5. Parse and run the job schedule in lifebot.js.

Create a new package and repository

First make a new empty folder for your project. Change to that folder as your present working directory.

mkdir lifebot
cd lifebot

To create an npm package add a package.json file to the directory. The init command built into npm provides a wizard of sorts to guide you through this. I always use it to get the most common fields set out in correct JSON—even if I intend to modify this file by hand later.

npm init

It's generally okay to accept the defaults. I usually change the "entry point" field to a filename specific to my project. Here is how I answered the prompts this time.

package name: (lifebot) 
version: (0.1.0) 
description: Automate life, bot.
entry point: (index.js) lifebot.js
test command: 
git repository: 
keywords: automation, life, bots
license: (MIT) 

This yields a package.json file in the root of your project folder like this. It's a good starting point. So when npm init prompts Is this okay? go ahead and let it create the file.

{
  "name": "lifebot",
  "version": "0.1.0",
  "description": "Automate life, bot.",
  "main": "lifebot.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "automation",
    "life",
    "bots"
  ],
  "author": "Trevor Reid",
  "license": "MIT"
}

_{Note—your results may vary depending on the defaults you have set. Clearly, your author field should be different.}

Edit the new package.json to add a custom start script for convenience.

. . .

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node lifebot.js"
  },

. . .

Now when you type npm start from your project folder it will launch your work in progress.
_{(Note—a technique for hot loading your work using nodemon will be discussed in Part II of this series)}

Install the dependencies

Run this command:

npm install dotenv log4js node-schedule --save

This will insert a "dependencies" field in your package.json file identifying the modules that your script will use. It will also add the file package.lock.

This latter file is a precise representation of the dependency tree npm interprets from package.json. A precise tree in turn guarantees consistency when your work is shared in different environments.

For those using git it is important by this point to make sure your .gitignore file is in order. Since installing dependencies will create a deep and potentially large folder tree make sure it is ignored. On the other hand both package.json and package-lock.json are intended to be committed to your repository.

GitHub's standard template for Node.js provides a quick way to handle all that.

Create the main script: lifebot.js

In a new file called lifebot.js use require() to load the log4js module and call getLogger('lifebot') to create a default logger instance that will tag messages as originating from 'lifebot'.

Usually global variables are not preferred. However it's warranted in this case as logger will be universally useful across all scripts that we add to the framework. In a sense it is taking the place of the ubiquitous console.log() statement.

Note how several IDE's allow us to designate this global variable by convention using a comment.

We'll also require the node-schedule module at this point. I called it cron just because it's short and alludes to the job scheduler utility in Unix-like computer operating systems. In fact it's the cron utility's conventional notation that we'll be using to specify our schedules. So it all fits.

By default the logger is set to silent. That is it will not output any messages. This is by design so that the module does not pollute stdout in a production environment or when included in another library.

For our purposes though it's convenient to see our debug and informational messages by default in the development environment. And in any case we want to be able to override this setting at run time.

Importantly, this is one environment variable that won't be loaded from the .env file that we address in the subsequent step. This is because the logger is being configured first thing, before that .env file is loaded.

If the user wanted to tweak the logging level they would do so on the command line when executing our script like this:

LIFEBOT_LOGGER_LEVEL=trace node lifebot.js

Speaking of the .env file pulling in its values is what comes next inlifebot.js

As we'll see in a moment this will append any variables added to .env in the form NAME=VALUE to process.env.

Before turning our attention to setting up the actual jobs and environment variables let's create a placeholder block. We can return to it later.

Create the jobs and specify the schedules

So far we have almost all the pieces. Still missing are:

• Some things to do
• When to do some things

For something to do let's create two more simple script files. Each will export a single function.

This is an important concept. Jobs will be able to do anything that JavaScript/Node.js can do—so long as you can roll the entry point into a single function for the module to export.

For now let's keep it very simple.

helloWorld.js

makeCoffee.js

Let's hope it's obvious that these contrived examples simply write some text to stdout. The main thing to note is the use of try-catch blocks. While you could use your cron instance in lifebot.js to call any function you'll be much happier if you are stringent about handling errors with try-catch. Otherwise if your scheduled code fails you will have to restart to resume scheduled jobs.

Alright now let's move on to scheduling. Create one more file in the project folder.

.env

The node-schedule package understands a cron-style notation to define time and frequency. Its six part notation works like this:

*    *    *    *    *    *
┬    ┬    ┬    ┬    ┬    ┬
│    │    │    │    │    │
│    │    │    │    │    └ day of week (0 - 7) (0 or 7 means Sunday)
│    │    │    │    └───── month (1 - 12)
│    │    │    └────────── day of month (1 - 31)
│    │    └─────────────── hour (0 - 23)
│    └──────────────────── minute (0 - 59)
└───────────────────────── second (0 - 59, OPTIONAL)

The * essentially means any and you can also use the slash character (/) for divisible by. So to run a job once per minute we would write it out like so:

*/1 * * * *

And another job every two minutes:

*/2 * * * *

Notice how these examples use only five of the six parts because the seconds are an optional time unit.

We can write our parsing routine in lifebot.js to split this environment variable into jobs delimited by commas. And then it will split the file name from the schedule delimited by pipe. Think of the first piece, that portion of each job preceding the | character as directly equivalent to a string passed into require().

So in .env we can tell lifebot that it should run what's defined in helloWorld.js once per minute and also brew us a cup of coffee ever other minute:

LIFEBOT_JOBS=./helloWorld.js|*/1 * * * *,./makeCoffee.js|*/2 * * * *

This syntax for our environment variable allows us to schedule an arbitrary number of jobs, each with its own simple, yet powerful and flexible schedule.

Parse the job schedule in lifebot.js

Returning our attention to lifebot.js add the following code to our placeholder block.

We've parsed process.env.LIFEBOT_JOBS into an array of job objects. Let's summarize that result by logging the number of jobs to stdout.

All that remains is to write code that schedules the jobs.

Conclusion

At last it's time to run the code, sit back, relax and enjoy all that sweet automated brew. For reference you can compare your results to my version of this project at https://github.com/tdreid/lifebot.

Curriculum

This is intended as the first of a curriculum in three parts:

• Part I: Today we've set up and tried out a basic framework to automate just about anything we can do in JavaScript.
• Part II: Next we'll power up the framework by passing arguments to our automated functions as well as explore alternative ways of setting the environment variables and schedule.
• Part III: In the third step and beyond we'll explore some real world examples of functions and apps that can be put to good use whenever you build a lifebot of your own.

Proof of Authorship & Work Done

https://github.com/tdreid/lifebot