What's CRON with you?

By Ben Carley, Mon 16 January 2017, in category Linux

Scheduling

  
@

For those of you who have screen sessions open with a variation of the following:

    #!/bin/bash
    while :
    do
        ...do something...
    sleep 360
    done

It doesn't need to be like this, you can leverage Cron to schedule these jobs for you.

The software utility Cron is a time-based job scheduler in Unix-like computer operating systems

While Cron is the de facto tool for scheduling jobs, it can be tricky to get working and it is infuriating to debug. This blog post documents information that would have saved me time (while using RHEL6, your mileage may vary.)

How to set up Cron

Cron Directories

The simplest way to use cron is by putting executable files in the /etc/cron.* directories:

/etc/cron.hourly
/etc/cron.daily
/etc/cron.weekly
/etc/cron.monthly

The files in these folders will be run at their each of their designated times by Anacron. Anacron is similar to Cron, but accepts that machines might not be turned on 100% of the time. If you look into the contents of the /etc/anacrontab file it should look similar to below. As you can see, the jobs in daily, weekly and monthly are started at 03:05, 03:25 and 03:45 respectively.

# /etc/anacrontab: configuration file for anacron

enter code here`# See anacron(8) and anacrontab(5) for details.

SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22

#period in days   delay in minutes   job-identifier   command
1       5       cron.daily              nice run-parts /etc/cron.daily
7       25      cron.weekly             nice run-parts /etc/cron.weekly
@monthly 45     cron.monthly            nice run-parts /etc/cron.monthly

The nice run-parts /etc/cron.* command is using the command nice (which specifies how much CPU time to use) to call run-parts which will execute all files in the listed directory.

The MAILTO=root line causes output from the executed files to be redirected to /var/spool/mail/root

NOTE: Executable files must have the permission -rwxr-xr-x i.e. chmod 755 executable.sh

Cron.d

To have more control over the execution of your files, e.g. you want to run something twice an hour throughout the day and then not at night, you can use the directory /etc/cron.d/. Files in /etc/cron.d/ have the following format:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
# For details see man 4 crontabs
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * username  command to be executed
0 7 * * * ben bash good_morning.sh
# NOTE: THERE IS A NEWLINE AT THE END OF THE CRON FILE. V IMPORTANT.

This will run the command bash good_morning.sh as the user ben at 0700.

It is a common mistake to put the incorrect file path, always use absolute paths to reduce this chance of error. i.e. /home/ben/good_morning.sh instead of the command above which would direct to /good_morning.sh

NOTE: Do not include dots, dashes or anything else in the name of your Cron files otherwise Cron will not run them. This is documented behaviour but it is an enormous gotcha.

NOTE: As mentioned above, make sure that your Cron files always end with a new line.

NOTE: Scripts run with /etc/cron.d/ do not have environment variables loaded.

Crontab

Use the command crontab -e to edit the current users crontab, the syntax is similar to the files found in /etc/cron.d/* but usernames should not be included, as these files are user specific.

As root you can edit other users crontabs by using the command crontab -eu <user_name>. These files are stored in /var/spool/cron/<user_name>. It is not recommended that you edit these files directly, if you intend on using crontabs, edit them using the crontab command.

Jobs executed with crontab will have environment variables avaliable to them.

Time Format

This is a classic question and there are many good resources out there, however I will include a few examples below:

 # Run Cleanup - Every hour at XX:00
 0 * * * * root bash /opt/clean_up.sh

 # Run Loader - Twice per hour at XX:15 and XX:45
 15,45 * * * * root bash /opt/load.sh

 # Run Log Rotation - Once per day at 23:45
 45 23 * * * root python /opt/rotate_logs.py

NOTE: Don't fall into the trap of using * 1 * * * <user> <file> to run a job once at one, this will run your job at 1:00, 1:01, ... 1:59. NOT GOOD

Debugging Cron

Debugging Cron can be difficult because you don't have as much control over it you would have as a script executed from the command-line. This the following steps apply to cron.d and crontab methods once you've come across a 'Why didn't that run?' moment:

For Everything

For cron.d

For cron.d/crontab

If you start seeing Ran! showing up, you can be sure that Cron has access to the file and it can run it. Now slowly add your original functionality back in to good_morning.sh and set the run dates correctly. It could be a case that your script relies on environmental variables which aren't available to cron.d, if that is the case, should you be using crontab?

Cron with (Careful) Version Control

One nice things about using /etc/cron.d/ is that you are able to symlink files into respective folders and then version control these files. If I go into my environment and wonder why something has stopped running, this can cut a few steps out of debugging if something goes wrong and services stop running. For example, in my git repo at /opt/project/ I have my project files and scripts and load_cron. I can create a link to this file using ln /opt/project/load_cron /etc/cron.d/load_cron.

NOTE: Ensure that only privileged users have access to /opt/project/load_cron these cron jobs are run as root, if a non-privileged user was able to access this they could run anything they want. NOT GOOD.

NOTE: Check your symlinked file actually works before going down a rabbit hole of debugging. Verify the contents with cat /etc/cron.d/load_cron

NOTE: If you prefer creating soft links i.e. ln -s <file> <dest> ensure you run touch /etc/cron.d/load_cron after editing your file in /opt/project. The cron service checks for last modified dates and if you do not touch <file> it will not see there has been a change. This leads to some really weird behaviour and questions like How is this still running?.

TL;DR

Things that can go wrong:

Goodluck!