For those of you who have screen sessions open with a variation of the following:
#!/bin/bash while : do ...do something... sleep 360 done
It doesn't need to be like this, you can leverage Cron to schedule these jobs for you.
The software utility Cron is a time-based job scheduler in Unix-like computer operating systems
While Cron is the de facto tool for scheduling jobs, it can be tricky to get working and it is infuriating to debug. This blog post documents information that would have saved me time (while using RHEL6, your mileage may vary.)
The simplest way to use cron is by putting executable files in the
/etc/cron.hourly /etc/cron.daily /etc/cron.weekly /etc/cron.monthly
The files in these folders will be run at their each of their designated times by Anacron. Anacron is similar to Cron, but accepts that machines might not be turned on 100% of the time. If you look into the contents of the
/etc/anacrontab file it should look similar to below. As you can see, the jobs in daily, weekly and monthly are started at 03:05, 03:25 and 03:45 respectively.
# /etc/anacrontab: configuration file for anacron enter code here`# See anacron(8) and anacrontab(5) for details. SHELL=/bin/sh PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root # the maximal random delay added to the base delay of the jobs RANDOM_DELAY=45 # the jobs will be started during the following hours only START_HOURS_RANGE=3-22 #period in days delay in minutes job-identifier command 1 5 cron.daily nice run-parts /etc/cron.daily 7 25 cron.weekly nice run-parts /etc/cron.weekly @monthly 45 cron.monthly nice run-parts /etc/cron.monthly
nice run-parts /etc/cron.* command is using the command
nice (which specifies how much CPU time to use) to call
run-parts which will execute all files in the listed directory.
MAILTO=root line causes output from the executed files to be redirected to
NOTE: Executable files must have the permission
chmod 755 executable.sh
To have more control over the execution of your files, e.g. you want to run something twice an hour throughout the day and then not at night, you can use the directory
/etc/cron.d/. Files in
/etc/cron.d/ have the following format:
SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # For details see man 4 crontabs # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * username command to be executed 0 7 * * * ben bash good_morning.sh # NOTE: THERE IS A NEWLINE AT THE END OF THE CRON FILE. V IMPORTANT.
This will run the command
bash good_morning.sh as the user
It is a common mistake to put the incorrect file path, always use absolute paths to reduce this chance of error. i.e.
/home/ben/good_morning.sh instead of the command above which would direct to
NOTE: Do not include dots, dashes or anything else in the name of your Cron files otherwise Cron will not run them. This is documented behaviour but it is an enormous gotcha.
NOTE: As mentioned above, make sure that your Cron files always end with a new line.
NOTE: Scripts run with
/etc/cron.d/ do not have environment variables loaded.
Use the command
crontab -e to edit the current users crontab, the syntax is similar to the files found in
/etc/cron.d/* but usernames should not be included, as these files are user specific.
As root you can edit other users crontabs by using the command
crontab -eu <user_name>. These files are stored in
/var/spool/cron/<user_name>. It is not recommended that you edit these files directly, if you intend on using crontabs, edit them using the crontab command.
Jobs executed with crontab will have environment variables avaliable to them.
This is a classic question and there are many good resources out there, however I will include a few examples below:
# Run Cleanup - Every hour at XX:00 0 * * * * root bash /opt/clean_up.sh # Run Loader - Twice per hour at XX:15 and XX:45 15,45 * * * * root bash /opt/load.sh # Run Log Rotation - Once per day at 23:45 45 23 * * * root python /opt/rotate_logs.py
NOTE: Don't fall into the trap of using
* 1 * * * <user> <file> to run a job once at one, this will run your job at 1:00, 1:01, ... 1:59. NOT GOOD
Debugging Cron can be difficult because you don't have as much control over it you would have as a script executed from the command-line. This the following steps apply to
crontab methods once you've come across a 'Why didn't that run?' moment:
service crond status
benand see if it works as expected,
chown ben /home/ben/good_morning.sh?
* * * * * ben bash /home/ben/good_morning.shand execute
tail -f /var/mail/rooti.e. run ever single minute and watch the logs to check if
Ran!is showing up.
If you start seeing Ran! showing up, you can be sure that Cron has access to the file and it can run it. Now slowly add your original functionality back in to
good_morning.sh and set the run dates correctly. It could be a case that your script relies on environmental variables which aren't available to
cron.d, if that is the case, should you be using
One nice things about using
/etc/cron.d/ is that you are able to symlink files into respective folders and then version control these files. If I go into my environment and wonder why something has stopped running, this can cut a few steps out of debugging if something goes wrong and services stop running.
For example, in my git repo at
/opt/project/ I have my project files and scripts and
load_cron. I can create a link to this file using
ln /opt/project/load_cron /etc/cron.d/load_cron.
NOTE: Ensure that only privileged users have access to
/opt/project/load_cron these cron jobs are run as root, if a non-privileged user was able to access this they could run anything they want. NOT GOOD.
NOTE: Check your symlinked file actually works before going down a rabbit hole of debugging. Verify the contents with
NOTE: If you prefer creating soft links i.e.
ln -s <file> <dest> ensure you run
touch /etc/cron.d/load_cron after editing your file in
/opt/project. The cron service checks for last modified dates and if you do not
touch <file> it will not see there has been a change. This leads to some really weird behaviour and questions like How is this still running?.
Things that can go wrong:
.'s in file names of cron.d files - Won't run
touching symlinked files - Won't reflect changes
chmod 755 executable.sh- Won't run
chown foo executable.sh- Won't run
* N * * *to run a job once once at a certain hour - Will run LOADS