LinuxBeginner 13 min Lesson 13 of 24

Cron and Scheduled Jobs

Schedule recurring work reliably: crontab syntax, the environment trap, capturing output, locking and systemd timers.

Linux · Lesson 13 of 24
0/24 done(0%)

What is it? #

Cron runs commands on a schedule: nightly backups, hourly cleanups, weekly reports.

The syntax is five fields — minute, hour, day of month, month, day of week — followed by the command.

Most cron problems are not about the schedule. They are about the environment: cron runs with a minimal PATH, a different working directory, and no shell profile loaded, so a command that works in your terminal frequently fails under cron.

The other recurring problem is output. If you do not redirect it, you either lose it or generate mail nobody reads.

Think of it like this #

An alarm clock that runs an errand for you. It is reliable about the time, and completely literal about the instructions.

If the errand assumes you are already standing in the kitchen with your wallet, and the alarm starts you in the hallway with nothing, it fails — not because of the timing but because of the context.

Simple example #

A nightly database backup must run at 2am, log its output, not overlap with a previous run that is still going, and alert someone if it fails.

Code #

TEXT
Crontab syntax

*     *     *     *     *   command
│     │     │     │     └── day of week  (0-7, 0 and 7 are Sunday)
│     │     │     └──────── month        (1-12)
│     │     └────────────── day of month (1-31)
│     └──────────────────── hour         (0-23)
└────────────────────────── minute       (0-59)

*/15 * * * *      every 15 minutes
0 2 * * *         every day at 02:00
0 */6 * * *       every 6 hours
30 3 * * 1        Mondays at 03:30
0 0 1 * *         first day of each month
@reboot           once at boot
BASH
crontab -e            # edit the current user's crontab
crontab -l            # list it
sudo crontab -e       # root's crontab
# system-wide jobs also live in /etc/cron.d/ and /etc/crontab
BASH
# A production-quality cron entry
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin
[email protected]

0 2 * * * /usr/bin/flock -n /tmp/backup.lock /srv/app/scripts/backup.sh >> /var/log/backup.log 2>&1
# flock -n   : skip this run if the previous one is still going
# >> log     : append output to a file
# 2>&1       : include errors in the same file
BASH
#!/usr/bin/env bash
# /srv/app/scripts/backup.sh
set -euo pipefail                     # fail on error, undefined variable or pipe failure

cd /srv/app                           # never rely on the working directory
source /srv/app/.venv/bin/activate    # cron does not load your shell profile

echo "[$(date -Is)] backup starting"
if ! pg_dump "<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>D</mi><mi>A</mi><mi>T</mi><mi>A</mi><mi>B</mi><mi>A</mi><mi>S</mi><msub><mi>E</mi><mi>U</mi></msub><mi>R</mi><mi>L</mi><mi mathvariant="normal">&quot;</mi><mi mathvariant="normal">∣</mi><mi>g</mi><mi>z</mi><mi>i</mi><mi>p</mi><mo>&gt;</mo><mi mathvariant="normal">&quot;</mi><mi mathvariant="normal">/</mi><mi>b</mi><mi>a</mi><mi>c</mi><mi>k</mi><mi>u</mi><mi>p</mi><mi>s</mi><mi mathvariant="normal">/</mi><mi>d</mi><mi>b</mi><mo>−</mo></mrow><annotation encoding="application/x-tex">DATABASE_URL&quot; | gzip &gt; &quot;/backups/db-</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.0278em;">D</span><span class="mord mathnormal">A</span><span class="mord mathnormal" style="margin-right:0.1389em;">T</span><span class="mord mathnormal">A</span><span class="mord mathnormal" style="margin-right:0.0502em;">B</span><span class="mord mathnormal">A</span><span class="mord mathnormal" style="margin-right:0.0576em;">S</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0576em;">E</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.109em;">U</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord mathnormal" style="margin-right:0.0077em;">R</span><span class="mord mathnormal">L</span><span class="mord">&quot;∣</span><span class="mord mathnormal" style="margin-right:0.0359em;">g</span><span class="mord mathnormal" style="margin-right:0.044em;">z</span><span class="mord mathnormal">i</span><span class="mord mathnormal">p</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">&gt;</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="katex-base"><span class="katex-strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">&quot;/</span><span class="mord mathnormal">ba</span><span class="mord mathnormal">c</span><span class="mord mathnormal" style="margin-right:0.0315em;">k</span><span class="mord mathnormal">u</span><span class="mord mathnormal">p</span><span class="mord mathnormal">s</span><span class="mord">/</span><span class="mord mathnormal">d</span><span class="mord mathnormal">b</span><span class="mord">−</span></span></span></span>(date +%F).sql.gz"; then
    echo "[$(date -Is)] backup FAILED" >&2
    curl -fsS -m 10 "https://alerts.example.com/backup-failed" || true
    exit 1
fi
echo "[$(date -Is)] backup finished"
INI
# The systemd alternative, with logs and status built in
# /etc/systemd/system/backup.timer
[Unit]
Description=Nightly database backup

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true            # run on next boot if the machine was off
RandomizedDelaySec=300     # avoid a thundering herd across servers

[Install]
WantedBy=timers.target

How it works #

The five schedule fields are read left to right, smallest unit first. */15 means every fifteenth value, so */15 * * * * fires four times an hour.

The environment trap is the most common failure. Cron provides a minimal PATH, so python may not be found; it does not load .bashrc, so aliases and virtual environments are absent; and it starts in the user's home directory, so relative paths break. The script handles all three explicitly: absolute paths, an explicit cd, and activating the environment.

flock -n takes a lock file and skips the run if it is already held. Without it, a backup that takes longer than the interval overlaps with the next one, which can corrupt output or exhaust resources.

Redirecting with >> file 2>&1 keeps a record. Without redirection, cron mails the output to the user, which on most servers means it is discarded silently.

set -euo pipefail makes the script stop at the first failure rather than continuing after a broken step. Without it, a failed pg_dump still produces a gzip file — an empty one that looks like a successful backup.

The explicit failure path pings an alert endpoint. A silent cron job that has been failing for three weeks is worse than no cron job, because you believed it was working.

systemd timers are the modern alternative: each run is a unit with logs, status and Persistent=true to catch up after downtime.

Real-world use #

Backups, cleanup of expired data, report generation, certificate renewal, cache warming and health checks are typical scheduled work.

The most dangerous failure mode is a silent one. Every scheduled job should either log to a place you check or, better, report success to a monitoring service that alerts when the expected check-in does not arrive.

Running cron on every application server means the job runs once per server. A single designated scheduler, or a distributed lock, is required — the same point made in the background workers lesson.

Timezones matter. Cron uses the system timezone, so a job scheduled for 2am runs at 2am local time, with the daylight-saving edge cases that implies. Servers set to UTC avoid the ambiguity.

Overlapping runs cause real incidents: a cleanup job that normally takes two minutes takes four hours after data growth, and suddenly three copies are running at once.

Common mistakes #

  • Assuming cron has your shell environment, PATH and working directory.
  • No output redirection, so failures are invisible.
  • No locking, so a slow run overlaps with the next one.
  • No alerting, so a job that has failed for weeks goes unnoticed.
  • Running the same cron entry on every server in a cluster.

Practice #

Write a script that logs a timestamped line and fails deliberately on one branch. Schedule it every five minutes with flock, output redirection and a failure alert. Then check the log file after fifteen minutes and confirm both the successful and failing runs are recorded.

Quick quiz

  1. 1. What does `*/15 * * * *` mean?

  2. 2. Why do cron jobs often fail even though the command works in your shell?

  3. 3. What does `flock -n` achieve?

  4. 4. What happens to cron output that is not redirected?

  5. 5. What does `Persistent=true` on a systemd timer do?

Summary

  • Cron schedules commands with five time fields.
  • Most failures are environment problems — use absolute paths and set up explicitly.
  • Always redirect output and use flock to prevent overlapping runs.
  • Alert on failure; silent broken jobs are worse than none.
  • systemd timers add logging, status and catch-up after downtime.