I created a little script to generate a daily tarball as a backup of a customer website.

Everyday, though, I would get the following error:

tar: .../public_html/wp-content: file changed as we read it

As you can see, the website is using Wordpress.

The error, though, I could not replicate when I would run the tar command by hand. What gives?

Looking closer into it, I noticed that my crontab setup would actually run the cron.php at the same time as the backup would run:

# m  h dom mon dow   command
  0  3 *   *   *     /.../backups/backup.sh

*/10 * *   *   *     cd /.../public_html && php -q wp-cron.php

As we can see in the crontab, the Wordpress wp-cron.php script runs every 10 minutes, including at 03:00 in the morning as the backup.sh script is started too.

The trick was to change the backup.sh startup minutes to something else, assuming the wp-cron.php script was fast enough (i.e. did not take 5 min. to run) then backup.sh will be running on its own (unless a customer works at 03:05 in the morning?)

# m  h dom mon dow   command
  5  3 *   *   *     /.../backups/backup.sh

*/10 * *   *   *     cd /.../public_html && php -q wp-cron.php

So I changed the 0 into a 5 and the tar error was gone.

In case one of your scripts takes more than the amount of time allowed, you will want to use a lock instead.

In your script, you can use flock as follow:

LOCK=/run/lock/cron.lock
(
    flock --exclusive 3

    ...do your work...

) 3>$LOCK

The flock here works in a way similar to flock() in a C/C++ program. Whenever the program closes the file descriptor (3 in my example) the lock is removed.

Assuming all your scripts use the same flock, they will never be executing in parallel.

If you do not want to wait forever for the lock to be released, use the --wait option. For example, the following would wait one minute before giving up.

LOCK=/run/lock/cron.lock
(
    flock --wait 60 --exclusive 3

    ...do your work...

) 3>$LOCK

I use such in my script that runs once every 10 minutes anyway. There is no need for that one to wait forever. It will run again soon anyway.