six demon bag

Wind, fire, all that kind of thing!

2020-02-24

Shell Patterns (1) - Logging

This is a short series describing some Bash constructs that I frequently use in my scripts.

What do you do when you run fully automated scripts in the background, but still want to keep track of what they're doing and, more importantly, when something goes wrong? The answer is, of course, you log what the script is doing (or is about to do).

There are two commonly used ways of implementing logging in Bash scripts:

Personally, I prefer the latter, since it allows not only for managing log files independently of the process creating the log output, but also for filtering log data and/or forwarding it to a central loghost.

Both approaches can be used either inside the script for logging specific messages:

some_command >>/var/log/myoutput.log 2>>/var/log/myerror.log
some_command 2>&1 >>/var/log/mycombined.log
echo 'my message' >>/var/log/myoutput.log

some_command 2>&1 | logger -t 'mytag' -p 'local0.info'
logger -t 'mytag' -p 'local0.info' 'my message'

or outside the script for capturing the entire script output:

./myscript.sh >/var/log/myoutput.log 2>/var/log/myerror.log

./myscript.sh 2>&1 | logger -t 'mytag' -p 'local0.info'

There are downsides to either approach, though. Putting the logging statements inside the script tends to clutter the code, and may also miss some output you actually wanted logged. Putting the logging statements outside the script means that the script is no longer self-contained.

So what can you do to capture all script output without relying on something outside the script? For the output redirection approach you can use the exec command to have all output written to files:

exec >/var/log/myoutput.log 2>/var/log/myerror.log
# rest of your code comes after this

exec 2>&1 >/var/log/mycombined.log
# rest of your code comes after this

For the logger approach the I/O redirection can be combined with another useful Bash feature called process substitution, which allows redirecting the output of one or more processes into another process:

exec > >(logger -t 'mytag' -p 'local0.info') 2> >(logger -t 'mytag' -p 'local0.err')
# rest of your code comes after this

Usually I define the script name as the log tag, and also throw in the Bash process ID for good measure:

tag="$(basename "$0")"
exec > >(logger --id="$$" -t "$tag" -p 'local0.info') 2> >(logger --id="$$" -t "$tag" -p 'local0.err')

Posted 20:47 [permalink]