Systems Monitoring Via E-Mail

From attempted hacks to security breaches, systems monitoring is an important undertaking. And with all the logs that are available on a Linux system, a systems administrator certainly has the tools available to look into any suspected misuse or abuse of a system. The problem? All those logs on all those machines can lead to many hours of laborious work, sifting through information while looking for that proverbial needle in a haystack.

Logwatch, a utility that comes with most flavors of Linux these days, is an open-source tool that can help. Developed by Kirk Bauer (kaybee.org:81/~kirk), Logwatch enables systems administrators to keep an eye on an IT environment via e-mail, summarizing logs daily and sending the highlights to the systems administrator.

Logwatch is capable of summarizing and communicating many different types of logs, including ftpd, sshd, httpd and mail. Logwatch can also send information related to firewall logs from iptables and report on disk-space usage. One of the nicest features of Logwatch is that it is a pluggable environment, meaning that anything that isn't summarized out-of-the-box can have a plug-in developed to include the types of information one sees fit. Several samples of plug-ins are available in the Logwatch tarball (the standard way to pass around source-code distributions) and are usually installed in /etc/log.d/scripts. Use these for a good starting point in developing any custom plug-ins that may be needed.

I use Logwatch to monitor my home servers. My configuration includes summaries of activity on the mail server, all ssh activity, http probes, ftp activity, samba and security logs, and available disk space. Logwatch can include detailed information--making for long e-mail messages--or can be summarized in brief. I typically begin a Logwatch configuration by using very verbose output and turn down the detail level when things look the way I want them to. Using the low detail-level setting can still give a systems administrator plenty of information with which to spot a problem; if more detail is required, Logwatch can be rerun from the command line.

Most systems administrators prefer to set up Logwatch to run daily via a cron job, which executes the program at a preset time. Be careful when setting up the "range" parameter because this can cause things to look a little different than planned. I usually set up the cron job to run just after midnight and use a range of "yesterday," which gives me an accurate picture of all activity on a server for the prior day.

In addition to summarizing all logs via e-mail, Logwatch is also capable of selecting individual services and particular log files, including or excluding archives and including debug output for troubleshooting configuration. Be careful when turning on debugging information--the output becomes tremendous.

Kevin Carlson ([email protected]) is with Watchfire, a business-management software and services provider based in Waltham, Mass.

Tools of Distinction

The latest version of Logwatch can be downloaded from www.logwatch.org. But in addition to Logwatch, there are several other tools available that can help systems administrators keep on top of their servers' activities:

Logconf: Configures logwatch across multiple servers. This utility, which was also authored by Kirk Bauer, is available on sourceforge.net/projects/longconf.

Logrotate: Regularly archives log files. Handles several common log-file types with no customization needed. Configuration files can be easily created to include new log files into the archiving process. This utility is packaged with most Linux distributions.

ModLogAn: Available from sourceforge .net/projects/modlogan, this modular log-file analyzer is capable of creating reports from multiple log files, including ftp, httpd, mail and firewall.

RegWatch: A PERL-based log-file watcher that keys off of regular expressions defined by the user. Actions are taken based on the regular expression that is matched.