Multi-Platform Performance Monitoring on the Cheap

Dale Southard

The monitoring system presented in this article grew out of a simple need for a portable system with which I could remotely monitor performance metrics on UNIX hosts. When I began looking for a solution, I considered many of the available commercial products. The standard xload is easy to understand, but only provides a single metric (load average) and doesn’t retain information across invocations. The top command is better, but again provides only a snapshot, not a history trail. Sun’s perfmeter and rstatd provide more metrics and the ability to save trails, but are only available under a few architectures. SGI’s Performance Co-Pilot can monitor and save an incredible number of metrics, but at the time was only available under IRIX (it has since been ported to Linux as well). Finally, SNMP looked like a future contender, but still suffered from a lack of affordable monitoring packages and security issues on some platforms.

What I really wanted was the ability to collect and save a group of performance metrics and then reduce them to a form that is easy to understand. Ideally, the tools should be portable to a wide range of UNIX flavors. Upon further consideration, I found my needs were simple enough to be met with syslog and some common UNIX utilities.

The original inspiration for the design came from one of syslogd’s built-in features, the mark timestamp. Most modern syslog daemons provide a mark function that places a timestamp in the logfile at regular intervals. This is often used to help fix the time of catastrophic system events (such as sudden power loss) that would otherwise provide no log evidence. What limits the usefulness of the standard syslogd mark function is that it provides only a mark indicating that the machine is powered on and running syslogd.