Nov 042012
 
Article Apache

We have covered in previous posts some alternatives to process the access logs of a web server.

It is equally important to analyze the error log, to detect errors that could be affecting the service. This post presents a simple script to automate this analysis.

The script will detect any error messages that appear in the file being processed a number of times above a configured minimum threshold.

An entry in the error log looks like the sample entry below (it is a single line in the file, although we have split it in two to make it easier to read):

We can see some information enclosed in square brakets “[ ]”, preceding the error message itself:

  • A timestamp
  • The type of entry (“[error]”, “[info”], etc…)
  • The IP address of the user whose request produced the message

The script to process the error log is as shown below::

The process performed by the script is:

In lines 3, 4 the file to be processed and the minimum number of appearances of a message in order to be reported are established.

In lines 6 to 9 the log file is run through a series of filters:

  • Line 7: Remove the leading data enclosed in square brackets  “[ ]” (timestamp, message type and client IP)
  • Line 8: sort and group messages. Compute the number of appearances of each one. Sort again by number of appearances.
  • Line 9: print messages appearing more than MIN_REPEAT times.

From a sample log file, the script generates the following output:

From this results we can say that in the scripts “analysis.php”, “index.php” and “zones.pl” there are some issues that need to be fixed.

Once we are satisfied with the functionality of the script, we will probably want to add a cron entry to run the script daily, sending an email to the administrator with the resulting output.

 Posted by at 4:47 pm

 Leave a Reply

(required)

(required)