Nov 042012
Article Apache

We have covered in previous posts some alternatives to process the access logs of a web server.

It is equally important to analyze the error log, to detect errors that could be affecting the service. This post presents a simple script to automate this analysis.

The script will detect any error messages that appear in the file being processed a number of times above a configured minimum threshold.

An entry in the error log looks like the sample entry below (it is a single line in the file, although we have split it in two to make it easier to read):

We can see some information enclosed in square brakets “[ ]”, preceding the error message itself:

  • A timestamp
  • The type of entry (“[error]”, “[info”], etc…)
  • The IP address of the user whose request produced the message

The script to process the error log is as shown below::

The process performed by the script is:

In lines 3, 4 the file to be processed and the minimum number of appearances of a message in order to be reported are established.

In lines 6 to 9 the log file is run through a series of filters:

  • Line 7: Remove the leading data enclosed in square brackets ¬†“[ ]” (timestamp, message type and client IP)
  • Line 8: sort and group messages. Compute the number of appearances of each one. Sort again by number of appearances.
  • Line 9: print messages appearing more than MIN_REPEAT times.

From a sample log file, the script generates the following output:

From this results we can say that in the scripts “analysis.php”, “index.php” and “”¬†there are some issues that need to be fixed.

Once we are satisfied with the functionality of the script, we will probably want to add a cron entry to run the script daily, sending an email to the administrator with the resulting output.

 Posted by at 4:47 pm

 Leave a Reply