We have covered in previous posts some alternatives to process the access logs of a web server.
It is equally important to analyze the error log, to detect errors that could be affecting the service. This post presents a simple script to automate this analysis.
The script will detect any error messages that appear in the file being processed a number of times above a configured minimum threshold.
An entry in the error log looks like the sample entry below (it is a single line in the file, although we have split it in two to make it easier to read):
|
1 2 3 4 |
[Sun Nov 04 12:01:09 2012] [error] [client 110.85.106.113] script '/web/openalfa/pub/signup.php' not found or unable to stat |
We can see some information enclosed in square brakets “[ ]”, preceding the error message itself:
- A timestamp
- The type of entry (“[error]”, “[info”], etc…)
- The IP address of the user whose request produced the message
The script to process the error log is as shown below::
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
#!/bin/bash LOGFILE=/var/log/apache2/error_log.2012-11-04-00_00_00 MIN_REPEAT=10 LINES=`cat $LOGFILE | sed -e 's/^.*] *//' | sort | uniq -c | sort -nr | perl -ne "/^s*(d+)s/; print $_ if( \$1 > $MIN_REPEAT );"` if test -n $LINES 2>/dev/null; then exit 0 fi NUM_LINES=`cat "$LOGFILE" | wc -l` echo echo "=======================================================================" echo "Log file: $LOGFILE" echo "total number of lines: $NUM_LINES" echo "lines that occur $MIN_REPEAT or more times:" echo "=======================================================================" echo IFS=' '; for i in $LINES; do echo $i; done; |
The process performed by the script is:
In lines 3, 4 the file to be processed and the minimum number of appearances of a message in order to be reported are established.
In lines 6 to 9 the log file is run through a series of filters:
- Line 7: Remove the leading data enclosed in square brackets “[ ]” (timestamp, message type and client IP)
- Line 8: sort and group messages. Compute the number of appearances of each one. Sort again by number of appearances.
- Line 9: print messages appearing more than MIN_REPEAT times.
From a sample log file, the script generates the following output:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
======================================================================= Log file: /var/log/apache2/openalfa/error_log.2012-11-04-00_00_00 total number of lines: 3962 lines that occur 10 or more times: ======================================================================= 1120 PHP Notice: Undefined index: Baiduspider in /web/analysis.php on line 29 1120 PHP Notice: Undefined index: Baiduspider in /web/analysis.php on line 185 178 PHP Notice: Undefined index: ts_fecha in /web/index.php on line 17 46 zonas.pl: "my" variable $irow masks earlier declaration in same scope at /web/zones.pl line 322. |
From this results we can say that in the scripts “analysis.php”, “index.php” and “zones.pl” there are some issues that need to be fixed.
Once we are satisfied with the functionality of the script, we will probably want to add a cron entry to run the script daily, sending an email to the administrator with the resulting output.
—