http access_log analys part 1

Part 1 in log analysis I will provide you with some useful http logs, and try and anlyse them, and if possible correlate them.

First of all, the logs that I provide are all from Linux systems, but the logs should be similar if you're running apache on a windows box. (Which you should try to avoid if possible).

The logs from this site, has a few hundred unique visitors a month, and not loaded with lots of traffic, so It's quit easy to go thru these logs manullay and with some small scripts. This is to get a better understanding of the logging format and how you can learn to identify malicious traffic that your httpd daemon logged.

It's very common to find logs like these in your httpd access log;
192.168.2.88 - - [21/Nov/2005:13:23:18 +0100] "GET / HTTP/1.1" 403 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/2
0050919 Firefox/1.0.7"
192.168.2.88 - - [21/Nov/2005:13:23:18 +0100] "GET /favicon.ico HTTP/1.1" 404 288 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.
7.12) Gecko/20050919 Firefox/1.0.7"
192.168.2.88 - - [21/Nov/2005:13:23:45 +0100] "GET / HTTP/1.1" 403 63 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/
20050919 Firefox/1.0.7"
192.168.2.88 - - [21/Nov/2005:13:23:46 +0100] "GET /favicon.ico HTTP/1.1" 404 288 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.
7.12) Gecko/20050919 Firefox/1.0.7"

These are ordinary http GET requests and should be treated like vanila traffic. These are usefull for statistics and info gathering about your visitors. There is a bunch of good web analys tools out there, that can easily accomplish the task of presenting the log data in more human readable form.
Webalizer is one of such tools, and it's installed by default in many Linux distributions along with Apache.
Usually you can find Webalizer's script output under /var/www/usage or similar.

The cron job (for Webalizer) is found under /etc/cron.daily/00webalizer. The script looks like this;

#! /bin/bash
# update access statistics for the web site

if [ -s /var/log/httpd/access_log ] ; then
/usr/bin/webalizer
fi

exit 0

A simple bash script that calls the webalizer binary (/usr/bin/webalizer) and parses the access_log file under /var/log/httpd/. So remember to change the PATH or the name of the access_log file if you don't run the default prefixes.

End of part I

To be continued ....