Sunday, September 13, 2009

How to get AWStats to show Intranet location stats

I was asked by Teva's HR department to do something about the HR intranet "Portal". The requirement is to know how the HR intranet is used, based on the 3 W rules: What, When, and Where. Teva has multiple offices in Israel, and HR went through a major reorganization and restructuring of its services and methodology during the past years.

It is good they came to me first, as some of my colleagues would have made it into a multi-million dollars data warehouse analysis project :)

Having good experience with Awstats, I decided to use this tool to analyze the HR intranet logs.

AWstats provides good enough stats about the What and the When, but I hit a blank wall regarding the Where. There are several plugins that enable geo-ip analysis, but according to the documentation, all of them are useless for intranet only log files.

The plugins from MaxMind use a proprietary format, and do not include my 10.* network anyway.

This left me with the IPFree database and plugin as the only viable option to add branch awareness into the intranet stats.

Geo-IPfree can be found on CPAN. First thing is to download it.
We need a couple of tools that may or may not be included in the IPfree package. txt2ipct.pl and ipct2txt.pl are required for us to make the modifications to the IPfree database.
First, extract the IP database from ipscountry.dat into an editable text file.
perl ipct2txt.pl ./ipscountry.dat ./ips-ascii.txt
Let's suppose that you have 3 branches: London, Rome and Tel-Aviv. Your WAN network segments are respectively 10.1.*, 10.2.*, 10.3.*, and must be mapped into the IPfree database.

Open the ips-ascii.txt file with your favorite text editor and find your LAN/WAN IP range.
You have a line that looks like this
ZZ: 10.0.0.0 10.255.255.255
We will map your locations into codes Z1, Z2 and Z3, because there are no such ISO country codes.

So, replace your ZZ line with these five
ZZ: 10.0.0.0 10.0.255.255
Z1: 10.1.0.0 10.1.255.255
Z2: 10.2.0.0 10.2.255.255
Z3: 10.3.0.0 10.3.255.255
ZZ: 10.4.0.0 10.4.255.255
Just for safety, rename your current ipscountry.dat file, and execute
perl txt2ipct.pl ./ips-ascii.txt ./ipscountry.dat
Now, create a new Geo folder under the plugins folder in the awstats installation, and copy the IPfree.pm and ipscountry.dat files into the new folder.

We have to modify the lib/domains.pm file to recognize the new Z1, Z2 and Z3 domains. Just add them to the end of the list, and keep the new domain names in lower case. The last line of the domains.pm file will now look like this
'zm','Zambia','zr','Zaire','zw','Zimbabwe', 'z1', 'London', 'z2', 'Rome', 'z3', 'Tel Aviv'
The last thing to do is to turn on the geoipfree plugin in the awstats configuration file.

Have fun!!

P.S.
Maybe I will initiate the multi-million dollar web data warehouse project after all ...

1 comment:

  1. Excellent tutorial - got it working fine second try, at first I forgot to use lower case "z" in domains.pm
    I have added icons for our Intranet sites in the awstats icons folder (e.g. z1.png, z2.png) so that they show on the country stats.

    ReplyDelete