Menu

Subscribe

Parsing IIS Logs

I had to manually work with some IIS logs to carry out statistical analysis for a one off task. Yes I know if I had time I could express a business case for say Splunk.

Now, I remember the useful PREPLOG tool from MicroSoft, a simple tool written in C++ to pre-process logs for further processing in say Excel.

So I prepared to use it, however it crashed. After several attempts, I found that others had this problem, as the code is buggy. It is a simple fix however.

So all that I needed to do was to make some minor changes to the C++ code and recompile.

I geared up to add the additional components to Visual Studio 15 to allow for C++ compilation, however it requires 4GB of disk space!

Development Workflow

I found a compiler from Embarcadero that is less than 190MB. It's a command line one but works well. To use it. install, add an entry to you PATH environment variable ( see excellent guide here http://rudeserver.com/how-to/install-borland/#step3 ) and compile using: 

bcc32c.exe preplog.cpp

Visual Studio Code makes an excellent editor for *.cpp files, so I don't get slowed down by an IDE.

I suppose I could configure GULP or some other task runner to recompile, may be later.

Result

Using:

PRPEPLOG.EXE  iis.log > out.log

Result

Result, is a beautifully parsed file, ready for further processing.

More improvements

I found that my log files contained multiple headings across the length of the file. GRRR!
Naughty Log File

So I tweaked the PREPLOG program by:

  1. check if row contains the headers, keep the row, but chop off the first few (9) characters
  2. ignore any subsequent rows as found in 1)
  3. skipping any *other* row beginning with #

The tweaked C++ code file can be viewed as a GIST.