Parsing IIS Logs
I had to manually work with some IIS logs to carry out statistical analysis for a one off task. Yes I know if I had time I could express a business case for say Splunk.
Now, I remember the useful PREPLOG tool from MicroSoft, a simple tool written in C++ to pre-process logs for further processing in say Excel.
So I prepared to use it, however it crashed. After several attempts, I found that others had this problem, as the code is buggy. It is a simple fix however.
So all that I needed to do was to make some minor changes to the C++ code and recompile.
I geared up to add the additional components to Visual Studio 15 to allow for C++ compilation, however it requires 4GB of disk space!
Development Workflow
I found a compiler from Embarcadero that is less than 190MB. It's a command line one but works well. To use it. install, add an entry to you PATH
environment variable ( see excellent guide here http://rudeserver.com/how-to/install-borland/#step3 ) and compile using:
bcc32c.exe preplog.cpp
Visual Studio Code makes an excellent editor for *.cpp files, so I don't get slowed down by an IDE.
I suppose I could configure GULP or some other task runner to recompile, may be later.
Result
Using:
PRPEPLOG.EXE iis.log > out.log
Result, is a beautifully parsed file, ready for further processing.
More improvements
I found that my log files contained multiple headings across the length of the file. GRRR!
So I tweaked the PREPLOG program by:
- check if row contains the headers, keep the row, but chop off the first few (9) characters
- ignore any subsequent rows as found in 1)
- skipping any *other* row beginning with #
The tweaked C++ code file can be viewed as a GIST.