Doing the data analysis job better

As I was talking to a collegue this week, there is something that makes me think that we can do stuff better. We agreed that there is a certain mentality to not be satisfied with the way things are and to try to build upon things and improve process, flow, efficiency. (Please don't confuse me as a humorless German because of that- I just like things to work better and I abhore the tedious tasks whenever possible.)

So on Friday I was looking at this matter I have been assigned to and itmhad a ton of Event logs to go through and examine for information that may lead me to understand how a group of servers may have been compromised in the first place. In this case there were about 25 different Event log files with many of them in different places in the directory structure that was sent to me. Going through each of these was going to be a challenge. I am always hesitant of moving files around when it involves a case I am working on, mainly because you making track into a situation. (Keep in mind that I always work on the backup of the data so if the rare occasion disaster strikes, I dont have to go back to the client asking for a do-over. Bad Ju Ju!) I also think when you move the data around you may have issues keeping one set event logs separated and distiguisable from another set. I put into use a tool that Harlan Carvey came up with in his book Windows Forensic Analysis, that takes event logs and can output them in .CSV format so they can be inported into Excel or any other spreadsheet. (I like to run them through a strings-searching tool myself, as sometimes the resultant .CSV files are too big for Excel to handle.) So you would think it is easy; just take each of the event logs and run them through LSEVT2.EXE and I would get my .CSV files to search through! Easy for one or two, but for 25, its a pain.

So I figured out the way to make a list of each of the files, in each of the directories they resided in; remove the white-space (I hate white-space; only in a few cases does anything good ever comes from it and you will see one of those today) from the listings; put a numerical designation beside each file listing so I could tell them apart when the .CSV files were made; and perform a For - Do batch file loop that would read my listfile and for each listing take the filename and run it through LSEVT2.EXE and then redirect the output to a .CSV file with the number as the file name. It worked and I was able to get off the keyboard and let this run by itself without me directly managing it. It took just under 2 hours to run so in my mind it saved me (and the client) 2 hours of tedious bordom just by spending 20 minutes of a little programming, researching, troubleshooting, and testing. Here is what it looks like in a 'more visual sense':

Suppose I have 6 event logs, all in separate directories and the only thing similar is their extention (.EVT):

F:\Big Case\Web Server\SystemEvent.evt
F:\Big Case\Web Server\ApplicationEvent.evt
F:\Big Case\Web Server\SecurityEvent.evt
F:\Big Case\BES Server\System.evt
F:\Big Case\BES Server\Application.evt
F:\Big Case\BES Server\Security.evt

A DIR /s *.evt >> filelist.txt would put the files in one file, filelist.txt. But it still has white-space (which broke it the first couple times I tried it.) I would put it through Notepad and replace all white space so it would look something like this:

F:\Big_Case\Web_Server\SystemEvent.evt
F:\Big_Case\Web_Server\ApplicationEvent.evt
F:\Big_Case\Web_Server\SecurityEvent.evt
F:\Big_Case\BES_Server\System.evt
F:\Big_Case\BES_Server\Application.evt
F:\Big_Case\BES_Server\Security.evt

Then I would put a number next to each (the only time I approve of white-spaces is with tokens) and it would look like this:

F:\Big_Case\Web_Server\SystemEvent.evt 1
F:\Big_Case\Web_Server\ApplicationEvent.evt 2
F:\Big_Case\Web_Server\SecurityEvent.evt 3
F:\Big_Case\BES_Server\System.evt 4
F:\Big_Case\BES_Server\Application.evt 5
F:\Big_Case\BES_Server\Security.evt 6

This file would be saved as filelist.txt and in my program code I would designate %%num as the number and %%fil as the path-and-filename.

The heart of the for-do command would look something like this:
for /f "tokens=1,2" %%fil in (filelist.txt) do echo %%num && LSEVT2.EXE -c -f %%fil > %%num.csv

That is the general gist of how it works and it became a complete time saver. I owe heaps of gratitude to my colleague, Harlan Carvey. If it wasn't for his expertise and talent, I would never have dreamed this up.

Time form me to stop blogging and get on back to the case.