Analyze Process Monitor (ProcMon) XML in JSON format

The Process Monitor is the tool in Sysinternals toolset by Microsoft. It is handy to capture & analyze all process activities. But the GUI tool tends to consume more memory, and it’s time-consuming for searching for particular events.

Of course, ProcMon can export data as CSV or XML. CSV format is handy for analyzing events. But it lacks process information details.

"Time of Day","Process Name","PID","Operation","Path","Result","Detail"
"10:00:50.9444337","explorer.exe","7812","RegQueryKey","HKCU\Software\Classes","SUCCESS","Query: Name"
"10:00:50.9445179","explorer.exe","7812","RegQueryKey","HKCU\Software\Classes","SUCCESS","Query: HandleTags, HandleTags: 0x0"
"10:00:50.9445443","explorer.exe","7812","RegQueryKey","HKCU\Software\Classes","SUCCESS","Query: HandleTags, HandleTags: 0x0"

For XML, the ProcMon’s XML is like below (UTF-8 with BOM). All data is under tag <procmon>. That means if you’d like to process this XML with DOM, then that required to expand all data on memory. However, ProcMon’s data tend to grow huge. That could grow up to a few GBs within a few minutes.

U+FEFF<?xml version="1.0" encoding="UTF-8"?>
<procmon><processlist><process>
<ProcessIndex>1</ProcessIndex>
<ProcessId>10516</ProcessId>
......
</process>
</processlist><eventlist>
<event>
<ProcessIndex>5</ProcessIndex>
....
</event>
</eventlist></procmon>

Then, I tried to convert that XML into JSONL format just for tags under <processlist> and <eventlist>. Then, we can analyze data with the tool like jq. To convert ProcMon’s XML, I created a ruby script like below. (Please install gems before run)

$ gem install nokogiri
$ gem install activesupport

#!/bin/ruby

require 'nokogiri'
require 'json'
require 'active_support/core_ext/hash/conversions'

r = Nokogiri::XML::Reader(STDIN)
r.each do |node|
  if node.depth == 2
    next unless node.name == 'process' or node.name == 'event'
    puts Hash.from_xml(node.outer_xml).to_json
  end
end

Then, run like below. I prefer I split process and event after this.

$ cat Logfile.XML | ruby convert.rb > logfile.json
$ jq -c .event logfile.json | grep -v ^null > event.json
$ jq -c .process logfile.json | grep -v ^null > process.json

Then, you can analyze like below. (histogram for event result)

$ jq .Result event.json | sort | uniq -c | sort -n
   1 "NO MORE MATCHES"
   2 "FAST IO DISALLOWED"
   2 "NOT A DIRECTORY"
   3 null
   4 "NO SUCH FILE"
  10 "BUFFER TOO SMALL"
  14 "CANCELLED"
  20 "0xC00004AE"
  37 "FILE LOCKED WITH WRITERS"
  38 "NO EAS ON FILE"
  89 "NO MORE ENTRIES"
 103 "END OF FILE"
 173 "NAME INVALID"
 211 "NO MORE FILES"
 220 "NAME COLLISION"
 471 "PATH NOT FOUND"
 612 "FILE LOCKED WITH ONLY READERS"
 862 "OPLOCK HANDLE CLOSED"
1786 "IS DIRECTORY"
1864 "INVALID PARAMETER"
1910 "REPARSE"
3427 "NOT REPARSE POINT"
4849 "BUFFER OVERFLOW"
11970 "NAME NOT FOUND"
104456 "SUCCESS"