Established Member.

Custom parser for IIS issues

I have developed a flexconnector which parses a custom IIS log (using IIS advanced logging) into ArcSight. For the most part, it works fine.

However, I keep getting a parsing error like such:

[2017-07-05 20:03:36,634][ERROR][default.com.arcsight.agent.baseagents.k.a.a][processLine] [java.lang.RuntimeException: Parser : Unknown State !(a with state :2).  Line:2017-07-06|02:03:35.388|"REDACTED"|GET|/careers/|NONE|"https"|200|"REDACTED"|443|"android-app://com.google.android.googlequicksearchbox"|"NO COOKIE"|2017-07-06 02:03:35.388|2017-07-06 02:03:35.388|7093|509|"unknown_user"|"Mozilla/5.0 (Linux; Android 7.0; SM-N920V Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.125 Mobile Safari/537.36"|""|0|"HTTP/1.1"|"REDACTED"||"no_proxy_used"

The only difference I can find with errors like this is the trailing slash in the /careers/ field. This is based on comparing this event to others that are parsing correctly. There are other errors like this which all have the trailing slash. I am not 100% sure that this is causing the parsing issue, and have no other way to troubleshoot it. Is there another level of logging which would provide more information on the parsing error?

This field is from cs-uri-stem (renamed to url for this instance) and is parsed into the RequestUrl field.

I have tried using __replaceAll(url,\/$,"") in the parser to replace the trailing slash with nothing, but cannot seem to get it to work properly.

Here is the parser I am currently using:


 token[12].format=yyyy-MM-dd HH:mm:ss.SSS
 token[13].format=yyyy-MM-dd HH:mm:ss.SSS
 event.name=__stringConstant("IIS Action")
 event.deviceCustomString1Label=__stringConstant("WebSite Name")
 event.deviceCustomString2Label=__stringConstant("URL Query Date")
 event.deviceCustomString4Label=__stringConstant("Protocol Substatus")
 event.deviceCustomString5Label=__stringConstant("Protocol Version")

IIS fields in the log file:

date time sitename method url url_query_data protocol server_status_code content_path server_port referer cookie_data request_start_time request_end_time bytes_sent bytes_received user_name browser cliient_ip protocol_substatus protocol_version server_hostname server_ip proxy

Parses correctly:

2017-07-05|00:00:00.671|"REDACTED"|GET|/about/about.html|NONE|"http"|403|"REDACTED"|80|"NO REFERER"|"NO COOKIE"|2017-07-05 00:00:00.671|2017-07-05 00:00:00.671|1445|76|"unknown_user"|"unknown_browser"|""|4|"HTTP/1.0"|"REDACTED"||"no_proxy_used"

Can anyone assist? I know I am close to getting this to work properly, but am missing something. It is critical that I get this working.

Thanks in advance!

Labels (2)
1 Reply
Outstanding Contributor.. LakeHealthInfoS Outstanding Contributor..
Outstanding Contributor..

Re: Custom parser for IIS issues

I see what your going for here but would also submit - if all you want is a specific field or group of fields -


the standard multi-server IIS log reader will work then you can create a filter to parse deeper content out of the parsed fields 


We do this for Active Synch access on the Exchange CAS servers - for corporate mobile mail monitoring - to unauthorized devices as each user needs to request mail acces on the phoes - we also use it to enforce VMware Boxer access as the USER agent - to ensure AirWatch compliance.

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.