Highlighted
Trusted Contributor.. daniel.eddy Trusted Contributor..
Trusted Contributor..
1037 views

Microsoft DNS Trace Log Parser Errors

I am seeing a large amount of parsing errors with the Microsoft DNS Trace Log connector.  I've followed the connector guide to setup the logging options on the DNS server but I still see a large amount of errors in the connector's agent.log file. This is occurring on both Windows Server 2008 R2 and 2012 R2.  The parsing issues seem to be constant but it's worth noting that events still make it to their destination - I'm just not sure if all events are being parsed correctly.

Below is an example of the agent.log file (with confidential information redacted).  This image is also attached to this post in case the inserted image is unreadable.

01-06-2017 9-15-14 AM.png

Has anyone else experienced these issues?  I've tried every modern version of the Windows connector but still get the same results.

Thanks in advance for any help you may be able to provide!


Danny

Labels (1)
0 Likes
4 Replies
Outstanding Contributor.. andrew.dalbor Outstanding Contributor..
Outstanding Contributor..

Re: Microsoft DNS Trace Log Parser Errors

Hey Danny,

I have had the same issue for a long time now.  I opened a case when I first experienced the issue and was told by support that it was because my trace logs had too much data per entry/event.

My DNS trace settings are nothing out of the ordinary and only log the normal stuff.

Andrew

0 Likes
kevquinlan Honored Contributor.
Honored Contributor.

Re: Microsoft DNS Trace Log Parser Errors

Yes there are a few issues with this Connector - i have had the same issues.

If you are not using US data format (i.e. UK OS Locale) then the first issue is that the timestamp parsing does not work.

the old parser is:

Old Setting

event.deviceReceiptTime=__parseMultipleTimeStamp(DateTime,"yyyyMMdd HH:mm:ss","MM/dd/yyyy hh:mm:ss a","MM/dd/yyyy hh:mm:ss")

i have used an override to set to:

New Setting

event.deviceReceiptTime=__parseMultipleTimeStamp(DateTime,"dd/MM/yyyy HH:mm:ss","yyyyMMdd HH:mm:ss","MM/dd/yyyy hh:mm:ss a","MM/dd/yyyy hh:mm:ss")

this is logged with Support and they have a Bug ID CON-16477 which you can quote if it affects you - they should be able to give you a parser override or talk you through it if you haven't done one before.

Apart from this there is a serious lack of parsing of submessages such as DSPOLL, DSWRITE, LOOKUP, RECURSE and probably others, there is a lack of "catch all regex" for some other submessages - all of which caused high use of memory in this Connector when deployed in even a tiny DNS environment (less than 20 hosts).

I have shared a parser override with support that includes updates for these submessages - if this issue is affecting you then raise a ticket and hopefully they will give you some help.

the final issue i have found with it is that this is a multiline file reader and it really struggles to group the event data correctly when either the DNS.log rotates or if it can't keep up with the event flow - meaning on regular intervals you will get a flurry of unparsed / regex doesn't match errors because suddenly the log file appears to start in the middle of the actual event. i haven't got a good fix for this as yet, but with an increase in memory and the above fixes it becomes manageable.

To keep it going through these errors i have been increasing the memory to at least 1024 Mb within the service wrapper.

Make sure you raise tickets with support - otherwise they will not know what fixes to prioritise.


I would however also say that monitoring of this DNS debug log is always going to be high volume / impractical - Admins don't like turning the logging on and it only gives you a one sided view of the conversation. A dedicated DNS Server device / application such as BIND can be better performance wise, but for true DNS Monitoring consider the ArcSight DNS Malware Analytics (DMA) or other tools that carry out the raw analysis and forward the alerts to ArcSight.

Outstanding Contributor.. andrew.dalbor Outstanding Contributor..
Outstanding Contributor..

Re: Microsoft DNS Trace Log Parser Errors

Thanks Kevin!

0 Likes
shezaf1 Acclaimed Contributor.
Acclaimed Contributor.

Re: Microsoft DNS Trace Log Parser Errors

Andrew,

Looking into your particular issue, it seems that you have a specific problem that might require looking into. A DNS event would look like this:

20140816 16:28:56 588 PACKET  01BDA5A0 UDP Snd 192.168.0.2 6a24 R Q [8081   DR  NOERROR] A     (10)akamaiedge(3)net(0)

As you can see, the errors you get are always on the section in the middle in square brackets which is the flags section. The connector seems to try to parse it using the full event regular expression based (the regular expression starts with a timestamp pattern) as if it was the entire event.

What could be the root cause?

  • First, it might be worth checking the original log file to see if such partial events with only a flag section are generated. If so, the culprit would be your server's DNS logging.
  • Otherwise, it might be yet another issue with the connector, either in reading the file or in the parser. You might want to raise a ticket on your issue.

~ Ofer

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.