Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
yaseminb1 Trusted Contributor.
Trusted Contributor.
2175 views

[SmartConnector - Windows DNS Analytics logs]

Jump to solution
Hi, I would like to know whether WUC or Windows Native connector support collecting Windows DNS Analytics logs. Path: "Event Viewer->Applications and Services Logs->Microsoft->Windows->DNS-Server->Analytical" https://www.solutionary.com/resource-center/blog/2016/01/dns-logging/
Labels (1)
0 Likes
1 Solution

Accepted Solutions
Honored Contributor.. gcrespo1 Honored Contributor..
Honored Contributor..

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution

 Hi everyone,

I developed a custom parser for this DNS Analytical logs. Check https://community.saas.hpe.com/t5/ArcSight-Questions/DNS-Analytic-Event-Connector/qaq-p/1516565

 

Regards,

Gabriel Crespo

 

EDIT: I have just attached the parser files.

5 Replies
Marijo Mandic Acclaimed Contributor.
Acclaimed Contributor.

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution

Hello,

1) SmartConnector for Microsoft Windows Event Log -- Native:
https://community.saas.hpe.com/t5/ArcSight-Connectors/SmartConnector-for-Microsoft-Windows-Event-Log-Native/ta-p/1585123

Create and Deploy Your Own Parser -> page 40

2) SmartConnector for Microsoft Windows Event Log -- Unified:
https://community.saas.hpe.com/t5/ArcSight-Connectors/SmartConnector-for-Microsoft-Windows-Event-Log-Unified/ta-p/1585246

Create and Deploy Parsers for System and Application Events -> page 33

3) So basically if events are located in your "Event Viewer" of Windows host you should be able to process those event but you will need to deploy Custom parser.

4) For some ideas:
a) Procedure for creating custom parsers in WINC:
https://community.saas.hpe.com/t5/ArcSight-Questions/Procedure-for-creating-custom-parsers-in-WINC/qaq-p/1521400

b) Custom parser for windows events - Terminal Services Gateway:
https://community.saas.hpe.com/t5/Interact-Questions/Custom-parser-for-windows-events-Terminal-Services-Gateway/qaq-p/1544019

Regards,
Marijo

Lewuu Super Contributor.
Super Contributor.

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution

Greets,
 I have been working on this for a bit as well - opened a case with support 5319966077, and basically validated that there isnt a connector for the analytic binary that you are trying to pull in. (%SystemRoot%\System32\Winevt\Logs\Microsoft-Windows-DNSServer%4Analytical.etl.)  As a workaround we started working on a stop gap solution but haven't vetted this in production yet.

We have a scheduled task that runs every 15min that stops the service, writes the file to csv, clear the log and restarts the service

cmd line to write the csv - "C:\Windows\System32\tracerpt -of CSV -o "D:\dns\test_%date:~4,2%%date:~7,2%%date:~10,4%_%time:~0,2%%time:~3,2%%time:~6,2%.csv" -l c:\Windows\System32\Winevt\Logs\Microsoft-Windows-DNSServer%%4Analytical.etl"

Then install a folder follower connector to the location you are writing the file to.. we are currently just renaming to .processed but if we go this route we will delete either instantly via connector or another script to clean up anything older than X.

agents[0].foldertable[0].configfile=dns_tracelog_file
agents[0].foldertable[0].configfolder=D\:\\Program Files\\ArcSightSmartConnectors\\Microsoft DNS POC\\current\\user\\agent\\\\flexagent\\dns_tracelog_file\\
agents[0].foldertable[0].configtype=sdkrfilereader+30

problem we found with this method is translating window epoch time to unix - haven't spent cycles furthering the parser yet

# TODO: Timestamp Conversion from LDAP EPOCH to Unix EPOCH
#
# https://stackoverflow.com/questions/4647169/how-to-convert-ldap-timestamp-to-unix-timestamp

 

dns_tracelog_file.sdkrfilereader.properties

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# Useful Link:
# https://technet.microsoft.com/en-us/library/dn800669(v=ws.11).aspx
trim.tokens=true
contains.empty.tokens=true
start.at.line=3

do.unparsed.events=true

# This REGEX is designed to pull the EVENT ID from the message so that it can be given to the sub-message for differentiation.
regex=(\\w+-\\w+-\\w+-\\w+)\\s*,\\s*(\\d+),\\s*(\\d+),(.*)

# Assign the four expected fields, ignore the first 2
token.count=4
token[0].name=cpIgnore1
token[0].type=String
token[1].name=cpIgnore2
token[1].type=String
token[2].name=cpEventId
token[2].type=String
token[3].name=cpMessage
token[3].type=String


event.deviceVendor=__getVendor("Microsoft")
event.deviceProduct=__stringConstant("DNS Log")
event.message=cpMessage

event.deviceEventClassId=cpEventId

# EVENT ID is the unique identifier and MESSAGE is the text to process for each sub-messages
submessage.messageid.token=cpEventId
submessage.token=cpMessage

# Parser written to allow parsing of the 26 known EVENT IDs but only parsing the ones we need (the rest just get named).
submessage.count=26

# TODO: Timestamp Conversion from LDAP EPOCH to Unix EPOCH
#
# https://stackoverflow.com/questions/4647169/how-to-convert-ldap-timestamp-to-unix-timestamp
#
# Potential: __createLocalTimeStampFromSecondsSinceEpoch(__(__divide(win epoch, 10000000),11644473600))
submessage[0].messageid=256
submessage[0].pattern.count=1
submessage[0].pattern[0].regex=\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S*,\\s*\\S*,\\s*\\S*,\\s*\\d*,\\s*,\\s*,\\s*\\S*,\\s*,\\s*\\S*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\d*,\\s*\\S(\\S+)\\.\\S.*
submessage[0].pattern[0].fields=event.deviceAddress,event.sourceAddress,event.deviceCustomString1
submessage[0].pattern[0].extramappings=event.name=__stringConstant("Lookup - Query Received")|event.deviceCustomString1Label=__stringConstant("QNAME")

submessage[1].messageid=257
submessage[1].pattern.count=1
submessage[1].pattern[0].regex=\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S*,\\s*\\S*,\\s*\\S*,\\s*\\d*,\\s*,\\s*,\\s*\\S*,\\s*,\\s*\\S*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\d*,\\s*\\d*,\\s*\\S(\\S+)\\.\\S.*
submessage[1].pattern[0].fields=event.deviceAddress,event.sourceAddress,event.deviceCustomString1
submessage[1].pattern[0].extramappings=event.name=__stringConstant("Lookup - Response Success")|event.deviceCustomString1Label=__stringConstant("QNAME")

submessage[2].messageid=258
submessage[2].pattern.count=1
submessage[2].pattern[0].regex=(.*)
submessage[2].pattern[0].fields=event.message
submessage[2].pattern[0].extramappings=event.name=__stringConstant("Lookup - Response Failure")

submessage[3].messageid=259
submessage[3].pattern.count=1
submessage[3].pattern[0].regex=(.*)
submessage[3].pattern[0].fields=event.message
submessage[3].pattern[0].extramappings=event.name=__stringConstant("Lookup - Ignored Query")

submessage[4].messageid=260
submessage[4].pattern.count=1
submessage[4].pattern[0].regex=\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S*,\\s*\\S*,\\s*\\S*,\\s*\\d*,\\s*,\\s*,\\s*\\S*,\\s*,\\s*\\S*,\\s*\\d*,\\s*\\d*,\\s*\\d*,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\S(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})\\S,\\s*\\d*,\\s*\\S(\\S+)\\.\\S.*
submessage[4].pattern[0].fields=event.destinationAddress,event.deviceCustomString2,event.deviceCustomString1
submessage[4].pattern[0].extramappings=event.name=__stringConstant("Recursive Query - Query Out")|event.deviceCustomString1Label=__stringConstant("QNAME")|event.deviceCustomString2Label=__stringConstant("Interface IP")

submessage[5].messageid=261
submessage[5].pattern.count=1
submessage[5].pattern[0].regex=(.*)
submessage[5].pattern[0].fields=event.message
submessage[5].pattern[0].extramappings=event.name=__stringConstant("Recursive Query - Response In")

submessage[6].messageid=262
submessage[6].pattern.count=1
submessage[6].pattern[0].regex=(.*)
submessage[6].pattern[0].fields=event.message
submessage[6].pattern[0].extramappings=event.name=__stringConstant("Recursive Query - Response Query Timeout")

submessage[7].messageid=263
submessage[7].pattern.count=1
submessage[7].pattern[0].regex=(.*)
submessage[7].pattern[0].fields=event.message
submessage[7].pattern[0].extramappings=event.name=__stringConstant("Dynamic Update - Update In")

submessage[8].messageid=264
submessage[8].pattern.count=1
submessage[8].pattern[0].regex=(.*)
submessage[8].pattern[0].fields=event.message
submessage[8].pattern[0].extramappings=event.name=__stringConstant("Dynamic Update - Update Response")

submessage[9].messageid=265
submessage[9].pattern.count=1
submessage[9].pattern[0].regex=(.*)
submessage[9].pattern[0].fields=event.message
submessage[9].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - IXFR Request Out")

submessage[10].messageid=266
submessage[10].pattern.count=1
submessage[10].pattern[0].regex=(.*)
submessage[10].pattern[0].fields=event.message
submessage[10].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - IXFR Request In")

submessage[11].messageid=267
submessage[11].pattern.count=1
submessage[11].pattern[0].regex=(.*)
submessage[11].pattern[0].fields=event.message
submessage[11].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - IXFR Response Out")

submessage[12].messageid=268
submessage[12].pattern.count=1
submessage[12].pattern[0].regex=(.*)
submessage[12].pattern[0].fields=event.message
submessage[12].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - IXFR Response In")

submessage[13].messageid=269
submessage[13].pattern.count=1
submessage[13].pattern[0].regex=(.*)
submessage[13].pattern[0].fields=event.message
submessage[13].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - AXFR Request Out")

submessage[14].messageid=270
submessage[14].pattern.count=1
submessage[14].pattern[0].regex=(.*)
submessage[14].pattern[0].fields=event.message
submessage[14].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - AXFR Request In")

submessage[15].messageid=271
submessage[15].pattern.count=1
submessage[15].pattern[0].regex=(.*)
submessage[15].pattern[0].fields=event.message
submessage[15].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - AXFR Response Out")

submessage[16].messageid=272
submessage[16].pattern.count=1
submessage[16].pattern[0].regex=(.*)
submessage[16].pattern[0].fields=event.message
submessage[16].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - AXFR Response In")

submessage[17].messageid=273
submessage[17].pattern.count=1
submessage[17].pattern[0].regex=(.*)
submessage[17].pattern[0].fields=event.message
submessage[17].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - XFR Notification In")

submessage[18].messageid=274
submessage[18].pattern.count=1
submessage[18].pattern[0].regex=(.*)
submessage[18].pattern[0].fields=event.message
submessage[18].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - XFR Notification Out")

submessage[19].messageid=275
submessage[19].pattern.count=1
submessage[19].pattern[0].regex=(.*)
submessage[19].pattern[0].fields=event.message
submessage[19].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - XFR Notify ACK In")

submessage[20].messageid=276
submessage[20].pattern.count=1
submessage[20].pattern[0].regex=(.*)
submessage[20].pattern[0].fields=event.message
submessage[20].pattern[0].extramappings=event.name=__stringConstant("Zone XFR - XFR Notify ACK Out")

submessage[21].messageid=277
submessage[21].pattern.count=1
submessage[21].pattern[0].regex=(.*)
submessage[21].pattern[0].fields=event.message
submessage[21].pattern[0].extramappings=event.name=__stringConstant("Dynamic Update - Update Forward")

submessage[22].messageid=278
submessage[22].pattern.count=1
submessage[22].pattern[0].regex=(.*)
submessage[22].pattern[0].fields=event.message
submessage[22].pattern[0].extramappings=event.name=__stringConstant("Dynamic Update - Update Response In")

submessage[23].messageid=279
submessage[23].pattern.count=1
submessage[23].pattern[0].regex=(.*)
submessage[23].pattern[0].fields=event.message
submessage[23].pattern[0].extramappings=event.name=__stringConstant("Lookup - Internal Lookup CNAME")

submessage[24].messageid=280
submessage[24].pattern.count=1
submessage[24].pattern[0].regex=(.*)
submessage[24].pattern[0].fields=event.message
submessage[24].pattern[0].extramappings=event.name=__stringConstant("Lookup - Internal Lookup Additional")

submessage[25].pattern.count=1
submessage[25].pattern[0].regex=(.*)
submessage[25].pattern[0].fields=event.message
submessage[25].pattern[0].extramappings=event.name=__stringConstant("Not Handled in Parser File, data in Message field.")

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

maybe someone can add to this

 

good luck....

 

Honored Contributor.. gcrespo1 Honored Contributor..
Honored Contributor..

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution

 Hi everyone,

I developed a custom parser for this DNS Analytical logs. Check https://community.saas.hpe.com/t5/ArcSight-Questions/DNS-Analytic-Event-Connector/qaq-p/1516565

 

Regards,

Gabriel Crespo

 

EDIT: I have just attached the parser files.

yaseminb1 Trusted Contributor.
Trusted Contributor.

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution
Thank you very much.
0 Likes
Highlighted
kenglim1 Valued Contributor.
Valued Contributor.

Re: [SmartConnector - Windows DNS Analytics logs]

Jump to solution

@gcrespo1 Hi, I download the parser and put it in the $ARCSIGHT_HOME\user\agent\fcp\winc\microsoft_windows_dnsserver_analytical folder, after that the connector was restarted. But I wasn't able to see the DNS logs.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.