Script for "SmartConnector for Microsoft DHCP File"?

[siemops]

Based-on the smart connector guide for this connector, it appears this connector can only obtain logs from a single DHCP server...or it "could" retrieve DHCP logs from multiple Microsoft DHCP servers if a script is utilized to retrieve the DHCP logs to where they can be read locally/centrally by the connector.

Given this, does anyone have a script (powershell or otherwise) handy to obtain DHCP logs from several Microsoft DHCP servers so our Microsoft DHCP connector can read logs from several Microsoft DHCP servers? Our connector is running on a Windows system.

On a side note, why haven't they created a smart connector, similar-to "SmartConnector for Microsoft DNS Trace Log Multiple Server File", so that this DHCP connector, out of the box, can obtain logs from more than one DHCP server? Am I missing something here?

[mschleich]

Dear Richard,

You can retrieve these logs with only one connector from multiple DHCP Servers without scripting!

I use 1 ArcSight SmartConnector for DHCP file to collect these logs from 8 DHCP Servers.

To achieve this, I have installed the SmartConnector on a VM (Windows Host)

then, I have mapped the Network Share to access the DHCP logs files for all DHCP servers (8 shares) in using the net use command.

and finally, in the connector config, I have added the 8 DHCP logs file locations.

It is working properly, I collect all logs in real-time.

Only 1 issue, currently, it is impossible to know which logs comes from which DHCP server.

[DeviceHostName] and [DestinationHostName] fields are empty.

I have tried to use field extraction but it is impossible (explained below)

This connector is a file reader but it does not put the DHCP log filename in each base events (only into agent:044 (File processing started) and agent:045 (File processing ended: Success))

I have already opened a case to the Support to ask to the HP ArcSight DEV Team to add this information into all DHCP base events based on the file reader thread, it should be possible.

I am waiting for an answer from them. I will inform you back directly I got it.

If you need more information or if you have any question about this, do not hesitate to contact me.

Thanks

Kind regards

Michael

[siemops]

Thank you, Michael.

We have over 400 DHCP servers that we need to collect from. Regardless of how we actually implement this, we'll have many VMs, with many DHCP connectors installed, and a group policy providing access to the DHCP logs folder.

Do you feel mapping drives is the best alternative for 400 + DHCP servers? (Not meant to be a rhetorical question. Just wondering if there is a more scalable alternative.)

[mschleich]

Hi Richard,

No there is other solution but it depends of what you could do in your infra.

You could ask to have a real-time copy of the DHCP logs and you place all these logs into different folder from the same share.

Then with the DHCP connector, you will access all of these DHCP logs file with one Network Share but with different path to access each DHCP server log file.

If we consider that H is the network share to access all DHCP logs from 400 Servers, you will configure the connector like this:

H:\DHCP\server1\DhcpSrvLog-'EEE'.log

H:\DHCP\server2\DhcpSrvLog-'EEE'.log

...

H:\DHCP\server10\DhcpSrvLog-'EEE'.log

The problem is that DHCP logs files have the same name on each DHCP server thus you cannot put all DHCP logs files at the same location.

It is one file by day.

After, the number of connector host will depend of the EPS by DHCP Server.

Maybe you can use the same filter Out than me which reduce the load of 80% (useless Events)

There is other solution like using Snare agent to read each file one by one but I am not sure you could use the DHCP connector.

Morevoer, you will use the categorization created by HP ArcSight.

Now it depends, if you need to collect this logs in real-time or if it is possible to collect them with a delay.

It is huge 400 DHCP servers but if you use the filter above, I am sure you could use only 1 connector for all the 400 servers.

Do you know how many events you have by server without NACK events?

The problem is that this connector is a file reader thus you need to find a way to access these files in real-time. You could install 400 Connectors locally, 1 on each DHCP server, and then you install a syslog connector forwarder mode (CEF) to have only 1 visible connector in ESM but I am not sure it is more scalable.

Thanks

KInd regards

Michael

[siemops]

Thanks for this additional information and help, Michael.

We'll take all of this under consideration during implementation.

I also had a "wild thought" and wondered if the "Microsoft DNS Multiple Server File" smart connector could be leveraged, with a lot of parser overrides...

[mschleich]

Hi Richard,

What do you need to do as parser override with this connector?

I have added parser override and complex map files with many connectors and also high EPS connectors, it is working well.

It is just an question of fine-tuning, increase parser and HTTP multi-threading and CPU, JVM RAM and if it is no enough thus you have to think to add another connector and use a load-balancer.

Thanks

Regards

Michael

[siemops · Accepted solution]

Hey Michael,

Actually we have this connector working nicely now:

For table parameters:

'\\FQDN\DHCPLog$\DhcpSrvLog-'EEE'.log'

This allows us to enter multiple DHCP servers in table parameters....as long as we have the same service account logon-as the connector service.

Also, we are now able to identify our DHCP servers in our DHCP logs, via deviceHostName, by modifying the following parameters in the agent.properties file. So now, we can identify what DHCP servers we are acquiring logs from:

agents[0].foldertable[0].extractfieldnames=deviceHostName

agents[0].foldertable[0].extractregex=\\\\\\\\(.*?)\\\\\\S+

agents[0].foldertable[0].extractsource=File Path

agents[0].foldertable[0].usefieldextractor=true

agents[0].foldertable[x].startatend=true

Thanks for your help.

[mschleich]

Hi Richard,

How you have done to have the deviceHostName information in [filePath] because as explained, this information is only present with agent:044 and agent:045 and not DHCP base events.

Could you please explain me how you have done this, I am really interested?

Thanks

Kind regards

Michael

[siemops]

Hi Michael,

To be honest, this information was previously-provided to us by HP Professional Services to populate deviceHostName for our DNS connectors. (A file-based connector.) Since the DHCP connector is also a "file-based" connector, I tried it and it worked.

However, your question has forced-me to try and understand what's going-on here (lol):

Based-on the FlexConnector Developer's Guide (Page 134, attached), "extractsource=File Path" is used in combination with "usefieldextractor=true". I believe "usefieldextractor" enables/disables this function and "extractsource" let's you choose/use the field "File Path" as part of this connector's events.

"File Path" is the name of the field that is part of the actual parameters we set in the DHCP "table parameters".

The regex is taking the File Path ('\\FQDN\DHCPLog$\DhcpSrvLog-'EEE'.log') and converting it to the deviceHostName.

So, we're not actually obtaining deviceHostName from any DHCP server log. We're obtaining this information from the "table parameters" set for these connectors.

[mschleich]

Hi Richard,

Yes, it is working.

I have no FQDN in the file Path thus I have to choose the proper regex and also I have to use a map file.

Thanks for your help.

I am very satisfied because I have searched for a long time and the support didn't help me.

Regards

Michael

[hkumar202]

Hello Everyone,

I have the similar issue but not sure how to fix it

Have tried the above solution but seems some issue in my regex .

Can someone help here

agents[0].extractfieldnames=deviceHostName
agents[0].extractregex=\opt\Share_Logs\DHCP\(.*?)\.*\S+
agents[0].extractsource=File Path
agents[0].files[0].filename='/opt/Share_Logs/DHCP/SCG-BSDHCP03/DhcpSrvLog-'EEE'.log'
agents[0].files[1].filename='/opt/Share_Logs/DHCP/SCG-BSDHCP04/DhcpSrvLog-'EEE'.log'

[vitz1]

agents[0].extractregex=\opt\Share_Logs\DHCP\(.*?)\.*\S+

you might need to escape your \ correct