Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class
1000 views

Aggregation Fields - WUC

Hello,

We recently installed the Unified connector for pulling Win 2008 domain controller logs.

We configured the connector to use field based aggregation. I added the target user name and source user name fields in addition to the default fields.

Should I include any other fields? Please help me, if you have done a similar setup.

Thanks,

Uday

Labels (2)
0 Likes
7 Replies
Captain
Captain

Hi, Uday

We are using the following fields for aggregation:

deviceAddress,deviceVendor,deviceProduct,deviceEventClassId,deviceSeverity,agentAddress,agentSeverity,sourceAddress, sourceHostName,sourceUserName,sourceUserId,destinationAddress,destinationHostName,destinationUserName,name,message

Also, take a look on the "filter-out" option - the windows logs collect a lot of white noise, you must filter out everything that is not relevant to you.

marian

0 Likes
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Thanks Marian for responding. One problem with aggregating on only these fields is that often critical information is logged in deviceCustomStrings. Example: When a user is added to a group, the custom fields will log very valuable information. Have you considered aggregating on these fields? Or does this not apply for your environment?

Regards,

Uday

0 Likes
Captain
Captain

Hello, Uday. We will look into your suggestion. On the other hand, I think a little unusually to have more than a single "add user to group" event for the lifespan of the aggregation window, so this event will fall out of the aggregation mechanism.

Thank you for your suggestion.

M.  

0 Likes
Vice Admiral
Vice Admiral

Uday,

This is what we currently use on our WUC.

deviceAddress,deviceVendor,deviceProduct,deviceEventClassId,deviceSeverity,agentAddress,agentSeverity,sourceAddress,sourceHostName,sourceUserName,sourceUserId,destinationAddress,destinationHostName,destinationUserName,deviceCustomNumber1,deviceCustomString6,name,message,fileName,filePath

and Filter out Condition:

( categoryOutcome EQ "/Success" And deviceEventClassId In ( "Microsoft-Windows-Security-Auditing:4624", "Microsoft-Windows-Security-Auditing:4634", "Security:538", "Security:540", "Security:576" ) And destinationUserName Contains "$" ) Or ( categoryOutcome EQ "/Success" And deviceEventClassId In ( "Security:672", "Security:673", "Security:674", "Security:617", "Security:677", "Security:676" ) ) Or name In ( "A Kerberos service ticket was requested.", "A Kerberos authentication ticket (TGT) was requested.", "Privileged Service Called", "Privileged object operation", "A new process has been created", "A process has exited" ) Or deviceEventClassId In ( "Microsoft-Windows-Security-Auditing:4662", "Microsoft-Windows-Security-Auditing:4672", "Microsoft-Windows-Security-Auditing:4932", "Microsoft-Windows-Security-Auditing:4933", "Security:836", "Security:837", "Security:552", "Security:562", "Security:565", "Security:566", "Security:576", "Security:578" )

0 Likes
Cadet 2nd Class Cadet 2nd Class
Cadet 2nd Class

Have you seen any performance differences with "in" vs "=". This is at the connector level so I'm not really sure it would make a difference since this isn't indexed yet, but just crossed my mind.

0 Likes
Fleet Admiral
Fleet Admiral

Good question - always a good idea to steer clear from CONTAINS and IN where possible, since they are relatively slow operators, especially on indexed data. I don't think its so critical at the connector level, but its always worth double checking.

Try using Logfu to make sure you have the relevant data on how the connector is working though. It should give you some indications if you have it operating slowly and not being able to keep up. Usually with excessive memory and caching being a good clue here, but you should be able to spot something.

0 Likes
Absent Member.
Absent Member.

The best way I believe to approach this is first to define the events that you want to send to ESM.

Once you have the events, then review the field mapping for each event to the ArcSight Schema.

What you should then have is a list of fields that you need.

When using aggregation you need to ensure that you will get the fields that you need. However it doesn't mean that you need to aggregate on all those fields. What I normally end up doing is having a list of key fields and I also enable "Preserve Common Fields".

For example if you want to maintain all the agent information, then I normally select agentID only. This is because all other agent related fields would be the same from the same connector and thus be shown, because you enabled "Preserve Common Fields".

Likewise if you select all the fields, then of course you won't gain much in aggregation. Proxy data is a typical example here. If you need the URL information, then you need to aggregate on it, but of course URL's are unique, so the aggregation benefit you gain is somewhat low (we normally get about 15-20%).

From a performance perspective we found that the time frame and event count has a much bigger impact on performance than the number of aggregated fields does.

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.