Highlighted
Absent Member.
Absent Member.
641 views

How to define Agregation Limits

Hi,

I am trying to decide which type of aggregation i need.

Does anyone help me about how to review the events or decide how to make a eficient aggregation?

Is there anyway to view the SmartConnector performance?

For example i have a SmartConnector that is receiving 10M events/day from CheckPoint firewall i need to aggregate it.

If i make aggregation the rules i have created are still valid for the events been aggregated?

Thanks and regards.

- Jose Manuel Mendez

Labels (2)
0 Likes
Reply
4 Replies
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: How to define Agregation Limits

Here is a suggestion for Checkpoint aggregation:

CheckPoint Aggregation

Time Interval:  10 Seconds

Event Threshold:  30 Events

Field Names:  deviceAddress,deviceVendor,deviceProduct,deviceEventClassId,deviceSeverity,deviceInboundInterface,agentAddress,agentSeverity,sourceAddress,destinationAddress,destinationPort,name

Preserve Common Fields:  Yes

The risk to enabling aggregation is that you can impact your existign content. You will need to do some validation to ensure that you are deriving any event toals from the aggregated event count instead of a count of the base events since they will often represent more than a single event after aggregation.

As to viewing performance, it depends on what you are after. You can monitor event rates via the ESM dashboards. More detailed analysis can be done at the connector by using the LogFu tool to track a large number of different data points. LogFu works by analyzing the connector logs, so if you want to run it offline you can simply downlaod the logs and place them into the logs directory of a local connector install and run logfu there.

HTH,

Dean

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: How to define Agregation Limits

Hi Dean,

Thanks for the information.

When you tell  Field Names, this fields must be also in Fields to Sum?

Regards

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: How to define Agregation Limits

Hey Jose,

When setting up aggregation there are 2 main settings ... one that controls a time count and one that controls an event count. Essentially they work like this assuming that you have set 30 seconds and 100 events ...

The connector will aggregate similar events until it either reaches 100 similar events within 30 seconds, or the 30 second window is crossed. If the connector reaches 100 events in say 10 seconds, it will then send that off. If it has only reached 70 events by the time 30 seconds comes around it will send what it has.

When choosing what fields to aggregate, you need to take careful consideration of your use cases. The reason being that only the data from the fields you choose to aggregate will be forwarded. So that means if you aggregate on say vendor, product, deci, src address, dest address, dest port, only that information will be available in the event in ESM.

There is an additional option called "preserve common fields" which will also send any other data that is common for all events through with the event. So say for the above example, the field agent hostname was the same for all events in a batch of 100, even though you didn't specifically aggregate on that field, that information would be sent to ESM.

Cheers

Mark

0 Likes
Reply
Highlighted
Absent Member.
Absent Member.

Re: How to define Agregation Limits

Hi,

Aggregation comes with lot of R&D with the events it takes time to decide an aggregation for a device.

Two things to consider

1)What are all the fields you dont want to lose data, choose all those fields

2)Create a filter with the above fields and apply it into a active channel... Start Analysing from here:)

After Analysis u can confirm the time duration and the fields.

Regards,

Vivek

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.