Welcome to the first of a 100 best practice posts, these range from newbie to advanced user and consist of tips, tricks and best practices for ArcSight.
Filter Criteria: type
When using the "event.type" CEF field in a filter or rule you have the following options:
Action, Base, Aggregated, Correlation
This is likely a field you want to leverage in ALL content! Why?
Well it specifies if you want to fire a correlation event from a base or non-correlation events, or subsequently if you would like your correlation event to be based on another correlation event! You can have rules that trigger other rules and so on but be warned!
Rules that do not specify:
type != Correlation
Can easily end up in an infinite loop
Therefore this ArcSight Pro Tip is to begin all rule content specifically with this filter entry unless you specifically want to match on rule generated events (non-base events):
type != Correlation
Reminder != means "not equal to"
Solved! Go to Solution.
This is a great general use tip. However, it's worth also adding that the console likes to put your "type != Correlation" condition at or near the very top of the conditions, which is good for resource types that query the database but really bad for rules. If it's the first condition in a rule and 90%+ of your events are non-correlation events, then 90%+ of your events match the first condition and have to move on to the second. (Instant entry in the "Partial Matches" list on the rules dashboard! )
Always make sure this condition is near the bottom in a rule!
Great info Chad, rule order and filter definition can in some cases greatly impact performance.
Don't give away all the pro tip's in one post! jk
But also if your wondering why use type != Correlation over type = Base
By using type=Base you will miss all aggregated events with your content which can cause some very confusing or unwanted results...
Sure, but if the first tip leads someone to impacting the performance of their manager because they didn't know to wait for a later tip, that's bad too. lol I'll keep an eye out for the future posts.
No, this will not lead to a Partial Match. Matching a condition of an event alone will not lead to a Partial Match. Partial matches occur when all the conditions of an event are matched before all the event types and aggregation threshold is met.
I know we're getting off topic from the original point, and I may be incorrect that it will show up on the dashboard (was just looking for a playful way to explain the impact), but having a condition at the top of the rule that matches on nearly everything DOES have a huge impact on performance. If suddenly all your rules have to move on to an additional condition before eliminating the event as a match, the correlation engine will begin to struggle (and potentially choke). Bad condition ordering alone can cause ESM to fail to handle incoming traffic and lead to connectors caching. I strongly disagree with saying there can be no negative impact to performance if this tip isn't utilized correctly... it's an extremely important tip, but it does come with a caveat.
Of course, we also shouldn't assume that Rules should ONLY run against Base events.
A ton of stock Rules run against Correlated events produced by other Rules.
Also if your looking at using !=correlated in rules, remember that the connector can, if configured send aggregated events (or aggregation can be enable latter) and your rule will stop evaluating those events, so I think a better way is to use
type in base,aggregated
Well, in fact If you use "type!=correlated", it will match both base and aggregated events (as well as any other types, if there are or will be any). Because aggregation at connector level never sets type to "correlated".
But many stock rules use "type=base" and that will lead to situation you described - after enabling aggregation at connectors, those rules will not process aggregated events.