Same filter: different active list and query results
Hello. I have the following problem.
I created a rule TEST that populates a new event called TEST. It also adds it to the active list to count it.
My filter looks for these new events named 'TEST".
When I look for them using an active list (Based on that filter) i get X results (the same as in the active list).
When I use this filter in a query I get Y results. Weird thing is, that Y > X.
Of course, the time period is the same. Sometimes the query shows doubled events.
One of my ideas is that I verified the rule with some events yesterday and maybe somehow query captures also that events (but that would be nonsense:/).
Thanks in advance,
Does the rule have the same name as the events it's firing off of? Try adding Event Type = Base to your query and see what you get. Also, is the rule aggregating events or is it a 1-1 ratio (Events to Firings)?
Hi chrisb, thanks for your reply.
1) The events that the rule is firing off of don't have the same name (they are oracle,win,unix logons and the actual rule has a complex name like rule.id.id2 - name etc) - so there is no collision at this point.
2) The rule is aggregating 1-1 and in the query I am looking for the correlated events (produced by the rule).
If the rule fired 3 times, in active channel I see those 3 events and in the AL the count is also 3.
But how in the world does the query return the value 222 ? (yes, those are the real numbers I'm facing).
I've rechecked rules, filters at least dozen times and everything seems to be OK.
Just to double check, you aren’t using generatorName field are you? If so you are potentially seeing all the events where items are being added to the AL. ArcSight does have an issue counting aggregated items under certain conditions but I don’t think that is the case here since you say there is a 1:1 ratio. Of course one thing to do is copy your query for the report and expand the fieldset so you see more than just the collapsed count of events.
Holy crap, that's a HUGE discrepancy. Yeah, I'm stumped at this point . I'm going to ask a stupid question, but is your query keying off the same filter(s) the rule is using or are you defining new conditions for the query independent of the rule's filters?
When you have the query, can't you make it show a report to show all events? When including fields like 'event id' & 'device event class category' & 'device event category' you should be able to pinpoint which events extra AS is showing. Go from there & find the why.
Sorry for no activity in my own topic, but I was caught with some other more prioritized tasks.
I changed the query to show more details about those events and I was pretty suprised.
In the active list and channel one event (lets call it Event1) had 3 occurences and in a query it had 222. It seems like those 222 are the real ones and my rule just did catch only 3.
Another event (Event2) in AL, AC has 11 occ. and the query has 2 more that are real and 4 that are doubled.
I run this report now (for past 2 days) and it seems ok - the numbers are equal.
I'm not sure what's going one, but this may be day-related.
Not that I believe this is strictly what is going on in your case, but this is similar to what you are experiencing.
Queries count events differently than active lists and active channels do.
If you ran a channel without a filter for a 24 hour time frame, this would show you how many events occured during those 24 hours. Example 100,000
If you had a trend without a filter count the events in the same time period, they will come up with drastically different numbers. Example 135,000
I'm not 100% sure on the cause of this, but I can tell you that it is because they count differently. IIRC, when a channel or list counts, it counts each occurance as one. Things like trends count by aggregated event count (i believe)
If an event with an aggregated event count of 5 happens, an active channel or list will show the count as 1
A trend will show the count as 5
Even if they are configured identically they just count differently.
Yep – it is a hateful, hateful thing that they have known about for years. What makes it really fun is in your connector you can have certain fields add their values if you are aggregating. So now you take that field on the ESM side and it is multiplied by the aggregate field count of the event itself. I have tried multiple ways to get around that behavior and there simply isn’t one.
Evnt1 – 2
Evnt2 – 6
Evnt3 – 0
Aggregated becomes 8 which ESM then will count/sum to 24 if you actually try to use that field (aggregatedEventCount = 3).
Aggregated event count is not the case, it's always 1 as these are different events (different orracle session ids) - I've just checked that.
You must remember that the topic is about correlated events, not base ones.
It seems that my rule is not catching all the events and I will check it now to see what's wrong (if anything is).