Highlighted
Absent Member.
Absent Member.
845 views

ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Hi everybody,

Roberto published this worrying note about issues in Logger related to boolean operators influencing search results. I quote it below.

Can anyone share any additional technical details related to this issue?

regards,

maciej

http://seclists.org/bugtraq/2015/Jul/147

HP ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators


From: roberto () logsat com

Date: Fri, 31 Jul 2015 03:14:17 GMT

HP ArcSight Logger is a log management software used to collect and analyze logs from multiple sources to aid in

investigations and audit.

There are several flaws in the search capabilities in the software that cause it to provide invalid search results for

any query that uses boolean expressions. This means that ANY query to search thru data in the logs ArcSight collected

is potentially incorrect if the query contains more than one search term.

The impact of these bugs are huge. Any court case where forensics evidence was provided via HP ArcSight Logger is

compromised as the resulting data is potentially incorrect and not forensically valid. Intrusions and attacks can go

undetected as log data relative to the attack can be missing from searches performed by ArcSight Logger.

The above are just some examples. The main problem is that the user/investigator is unaware that the results are

incorrect as usually such searches result in millions of returned records that need to be filtered by applying

conditions to remove non-relevant data. The bugs present in ArcSight result in incorrect filtering thus preventing the

display of relevant records that should have been returned but have not. This will prevent such data fro ever being

seen by an investigator/administrator thus missing the attack/intrusion, or even missing exculpatory evidence in case

someone is unjustly accused.

HP has confirmed several of the bugs affecting their product, and identified them as bugs with the following

identifiers:

LOG-14814 - deals with ArcSight Logger providing incorrect results when using the boolean operators "AND" "OR" "NOT" to

find records

LOG-14897 - deals with ArcSight Logger incorrectly allowing users to use the GUI to drill down record results by

clicking on some result fields, when in fact those fields are not searchable. This results in incorrect results since

the user is not informed that the boolean expression will not yield the data being looked for.

LOG-14896 - deals with the GUI not distinguishing between CEF vs non-searachable columns, again as in LOG-14897

resulting in incorrect results.

LOG-14895 - In full text searches some fields should not be available to click on and add to the search terms

The bugs affect ArcSight Logger v5 and v6. It is unknown if previous versions or if other ArcSight products are

affected.

Labels (3)
Tags (3)
0 Likes
Reply
5 Replies
Highlighted
New Member.

Re: ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Watching this closely as well. This has serious implications.

Thanks

0 Likes
Reply
Highlighted
Frequent Contributor.
Frequent Contributor.

Re: ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Please also see this post:

0 Likes
Reply
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Hi Maciej,

The official response is now posted on Protect724 . In a nutshell, the issues are limited to raw syslog search and stem from documented limitations of raw system search as opposed to CEF field based search. Most importantly, they are all in line with those limitations and do not imply "inconsistent search results" as the 3rd party announcement implied.

~ Ofer

0 Likes
Reply
Highlighted
Frequent Contributor.
Frequent Contributor.

Re: ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

This issue may be related to raw syslog but the similar issue I posted is specifically related to field based queries. 

0 Likes
Reply
Highlighted
Acclaimed Contributor.
Acclaimed Contributor.

Re: ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Just to reiterate the work that Mary and Ofer have done here - its unfortunate that these bug reports were posted up with the descriptions provided. I am all for sharing and openness, but we also need to understand the impact, relevance and detail involved. These particular reported bugs are based on raw event searches and "by design" operations - two are user interface options and certainly not critical.

However, I wanted to clarify something here - these are relevant when you are feeding direct raw syslog directly to Logger and not using any SmartConnectors at all. This is a feature which is supported and has been around for a number of years, but its also a very rare feature for customers to use. Almost all customers make use of SmartConnectors in some way and the issues reported here will not affect Logger. Its only when you are feeding raw syslog directly into Logger (not using SmartConnectors) do you get these usability issues.

Also, the point about "NULL" operators is by design. We follow the SQL rules of processing. However, we are investigating options to add some dual support in the future. Interestingly though, Splunk does not operate this way and I suspect that this is where the confusion has come from. Any SIEM / Log Management tool that utilizes or leverages SQL processing will operate this way - so it highlights a great point with regards to understanding how processing is done, what the impacts are and knowing what you expect to see - you can't assume all products are the same and portability of queries isn't as transparent as they might seem! Interestingly though, I understand that IBM QRadar also operates with the IS NULL / NOT NULL operators are for identifying NULL fields - so the same way that ArcSight operates (please correct me if I am wrong).

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.