ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

Hi everybody,

Roberto published this worrying note about issues in Logger related to boolean operators influencing search results. I quote it below.

Can anyone share any additional technical details related to this issue?



HP ArcSight Logger provides incorrect/invalid/incomplete results for queries with boolean operators

From: roberto () logsat com

Date: Fri, 31 Jul 2015 03:14:17 GMT

HP ArcSight Logger is a log management software used to collect and analyze logs from multiple sources to aid in

investigations and audit.

There are several flaws in the search capabilities in the software that cause it to provide invalid search results for

any query that uses boolean expressions. This means that ANY query to search thru data in the logs ArcSight collected

is potentially incorrect if the query contains more than one search term.

The impact of these bugs are huge. Any court case where forensics evidence was provided via HP ArcSight Logger is

compromised as the resulting data is potentially incorrect and not forensically valid. Intrusions and attacks can go

undetected as log data relative to the attack can be missing from searches performed by ArcSight Logger.

The above are just some examples. The main problem is that the user/investigator is unaware that the results are

incorrect as usually such searches result in millions of returned records that need to be filtered by applying

conditions to remove non-relevant data. The bugs present in ArcSight result in incorrect filtering thus preventing the

display of relevant records that should have been returned but have not. This will prevent such data fro ever being

seen by an investigator/administrator thus missing the attack/intrusion, or even missing exculpatory evidence in case

someone is unjustly accused.

HP has confirmed several of the bugs affecting their product, and identified them as bugs with the following


LOG-14814 - deals with ArcSight Logger providing incorrect results when using the boolean operators "AND" "OR" "NOT" to

find records

LOG-14897 - deals with ArcSight Logger incorrectly allowing users to use the GUI to drill down record results by

clicking on some result fields, when in fact those fields are not searchable. This results in incorrect results since

the user is not informed that the boolean expression will not yield the data being looked for.

LOG-14896 - deals with the GUI not distinguishing between CEF vs non-searachable columns, again as in LOG-14897

resulting in incorrect results.

LOG-14895 - In full text searches some fields should not be available to click on and add to the search terms

The bugs affect ArcSight Logger v5 and v6. It is unknown if previous versions or if other ArcSight products are


  • Watching this closely as well. This has serious implications.


  • Hi Maciej,

    The official response is now posted on Protect724 . In a nutshell, the issues are limited to raw syslog search and stem from documented limitations of raw system search as opposed to CEF field based search. Most importantly, they are all in line with those limitations and do not imply "inconsistent search results" as the 3rd party announcement implied.

    ~ Ofer

  • This issue may be related to raw syslog but the similar issue I posted is specifically related to field based queries. 

  • Just to reiterate the work that Mary and Ofer have done here - its unfortunate that these bug reports were posted up with the descriptions provided. I am all for sharing and openness, but we also need to understand the impact, relevance and detail involved. These particular reported bugs are based on raw event searches and "by design" operations - two are user interface options and certainly not critical.

    However, I wanted to clarify something here - these are relevant when you are feeding direct raw syslog directly to Logger and not using any SmartConnectors at all. This is a feature which is supported and has been around for a number of years, but its also a very rare feature for customers to use. Almost all customers make use of SmartConnectors in some way and the issues reported here will not affect Logger. Its only when you are feeding raw syslog directly into Logger (not using SmartConnectors) do you get these usability issues.

    Also, the point about "NULL" operators is by design. We follow the SQL rules of processing. However, we are investigating options to add some dual support in the future. Interestingly though, Splunk does not operate this way and I suspect that this is where the confusion has come from. Any SIEM / Log Management tool that utilizes or leverages SQL processing will operate this way - so it highlights a great point with regards to understanding how processing is done, what the impacts are and knowing what you expect to see - you can't assume all products are the same and portability of queries isn't as transparent as they might seem! Interestingly though, I understand that IBM QRadar also operates with the IS NULL / NOT NULL operators are for identifying NULL fields - so the same way that ArcSight operates (please correct me if I am wrong).