How to purge logger "raw events"
Need some assistance figuring out an issue
Looking for the way to purge (not let them age off) the data in the individual field that stores "raw events" that are on logger v6.0 and v6.1
Anyone have any ideas or process steps how to do this?
This is not possible, raw event is just another field in the event schema. Log data is and should be immutable (not able to be changed) once it is written to disk otherwise it loses it evidentiary integrity.
Thanks for responding. Evidence and immutability are not the issue.
But along the same line. when a bank issues a credit statement or a bill and you don't pay it, how can the admissibility of that proven in court? Immutable information and admissibility for log information and computer data is based on best evidence not original evidence.
What most don't understand is that the solution isn't what's accepted in court as admissible. What's admissible is the data and the administration and care of the data on the system (due care and due diligence). When an administrator following a procedure to remove the data in a field across the entire database documents process (checksums on the data integrity and a pre-production evaluation with benchmarks and documented testing), the data is going to be just as admissible as data that was not purged by field. Back to the database from a credit card company.
More, when data is extracted from the system and the export for evidence is run (to support best evidence), who's to say that the evidence is going to be accepted if there were no checks run on the data before and after it was prepared for introduction in court. Regardless of admissibility and evidence integrity is not the point. But thanks for responding.
I'm not an expert on the admissibility of evidence and I imagine the requirements vary at least subtly between jurisdictions, you seem far more conversant with it than I am. I also take the point that evidence and immutability may not be an issue in your case, but that isn't the case for everyone and HP may well have chosen the more restrictive option to be safe for those that do. I would make the following points though:
- I have worked in enironments where any modification of the collected log data as evidence from its original form (even normalization) makes it inadmissable.
- If I had to stand up and defend the reputability of the data stored in Logger I would find it far simpler to argue that it can't be deliberately or accidentally modified by an administrator of the system than that it can be if an arbitrary process has been followed and the steps documented (who's to say it has been documented in every case)
- My feeling (not an expert) is that, at some point at least, an immutable history of the steps taken to gather, store and preocess the evidence would be required. Were ArcSight to audit your modifications to the data where is it to store this audit trail?
- If not for evidetiary purposes, immutability is still important in ensuring that data is not tampered with by a third party attempting to cover their tracks, while other controls may be in place to mitigate this, it is an extra layer of assurance.
It's possible that support has a method for deleting a single field from event data but my understanding of the way the data is structured is that this would be a complicated undertaking (no harm i asking).
This is an interesting problem to think about though and your answer did make me re-assess this a bit 🙂
Concur. Thanks for the response and details. In this instance protective measures are in place to secure the data (highly restricted zone) and maintain integrity as well as the processes to validate the information in the system before and after the process runs. As an aside, I did find a way that should work but I won't share it for the same reasons you outlined because, even if I do/did put controls in place to do pre and post there's no telling who might abuse it.
Now back to the real world.
Thanks again for the response.