ESM, Auditors, Case Base Events, Backups and Logger (oh my!)
We currently use ESM 4.0 with connectors.
Right now, we use ESM as a method for assuring Change Control procedures are followed for SOX purposes. Our connectors feed data the raw SQL tracefiles and other logs through the connectors. At the connector level we filter some data out, and then use ESM rules to corrleate and generate cases that require review. Each case is reviewed against our Change Control database and then closed out. (Side note: Yes, I've seen the presentation about Change Control Monitoiring and Arcsightm, thanks.)
Our SOX auditors come to use a ferw times a year, and ask us to "prove" that we properly reviewed the cases. Normally, we will have the cases online, but the underlying correlation events and base events need to reloaded from old partitions in order to satisfy our auditors that we followed our procedures.
We are looking at moving up to EMS 4.5 SP2, as a new fresh install. During discussion with our account rep, we discussed one of my issues is the database management and backup of ESM. Right now, I'm not confident that we could really get a clean backup if something catostphic happened to the database. Currently, our online database is about 1TB in size, maybe about 1/2 in active use. We retain 30 days or so online, and then partition off the data.
Basically, our rep and the engineer questioned why not simply backup the enviroment only i.e, , not the daily data. I explained that if we were only using arcsight for threat management I could see that as a possbililty, but losing the events making up the case data would be problematic. However, they pointed out that a full restore of that database could take days to reload.
One suggestion was that we could use logger to help with this. CONNECTOR --> LOGGER --> ESM The idea is that all the events would be stored by logger (with a simpler backup process than a full backup of ESM). However, then we seem to lose the correlation with the correlation and cases. So, it was suggested that it should be possible to push out (via the forwarding connector, I presume), case data back into the logger.
At a presentation, we heard that in a post 5.0 ESM release, case events will be retained (or copied) into it's own partition, seperate from the overall event data, which presumabley could also help with this issue. Unfortunately, that is the distant future.
So, I'm looking for input on a few angles:
1) Anyone tried pushing case data to ESM? I think I saw notes somewhere that there are currently difficulties in pushing out correlations with the base events to ESM already. This alone would seem to be problematic for what I wish to do. Any thoughts?
2) I'm not stuck on the idea of pushing the events out to logger --- any other thoughts on exporting out closed cases with the underlying data. It doesn't even need to be Arcsight readable at that point, but I would need the underlying data to exist. Is there any other way to export the case data, other than to Remedy (not even sure that would be sufficient.)
3) Any other questions, food for thought,and suggestions would be greatly appreciated.
Re: ESM, Auditors, Case Base Events, Backups and Logger (oh my!)
What we are doing is taking a Case Package backup once in a week to be on a safer side.
Never thought about the scenario 'Case data backup to Logger'. Please let us know the feasibility if you try the same.