ArcSight ESM and Express Event Archive
I have been searching documentation for some detailed explanation about the "logger.archive.space.allocated-in-gb" option in the /opt/arcsight/logger/current/arcsight/logger/user/logger/logger.properties ArcSight ESM/Express file but I was not able to find information about the way it works.
I am considering the following scenario:
-> changing the default archive location from /opt/arcsight/logger/data/archives/ to for example /opt/arcsight/logger/data/archives2/ (I am going to mount an external storage to that new folder in order to extend the total archive capacity) - already done that with an ArcSight ESM, it works
-> now I need to change the logger.archive.space.allocated-in-gb option in order to match the new available space for archiving
My questions for anyone who might have tried this before or might have experience with this are:
1. When setting the new logger.archive.space.allocated-in-gb option, do I have to take into consideration both the space used up to now in the previous archive location (/opt/arcsight/logger/data/archives/) and the new available space as well (/opt/arcsight/logger/data/archives2/) - so to add the two - or I can set it for the exact amount of space available in the new location? What is the default behavior for evaluating this figure, does it sum up all current and previous archive locations that still have saved archives in them, or only the current used location for archiving?
2. The default e-mail notification which is sent when the long term storage event archive is filled up and archiving fails is sent based on the physical storage being exhausted or on reaching the limit set by the logger.archive.space.allocated-in-gb (since I might have 1 TB available space and set the option at 500 GB, and also the other way around as well even though if it would not make too much sense)?
3. Is there any documentation available that would describe in detail how this option is to be set and the expected behavior?
4. Considering attaching external storage, is there any limitation regarding the total space that can be used by ArcSight Express for offline event archiving?
I am testing different scenarios at the moment, but it is going slowly since I have to wait for the daily archiving job to run.
I could not find any documentation that would completely describe all of my questions above. However, I can share with you the following results of my testing:
-> The logger.archive.space.allocated-in-gb option is available on ESM and it works; as long as you provide enough archiving space to the archive folder (eventually by mounting a new external storage device) you can change the property to match your actual physical space. I would guess in calculating the limit it sums up all current archiving folders (if you have different for different Storage Groups), but I am not 100% positive on this one. As for the notification e-mail, it is sent both when you are out of physical space and/or when you reach the limit imposed by the logger.archive.space.allocated-in-gb property.
-> The property is not available on Express appliance - as I found out the property is not in the logger.properties file. I also found out from documentation that for Express the maximum hard-coded offline archive limit is of 200 GB. So for this one the only option would be to do archiving/backup of Express archives using native Linux utilities (such as a cron job for example).
If anybody knows more on the subject, feel free to share.
Yes this works and I have tested this for Express.
However, Support and PreSales are now saying this is "not supported" despite statements to the contrary previously.
Here are the steps I put together to do this but use at your own risk as it could void your support for Express:
The default folder size is 200 GB and the default folder location is /opt/arcsight/logger/data/archives
To reconfigure the archive folder size and location, follow the following steps:
- Log on to the Express appliance as the arcsight OS user
- Verify the space being used by the current archive directory
du -hs /opt/arcsight/logger/data/archives
- Stop all ESM services by running the following command:
/etc/init.d/arcsight_services stop all
- Backup /opt/arcsight/logger/current/arcsight/logger/user/logger/logger.properties
cp /opt/arcsight/logger/current/arcsight/logger/user/logger/logger.properties /opt/arcsight/logger/current/arcsight/logger/user/logger/logger.properties.bak
- Add the following line(s) at the end of file (change <archive storage size> to an integer number and <folder location> to the desired folder to be used to store the archives):
- Save the file
- Verify the changes
cat /opt/arcsight/logger/current/arcsight/logger/user/logger/logger.properties | grep logger.archive
- Restart all ArcSight services
/etc/init.d/arcsight_services start all
- Once the Manager starts up again, validate the setup.
After adding these two line in the logger.properties and restarting all the services, I still get the same capacity when I run df -h.
Thanks for your assistance.
Yes, which I would presume is totally wrong . I think when it was set the person who did it probably thought about Bytes, not noticing it is already in GB. Even so, it still is 2TB I guess which is a lot of space for local archive if it is not some external mapped storage.
All the best,