Trusted Contributor.. Trusted Contributor..
Trusted Contributor..

Enable raw logs in ESM

Dear All,

What would be the effect on ESM if we enable raw logs

Thanks and Regards, Sandeep

1 Reply
Micro Focus Expert
Micro Focus Expert

Hello Sandeep,

I am assuming that you refer to the enablement of "raw event logs" from SmartConnectors registered to ESM.

Since the addition of raw data to the event payload will typically increase the size of the data being sent and persisted to ESM, you may see some decrease in overall ingestion performance.

In the log files, you may start to see messages which indicate that incoming event batches cannot be fitted into single data chunks.  This is not a critical problem, since ESM will compensate and split the events accordingly, however there is a performance penalty for this.  Some indicate that this can be mitigated by altering the event batch sizes, but this is not something that should be taken lightly.  It is much better to switch off raw event logs after troubleshooting than alter ESM to cope with leaving them on, as those changes in ESM might improve ingestion on one-hand, but decrease read performance on the other.

As with all software solutions, performance behaviour varies across systems.  For ESM this will depend on the hardware and the content within ESM that has been enabled.   An ESM system that has been configured with some room to grow may not show a particularly noticeable change of performance,  whereas one which is only coping, could react more drastically .

The general recommendation would be to use raw events for troubleshooting by all means, but enable this feature only for as long as is necessary to collect the troubleshooting data.   Please switch off raw events from connectors when the troubleshooting session is complete.

I hope that this helps,

Best regards,


ArcSight Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.