Advanced Logger Forwarder Tuning.
In this discussion, I wanted to point out about a feature in the HPE ArcSight Loggers.
This feature is the capability of forwarding events from within Loggers to external destination/host.
This is done through FORWARDERS.
For those who are not familiar, Below is a small introduction taken from Admin Guide of Logger Version 6.3
Forwarders send all events, or events that match a particular filter, on to a particular host or destination such as ArcSight Manager. The ability to define a different filter for each forwarder allows Logger to divide traffic among several destinations. For example, because Logger can handle much higher event rates than ArcSight Manager, Logger might be used to forward events to a number of ArcSight Managers. Forwarder filters make it possible to split the flow between the Managers, using one forwarder for each Manager. Additionally, forwarding enables you to send a subset of events to other destinations for further processing while maintaining all events on Logger for long-term storage.
In many of the ArcSight deployments, logger is used to forward events to ESM Manager.
As an example, a typical configuration would be that Logger incoming EPS is within 2K to 3K range then all events are sent to Manager which makes the outgoing EPS also the same.
In this kind of configuration is better to have some best practices in mind as this will help you save lot of time and resources.
Best Practices - Summary
Below is from Admin Guide of Logger Version 6.3
As a best practice, do not add more than ten regular expression forwarders. Even though each additional forwarder improves the forwarding rate, the relation is not proportional. In high EPS (events per second) situations or situations where other resource-intensive features are running in parallel (alerts, reports, and several search operations) and the forwarding filter is complex, adding too many forwarders may reduce performance because forwarders have to compete for the same Logger resources besides competing for the same built-in connector for forwarding.
You can specify a regular expression or an indexed search query (Unified Query) for the filter. Doing so enables you to take advantage of the indexing technology to quickly and efficiently search for events to forward.
We recommend no more than 3-4 forwarders max. If the customer can use a single forwarder to forward events to a destination, we recommend using a single forwarder. Peak output is on average 3000 EPS total for Logger, NOT 300 EPS for each forwarder. Use one forwarder and apply a filter-out filter on the connector resource in ESM to exclude data that you do not want to forward. Avoid multiple forwarders at all costs Do not use basic aggregation for Logger’s built-in SmartConnector because it is resource intensive. (Basic aggregation is set using the Enable Aggregation (in seconds) field from the ArcSight Console.) The Logger should be able to forward events to a maximum of 1-2 ESM destinations using 5-10 forwarders for the average customer's environment (EPS In around 2-5K). If the forwarders use more complex filters and the "EPS In" is a 5-10K or more we can start seeing the Logger server's JVM running out of memory, in which case it is normally recommended to reduce the number of forwarders. Regardless, we recommend reducing the amount of forwarders as much as possible. Please consider having a fewer number of forwarders when forwarding events to the same ESM destination. Instead you can logically separate the events into several active channels once they arrive to the ESM (e.g from one forwarder). In other words usually there is no need to create several forwarders to send events to the same destination.
I have attached a document which is from 2014, but have some good best practices, recommendations and configurations.
Alternatively, we can also consider to forward from the Smart Connectors directly. Smart Connectors support sending to multiple destinations in real time. That is sending to both Loggers and ESM simultaneously.
Hope this helped.