Some Important Connector Settings

Below are some smartconnector settings that you may find useful in your environment, especially in a high EPS setup.

1. init and max java heap memory of smartconnectors

Default setting on smartconnectors is 256 MB.

On high EPS setups, 256 MB of heap memory may not be sufficient for your smartconnectors. So, increasing the heap memory seems to be an obvious choice.

However, from our experience, if you allocate too much heap memory and the EPS is not sufficiently high to use that memory, your connectors' performance may actually degrade. Also, in most setups, EPS on weekends is much lower than on week days. So you need to choose a setting that can accommodate both high EPS requirements of week days and relatively low EPS requirements of week ends.

From our experience, keeping both init and max values of java heap memory to 512 MB gives optimal performance. If you find that your connectors are still not able to process all the incoming EPS, you can change both the settings to 1024 MB. Generally, it is a good idea to keep both init and max settings to the same value.

2. CPU Multithreading

These settings affect syslog daemon connectors more than any other connector type.

By default, smart connectors do not use multithreading.

In our setups, we have found that without multithreading, syslog daemon connector performance degrades in case of high incoming EPS.

Configure below settings to enable multithreading for your connectors

syslog.parser.multithreading.enabled=true

syslog.parser.threadcount=4

The first setting enables multithreading on the connector.

The second setting controls the number of threads that the connector should use. If you do not configure this setting, the connector will use all threads on the system. This may not leave CPU resources for other connectors on the system. So, it is a good idea to limit the threads if you are running multiple connectors on a system.

3. Logger or ESM transport multithreading

By default smartconnectors use a single transport thread to send logs to ESM or logger destinations. If your connectors send high EPS to logger or ESM, and you observe caching on your connectors because the logs cannot be sent to the destination, you should increase transport thread count for your connectors.

For ESM destination

http.transport.multithreaded=true

http.transport.threadcount=2

For Logger destination

transport.loggersecure.multithreaded=true

transport.loggersecure.threads=2