srivatsa123 Absent Member.
Absent Member.
395 views

Agent queuing heavily

Dear All,

One of my syslog servers is Queuing logs heavily. I increased the Java heap memory size but still the same. A connector which is receiving about 1500 EPS is forwarding about 20 EPS to the ESM.

Also, what is the maximum EPS a connector can handle (v 6.0.4) and is there a treshold for the maximum Java Heap memory size that can be set for a connector?

Any help in this regard will be highly appreciated.

Thanks and Regards,

Srivatsa G.V.

Labels (2)
0 Likes
Reply
6 Replies
katzmandu1 Absent Member.
Absent Member.

Re: Agent queuing heavily

The maximum for a syslog connector is around 2k eps.

That said there are other tricks for boosting performance.

I wrote a blog post on this a while back. This stuff should all/still hold true.

Tuning Your Syslog Daemon | MetaNet IVS

0 Likes
Reply
paulpiscuc Absent Member.
Absent Member.

Re: Agent queuing heavily

Hi Jonathan. The article is great, but can you post to which value of EPS did you manage to reach using the tuned syslog deamon? Also, you mentioned ArcSight Syslog file used with a standard syslog deamon. What EPS rate did you manage to reach, and why do you think this setup is better?

Thanks,

Paul

0 Likes
Reply
katzmandu1 Absent Member.
Absent Member.

Re: Agent queuing heavily

For syslog daemon, with it being heavily tuned, we were able to sustain around 3k eps without too many issues. However, I would suggest changing your architecture to use syslog file if your eps is approaching 2.5k or greater.

With syslog file I've seen around 5k eps but I haven't really had a chance to stress-test it and find a true maximum.

0 Likes
Reply
paulpiscuc Absent Member.
Absent Member.

Re: Agent queuing heavily

Hi Johnathan. Thanks for the info. I was thinking of around 5k sustained, 7k peaks, filtered data. From my understanding, the best would be to go with some sort of load balancing, although I haven't tried it yet.

0 Likes
Reply
callawayone Absent Member.
Absent Member.

Re: Agent queuing heavily

Hello Jonathan, Did you consider installing multiple syslog SmartConnectors on the same device to "load balance" the events received. You don't have to have just one Syslog SmartConnector listening on port 514. You can have them on 515, 516, any unused port.

0 Likes
Reply
katzmandu1 Absent Member.
Absent Member.

Re: Agent queuing heavily

There have been instnaces where we've done this, but sometimes in customer environments we don't have the ability to have the network team change destination ports or hostnames. In one case there are 2500 Cisco devices reporting in and they all are configured to send to the same syslog server. It's too difficult to only configure a subset to send to a specific connector. Other times we have the ability to divvy-up by device, so Cisco systems will go to one syslogd, CEF-formatted messages from other devices (Palo Alto, Powerbroker, etc) will go to another syslogd, and OS syslog setups will go to a third.

0 Likes
Reply
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.