What are recommended on specifications of connector server for enterprise?
I want to know if there are any recommended specifications for connector servers (Collector Servers: Where logs are being collected by the connectors and processed before being sent to ESM).
What should be the key metrics to gauge the performance of connector servers for future capacity planning will be helpful?
How many devices should be sending logs to one connector server and should there be any limitation on the size of logs being sent? In my opinion recommended devices number is 10. However not sure about the log size.
Can you please help? Any guide that I should read for capacity bench marking / planning?
I'm just going to tell you my current Connectors performance and event flow maybe that can help you? My busiest Connector Appliance has average 1400 events per second coming in. Average size is 650,000 B/s. At this flow the connector is using about 10% cpu 17% memory this is an ArcMC C6500. As for devices sending logs its over 50. Hope this helps
Thank you Charles, that is definitely a help. However I am currently responsible for expanding the SIEM infrastructure for my company and I was looking forward to do the due diligence before recommending any thing.
Any guide from planning perspective for scalable, highly available and amazing performance of SIEM infrastructure would be much appreciated.
The informations are very Helpfull.
But kindly i need to know something , How did you get all this load values ?
I need to have the same with All my connectors.
It is actually very dependant ona few things.
- What is the EPS.
- What type of logs (windows, syslog, rest etc).
- How many connectors do you have on each server.
- Are you doing heavy batches or aggregation?
A few pointers would be, to try to stay with maximum 3000 EPS for syslog, and 2000 EPS for windows logs. Connectors should never have more than 2GB memory heap space allocated to prevent long GC times. And 1 CPU core for smaller connectors and 2-6 for much larger ones.
And last but not least, maximum connectors per server should normally not go over 8, i personally prefer max 4.
With that you can calculate how many cores and how much memory is needed, with an average of 2 cores + 1 GB memory per connector and a few GB for the OS.
4 Connectors on one server,
10-50GB cache size per connector to prevent logs from dropping, so some storage is handy.
But again, these have to be tweaked to your needs. If you have a large amount of connectors with low EPS, they would need less, while allowing to have more than 4 on one server.
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.