What is the best collection Method for WEC
I would like to understand and get feedback on the WEC Subscription method of collecting Microsoft logs. Basically, What is the best method when it comes to the collection, is it collector initiated or Source Initiated????
I have actually tried with the source initiated and notice a huge latency issue. I see many log sources shows active in the subscription runtime status, however, logs are not available from these hosts in the forwarded events, how is it possible??? and what could be the issue.
Collector initiated is a lot more resource intensive and requires you to add a user account to the local group "Event Log Readers" on every computer with events to be collected.
Source initiated does not require that and alleviates the resource requirement on the collector.
It is subjective which is best but I'd prefer to use source initiated.
When you say latency do you mean network latency or time between logs getting sent from source > collector. have you configured the "refresh=" option in the subscription url? I think by default its every 15 minutes but you can change it to a value in seconds.
You can also check the log in the source machine to make sure there are no errors?
Check "Applications & Services Logs>Microsoft>Windows>Eventlog-ForwardingPluginOperational" in case you are not sure where it is, it should say something like "The subscription x is created successfully" .
We are using WECs for collecting WKS logs + Sysmon, we have setup more than 20 WECs.
It is working pretty well, we have chosen Source Initiated because there are too many WKS and if you choose Collector Initiated you have, to add manually all endpoints.
Another point is that it is a GPO that manages the connection from the WKS to the WECs.
This permit us also to have a kind of organization which is very useful when you have more than 40K WKS because you have to take into account that the WEC host is not put inside the logs thus it is very difficult by example to identify which host is missing.
With Source Initiated you could by example config all Admin WKS to WEC1 and then with a simple mapping consider that all hosts from that WEC are Admin WKS. and then Sales WKS to WEC2, etc...
This is an extremely useful info when you have to track missing WKS, it is easier than sending everything to any WEC. After, it is your own decision if you don't need it.
One point very important is that the WEC need to be place in the same domain than the endpoints because if it is not the case, you have to use HTTPS which means installing a certificate on each host and you also need a Certification Authority for the revocation of the certificate. It is a nightmare to manage.
A last point, if the number of host is very low like 10, I recommend to do not use a WEC except if you have no choice by a security measure because it will work better, it is what we are using for domain controllers. Direct connection from the WiNC to the DC, we collect the logs at 1 sec after their generation, it is working very well. For me WiNC are the best ArcSight SmartConnector Type they have developed.
If you have question about WECs config and/or WiNCs, do not hesitate to contact me.
Thank you, Kyle And Michael, for looking into it and for your valuable comments.
I have deployed WEC Source Initiated for Servers, however, I do not see logs coming in from all the sources, when I check the Runtime status it says Inactive for most of them and Active for fewer hosts. Although It shows active I did not find logs for many of this active hots. Only a couple of host logs are seen in the Forwarded events. I'm sure all the hosts have logging policy enable and are writing logs and there is no Firewall preventing.
Any idea what could be the issue.
For Workstations, it is possible that the endpoints are down, you have to track this for a long period like 1 day.
If the GPO is properly configured and the subscription too, you should see the logs on the WEC.
If it is not the case, you have to check the subscription config (to be sure there is no mistake) and you have to contact the Operator or Admin who manage the GPO to confirm that the GPO is properly setup.
A good correlation would be to check the AD logs if you can see successful authentications from that host, this will tell you that the endpoints is up and running and will confirm you that there is an issue with either the GPO or the subscription. This is what I have done and most of the time, it was the GPO not setup for those hosts.
It is very difficult for me to help you more for troubleshooting this as I have no access to your infra.
I have created an AL with all endpoints and a lastUpdatedTime that will inform me when I have received the last event from that endpoint and with Query/QueryViewer with AD correlation, I can identify and count which hosts we do not collect events or we stop to collect events. It works pretty well.
I hope this will help you.
Thanks for your detailed reply,
I do receive logs on my WEC Forwarded events but like out of 300 hosts (that my subscription shows) I receive only for 4-6 hosts. When I right-click on my subscription and do runtime status, the result is Total 300 , active 81. Although it shows 81 active I do not see the logs from 81 sources in the Forwarded events pane.
Logs sources have auditing enabled
No firewall issue.
Hope I have explained clearly this time.
what do you think the issue is ?
Firstly you need to identify where is the issue at WEC or after WEC with the WiNC connectors.
Could you please confirm that the WEC is placed in the same domain than the devices sources?
Could you please use PowerShell to check the logs because if your WEC is undersized, what you will see in Event Viewer won't be correct or complete?
Then, could you please identify from the missing hosts something common like the same IP subnet, the same department, etc...
Is-it possible that the WEC is undersized in terms of CPU, RAM and network bandwidth and the sources cannot send everything , there is a bottleneck.
Could you please give me more info about your installation?
- How many WECs?
- WEC Host Setup (CPU, RAM)?
- How many Endpoints per WEC?
- How many EPS (approximately)?
You have to be sure that when you authenticate on the WEC, the user profile should not be loaded because this will impact performance of your WEC.
Could you please answer to those above questions and I will check if we have setup something specifically regarding the latency or other parameters on our WECs?