Micro Focus Contributor
Micro Focus Contributor
1054 views

Missing Syslog Header while Forwarding Events

Hi,

I have configured Office365 SmartConnector and forwarding the events to CEF Syslog. I have also configured Forwarding events=true. But after all this, I could not see Source Hostname/IP Address reflected in the CEF logs.

As per Syslog RFC, the header must contain timestamp and indication of hostname or IP address of the device. But we couldn't see any of that in the CEF header.

"CEF:0|Microsoft|Azure Active Directory||UserLoggedIn|UserLoggedIn|Unknown| eventId=7049 externalId=535882b8-5282-4b3f-96d3-b316505e9388 msg=UserLoggedIn art=1513679311609 act=UserLoggedIn rt=1513587916000 outcome=Succeeded shost=ecoprobe-dmz.gns.novell.com src=192.31.114.252 sourceZoneURI=\/All Zones\/ArcSight System\/Public Address Space Zones\/ARIN\/192.0.3.0-192.88.98.255 (ARIN) suid=user@user.onmicrosoft.com cs1=a2074eb0-1bbf-4dab-b189-d7b1932097fd cs4=10033FFFA292A076@langleytest.onmicrosoft.com cn3=0 cs1Label=Organization ID cs4Label=User Key cn3Label=User Type ahost=hackmee.atlas.com agt=164.99.175.33 agentZoneURI=\/All Zones\/ArcSight System\/Public Address Space Zones\/ARIN\/164.0.0.0-169.253.255.255 (ARIN) amac=00-50-56-A5-27-F3 av=7.6.0.8009.0 atz=Asia\/Calcutta at=office365 dtz=Asia\/Calcutta _cefVer=0.1 aid=3sS1aWF8BABCAAluGvyhJ1A\\=\\="

Also, we could see for many of the Sharepoint/Exchange logs, the Vendor name is blank. For example,

"CEF:0|Microsoft|||FileModified|FileModified|Unknown| eventId=6978 externalId=0f51a826-eab7-4287-7201-08d546bd9414 msg=FileModified art=1513673707616 cat=SharePointFileOperation act=FileModified rt=1513673438000 src=23.97.54.239 sourceZoneURI=\/All Zones\/ArcSight System\/Dark Address Space Zones\/23.0.0.0-23.255.255.255 (IANA) suid=user@user.onmicrosoft.com filePath=https:\/\/langleytest-my.sharepoint.com\/personal\/user_user_onmicrosoft_com\/Documents\/Atlas2_Linux.docx fileType=File oldFileName=Atlas2_Linux.docx oldFilePath=Documents oldFileType=docx request=https:\/\/user@user-my.sharepoint.com\/personal\/user_user_onmicrosoft_com\/ requestClientApplication=MSWAC cs1=a2074eb0-1bbf-4dab-b189-d7b1932097fd cs3=3b985f72-6a6a-4872-8365-1d21aad66b1a cs4=i:0h.f|membership|user@live.com cs5=SharePoint cn3=0 cs1Label=Organization ID cs3Label=Site cs4Label=User Key cs5Label=Event Source cn3Label=User Type ahost=hackmee.atlas.com agt=164.99.175.33 agentZoneURI=\/All Zones\/ArcSight System\/Public Address Space Zones\/ARIN\/164.0.0.0-169.253.255.255 (ARIN) amac=00-50-56-A5-27-F3 av=7.6.0.8009.0 atz=Asia\/Calcutta at=office365 dtz=Asia\/Calcutta _cefVer=0.1 ad.RecordType=6 aid=3sS1aWF8BABCAAluGvyhJ1A\\=\\="

So please let us know if this needs extra configuration to be done to get this values in the log.

Thanks.

Labels (1)
Tags (2)
0 Likes
5 Replies
Marijo Mandic Acclaimed Contributor.
Acclaimed Contributor.

Re: Missing Syslog Header while Forwarding Events

Hello,

1) SmartConnector for Microsoft Office 365:
https://community.softwaregrp.com/t5/ArcSight-Connectors/SmartConnector-for-Microsoft-Office-365/ta-p/1583309

2) From above guide we can see following:
a) ArcSight ESM Field -> ArcSight ESM Field
is populated by following information from RAW (original source event)
b) Device-Specific Field -> OriginatingServer

3) In your event extract I can see following:
shost=ecoprobe-dmz.gns.novell.com

4) ArcSight Common Event Format (CEF) Guide:
https://community.softwaregrp.com/t5/ArcSight-Connectors/ArcSight-Common-Event-Format-CEF-Guide/ta-p/1589306

5) From CEF Guide we can see that CEF Key Name "shost" is "sourceHostName" and this field would be populated if you for example check on ESM using Active Channel.

6) If you have doubt about this information you can enable RAW event on SmartConnector, for example CEF File Destination and then check "RAW Event". It should contain information "OriginatingServer" which is then parsed/mapped to "sourceHostName".

7) So once SmartConnector for Microsoft Office 365 processes information and you added CEF file Destination it will send processes CEF event via Syslog and then the receiving Syslog SmartConnector will parse the event according incoming the CEF event (in this case you have value "shost" in the event and it will be mapped to "sourceHostName".

😎 You should also be on latest SmartConnector framework (7.7.0) and parser update (7.7.3) to avoid any parsing issues that have been resolved.

Regards,

Marijo

0 Likes
Trusted Contributor.. Exitwounds Trusted Contributor..
Trusted Contributor..

Re: Missing Syslog Header while Forwarding Events

Hello akumargarai,

I would agree with marijo; if you have any doubts of the field mappings or event integritiy of your data, best case is to perform a RAW capture. Ensure you turn on "Enable Raw Event" on your SmartConnector.

-----------
The first civilian U.S. Government contractor to utilize ArcSight circa 2000 (http://ow.ly/HdtU30ffUDY). Harris Corporation | Technology to Connect, Inform and Protect. | https://www.harris.com
0 Likes
Karl2 Honored Contributor.
Honored Contributor.

Re: Missing Syslog Header while Forwarding Events

Hi Marijo,

I hope you're fine.

First of all, I'm sorry for retaking an old post, but I have a similar problem to the one described here.

 

I think that maybe you did not understand the issue described: the key is that when forwarding the events from a connector, using a "raw syslog destination", only the CEF message is included(which is fine), however there is no Syslog header added; the problem is that tipically you have a 3rd party syslog server that centralizes these logs and needs a syslog header, I'd say that any Syslog RFC is fine, but at least it should contain something like:

<privall> timestamp device CEF_Log

*device is a deviceHostname or deviceIP.

 

I'd say that would be fine if you can choose if you want to add such header(at least Syslog RFC 3164) when creating this kind of destination, do you know if it is possible?

 

Best regards,

 

Karl.

0 Likes
Marijo Mandic Acclaimed Contributor.
Acclaimed Contributor.

Re: Missing Syslog Header while Forwarding Events


Hello Karl,

thank you for asking, business as usual 🙂

1) I tested following two scenarios:
a) scenario 1
Windows SmartConnector -> CEF Syslog Destination -> Syslog Daemon SmartConnector -> ESM
b) scenario 2
Syslog Daemon SmartConnector -> CEF Syslog Destination -> Syslog Daemon SmartConnector -> ESM

2) Test results for scenario 1:
a) When enabling RAW event on Syslog Daemon SmartConnector and checking the RAW event on ESM Active Channel I can see event without standard Syslog header, so something like this:
CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4672|Special privileges assigned to new logon.|Low| eventId=1196 externalId=4672
b) this is because:
- input event did not have "Syslog Header"
- Windows SmartConnector retrieved event that is not CEF format
- once Windows SmartConnecor parsed event is transformed it to CEF format
- CEF format but without "Syslog Header", but this was not there in the event at the beginning

3) Test results for scenario 2:
a) send CEF Syslog event with "Syslog Header" and in RAW event on ESM Active Channel I could see something like this:
Sep 19 08:26:10 host CEF:0|Security|threatmanager|1.0|100|worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232
b) send CEF Syslog event without "Syslog Header" and in RAW event on ESM Active Channel I could see something like this:
CEF:0|Security|threatmanager|1.0|100|worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232
c) so each time event was consistent, nothing is added to the event

4) Regarding the missing device hostname in the RAW CEF event of forwarded Windows event there is "dvchost" which is deviceHostName. So the information is there and populated in ESM or Logger.

5) Now I understand what are you noting that third party Syslog solution does not "read" CEF event and does not take "dvchost" but expects it to come in "Syslog Header" which it does not. This can be observed in two ways:
a) third party Syslog solution is not compatible with CEF
b) you could file Feature Request to add this "Syslog Header" to the forwarded events (hostname in the event would need to be "dvchost" from CEF event and not the hostname of SmartConnector) to third party Syslog solution (because ArcSight Destinations like Logger/ESM can "map/read" "dvchost" and do not needed it, which makes sense why it was not included initially)

Regards,

Marijo

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Missing Syslog Header while Forwarding Events

Hello @Marijo Mandic and @Karl2

When mentioning "sending with syslog header to a cef destination", we are talking about the setting "transport.cefsyslog.header" right? 

If it is missing on Karl's side it might also be that "deviceHostName" and/or "deviceAddress" is not set, as it controls the header value with it's value in deviceHostname.

If this is missing, then creating a small parser to set this might resolve your issue.

The header configuration ensures that CEF is sent following the standard written down in RFC3164 , so if the third party syslog server does not recognise it, then it shouldn't be any issues with the format itself.

-----------------------------------------------------------------------------------------
All topics and replies made is based on my personal opinion, viewpoint and experience, it does not represent the viewpoints of MicroFocus.
All replies is based on best effort, and can not be taken as official support replies.
//Marius
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.