How to get the raw data record for a Sentinel event

How to get the raw data record for a Sentinel event

Getting the raw data record that caused an event in Sentinel is rather cumbersome (Hint: Please vote for enhancement 929916 - add ability to show raw data for an event). To get it, one has to take the RawDataRecordId from the All view of the event in the WebUI, then go to More -> Get raw data -> check if the appropriate Event source hierarchy and Event source is selected -> select the right file based on date and time -> click Download. Once the downloaded archive has been unpacked (twice!), one can finally search for the RawDataRecordId.

The script presented here will automate this process. It can be run on the Sentinel server itself and takes a set of exported events from Sentinel as input. It then searches for the respective raw data record in Sentinel's archive.

Events can be exported from Sentinel using the Log To File action. For our purposes, this action must be configured with the following properties:


  • Display Format: JSON

  • Display Data: All Data

  • Display Events: All Events



configure action - log to file

By default, exported events will be written to /var/opt/novell/sentinel/data/log_to_file_events.txt. This can be changed in the file integrator:

configure integrator - file

Once the action and the integrator have been configured, select the events you're interested in and choose Log To File from the Event Actions accordion. Click Execute:

execute event action

The last line of the log_to_file_events.txt file will contain the selected events from the most recent export in JSON (JavaScript Object Notation) format. To print it on the console use:

novell@sentinel:~/bin> tail -1 /var/opt/novell/sentinel/data/log_to_file_events.txt

For better readability, the output has been pretty printed here using the jq utility. jq is a lightweight and flexible command-line JSON processor. Think of it as sed/awk for JSON.


[
{
"CustomerId": "1",
"RefId01": "0",
"agent": "SUSE Linux Enterprise Server",
"det": "1433348016000",
"dip": "10.252.0.0",
"dp": "ssh2",
"dt": "1433348016948",
"dun": "unknownuser",
"ei": "{\"Method\":\"password\"}",
"estz": "Europe/Berlin",
"estzhour": "18",
"estzmin": "13",
"estzmonth": "5",
"evt": "sshd: Authentication denied",
"msg": "Failed password for invalid user unknownuser from ::1 port 48156 ssh2",
"obsclass": "SRV x86",
"obscountry": "DE",
"obscrit": "B",
"obsdep": "SIEM",
"piu": "105201",
"pn": "SUSE Linux Enterprise Server",
"port": "SLES EMEA",
"prot": "ssh2",
"repassetid": "0",
"repip": "127.0.0.0",
"rn": "localhost",
"rv1": "0",
"rv121": "BF000000E4F000000-FB70-1031-AE7B-00215A9B04E0",
"rv122": "6D1CFA60-3235-102C-B08F-0016E694D1D0",
"rv150": "AUTH",
"rv164": "1441152000000",
"rv165": "1496448000000",
"rv171": "3F000000F000000F000000-FB88-1031-8D30-00215A9B04E0",
"rv172": "BF000000E4F000000-FB70-1031-AEFA-00215A9B04E0",
"rv192": "SUSE Linux Enterprise Server",
"rv21": "C76D2820-C395-1029-BB86-001321B5C0B3",
"rv22": "3F000000F000000F000000-FB88-1031-A5A5-00215A9B04E0",
"rv23": "3F000000F000000F000000-FB88-1031-A5A9-00215A9B04E0",
"rv24": "3F000000F000000F000000-FB88-1031-A6A1-00215A9B04E0",
"rv25": "53D2BA31-EC2F-1032-A7AF-00215A9B07CA",
"rv30": "DE",
"rv32": "OS",
"rv39": "default",
"rv77": "0",
"rv81": "SRV x86",
"rv82": "SIEM Server - EMEA",
"rv84": "B",
"rv98": "SIEM",
"sev": "0",
"sip": "::1",
"sn": "sentinel",
"sp": "48156",
"spt": "1433348016948",
"src": "3F000000F000000F000000-FB88-1031-A5A5-00215A9B04E0",
"st": "N",
"sun": "unknownuser",
"vul": "0",
"xdasdetail": "0",
"xdasid": "4",
"xdasoutcome": "2",
"xdasoutcomename": "XDAS_OUT_DENIAL",
"xdasprov": "0",
"xdastaxname": "XDAS_AE_AUTHENTICATE_ACCOUNT"
}
]


Sentinel stores raw data in gzipped text files. One file per event source and hour. The directory structure is like:

[SECONDARY_STORAGE]/[some UUID]/rawdata_archive/[EventSourceID]/[YYYY]-[MM]/[DD]-[HH]00.gz

With jq we can easily extract the EventSourceID (rv24) and the timestamp (spt) from the event and construct the file's full path. Then we extract the RawDataRecordId (rv25) and grep the file for it. Finally the script prints the raw data record and the pretty printed event to stdout.

To run the script, use:

novell@sentinel:~> tail -1 /var/opt/novell/sentinel/data/log_to_file_events.txt |./bin/get-raw-data-for-event.sh

The result will be (plus the pretty printed parsed event shown above):

{"s_AppId":"sshd", "i_syslog_priority":"38", "CONNECTION_METHOD":"SYSLOG", "i_Hour":"18", "i_RXBufferLength":"108", "CONNECTION_MODE":"map", "s_Process":null,"s_RV25":"53D2BA31-EC2F-1032-A7AF-00215A9B07CA", "s_RV24":"3F5F33F2-FB88-1031-A6A1-00215A9B04E0", "i_Type":"2", "i_Second":"36", "s_RV23":"3F5F33F2-FB88-1031-A5A9-00215A9B04E0", "s_RV22":"3F5F33F2-FB88-1031-A5A5-00215A9B04E0", "s_Version":"2011.1r5-201502050555-preview", "s_RV21":"C76D2820-C395-1029-BB86-001321B5C0B3", "s_Body":"sshd[21777]: Failed password for invalid user unknownuser from ::1 port 48156 ssh2", "s_chainId":"1433228416625", "i_milliseconds":"1433348016000", "s_raw_message2":"Jun 3 18:13:36 sentinel sshd[21777]: Failed password for invalid user unknownuser from ::1 port 48156 ssh2", "s_MessageOriginatorPort":"58710", "i_Minute":"13", "s_Date":"Jun 03 18:13:36", "i_TrustDeviceTime":"", "i_DayOfMonth":"3", "s_chainSequence":"10398", "i_Year":"2015", "s_sha256Hash":"3f284178f8a811950ca93df83d006c7d4e1db249b3a7ac0c39d5a581deb18784", "s_SyslogRelayIp":"127.0.0.1", "s_MessageOriginatorHost":"sentinel", "s_Pid":null,"i_Month":"5", "i_syslog_facility":"4", "i_syslog_severity":"6", "EventSourceManagerID":"C76D2820-C395-1029-BB86-001321B5C0B3", "CollectorID":"3F5F33F2-FB88-1031-A5A5-00215A9B04E0", "EventSourceGroupID":"3F5F33F2-FB88-1031-A5A9-00215A9B04E0", "EventSourceID":"3F5F33F2-FB88-1031-A6A1-00215A9B04E0", "EventRecordID":"53D2BA31-EC2F-1032-A7AF-00215A9B07CA", "ChainID":"1433228416625", "ChainSequence":"10398", "EventDate":"06/03/2015 18:13:36.942 +0200"}


You may need to adapt two variables in the script to your environment:


  • SECONDARY_STORAGE: secondary storage location as configured under Storage -> Event

  • JQ: full path of the jq binary



DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Comments
I don't have access to Bugzilla, but maybe adding it to MF Idea Portal will do better job.

David

You can vote for "View single raw data event in the WebUI": https://www1.v1ideas.com/MFI/sentinel/Idea/Detail/12272

Top Contributors
Version history
Revision #:
1 of 1
Last update:
‎2015-06-04 00:34
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.