Recovering Sentinel Partitioned Event Data from Raw Data

Recovering Sentinel Partitioned Event Data from Raw Data

We encountered a very rare event where the Event Partition Data (index of raw event data) failed to be written to the secondary storage. While the compressed raw data was still intact and written to the secondary storage, it could not be queried as the index of its content was missing.

The resolution required several steps:

  1. Query the SIEM database for raw files of concern:
    SELECT
      file_name
    FROM
      raw_data_files_info
    WHERE
      (
        date_created >= '2015-01-01 10:00:00+10' AND
        date_created <= '2015-01-05 10:00:00+10'
      )
    ORDER BY
      date_created;
  2. Log into the Sentinel Console and Verify Integrity for the compressed raw files under Storage -> Raw Data
  3. Create recovery folder structure for the date in question:
    novell@xxxx:~> mkdir /mnt/VFILER/recovery
    novell@xxxx:~> cd /mnt/VFILER/(Sentinel UUID)/rawdata_archive
    novell@xxxx:/mnt/VFILER/(Sentinel UUID)/rawdata_archive> find ./ -type d -iname 2015-01 -exec mkdir -p '/mnt/VFILER/recovery/{}' \;
  4. Use output of Database Query to copy compressed files:
    novell@xxxx:~> cp '/mnt/VFILER/(Sentinel UUID)/rawdata_archive/{DB-ROW}' '/mnt/VFILER/recovery/{DB-ROW}'
  5. Decompress files:
    novell@xxxx:~> find /mnt/VFILER/recovery/ -type f -iname *.gz -exec gunzip '{}' \;
  6. Work out which directories belong to which Event Source (directory ID = Event Source ID).
  7. Create a File Connector under the relative Collector.
  8. Create a Event Source under the File Connector that is set as a Rotating File, points to the directory of the decompressed raw files, enter a RegEx pattern of \d\d\-\d\d\d\d(\-\d)?, and enable the Trust Event Source Time.

    File_Event_Source_Directory.png

    File_Event_Source_General.png

  9. Start the File Event Source and wait for it to process the directory. This can be validated by opening the Raw Data Tap on the Event Source.

NOTE 1: Keep an eye on the Server Cache directory. As the File connector will run quickly over a lot of files and records, it can back up the cache a bit but you don't want this getting out of control - this will depend on your system hardware.

NOTE 2: Only run one File Event Source at a time as this process can be memory intensive and too many File Connectors processing lots of events can cause Memory Core's on the Sentinel server - again depends on your hardware.

NOTE 3: We also found a few correlations were triggered while this process ran. There was no consistency to this, but the Observer Event Time had the correct value for the date/time of the event.

Labels (1)

DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Top Contributors
Version history
Revision #:
3 of 3
Last update:
‎2019-10-08 22:22
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.