Event and Raw Data Migration in Sentinel

Event and Raw Data Migration in Sentinel

This article explains the procedure to migrate the Sentinel event and raw data with less down time.

Problem and limitations:


 
Limitation with the current utilities available in Sentinel server:

"backup_util.sh"

  • "backup_util.sh" under bin directory has a limitation that a user can only take a backup and restore on the same Sentinel version.

  • The process of backup and restore with this script will also take longer than the approach we are going to use.


"Slink Integrator and Slink Action"

  • User can perform a search in Sentinel UI and can perform event action to forward the events to another Sentinel server using Slink Integrator and action.

  • This approach has performance limitations. Forward can happen with a specific batch size of events, search results by default 50k events and can forward only that in one shot.



Solution:


 
Here is the really cool solution to address the migration, the event and raw data problem which we have described above:

Sentinel Event and Raw data Migration:



Event Data




  • Sentinel stores the primary storage partitions in the /var/opt/novell/sentinel/data/eventdata directory, which is on the local file system.

  • Copy all the partitions from the source location to the Target server in the same location. (Make sure you have Novell permission set)

  • Login to Target WeBUI-->Storage-->Events


    • Under “Data Restoration”, click on “Find Data” and restore all the partitions.

      1



  • Refresh the UI to load the restored partitions under WeBUI-->Storage-->Events “Restored Data” and Select all the partitions and Click “Apply”.

    2



Raw Data



Slink connector does not store the raw data during this process of migration. Hence, we didn’t recommend the Slink approach to migrate.



In other words, if you forward the events from the source server using Slink Integrator, in the target server you will not see where the respective raw data file is being stored.


In-order to migrate the raw data. Here is the approach.



  1. Sentinel stores the primary storage rawdate files in /var/opt/novell/sentinel/data/rawdata/online directory, which is on the local file system.

  2. Copy all the sub directories under rawdata/online from source setup to the same location of target setup. (Make sure all the directories have Novell permission)

  3. Use the backup utility script to take the configuration data backup and restore in target setup. (You may require to choose other options in backup utility script to take other configuration data backup.

    Ex, Alerts, SI, Netflow with option –i).
    Backup in source setup:

    ./backup_util.sh -c -m backup -f /home/novell/config.tar.gz

    Restore in target setup:
    ./backup_util.sh -m restore -f /home/novell/config.tar.gz




DISCLAIMER:

Some content on Community Tips & Information pages is not officially supported by Micro Focus. Please refer to our Terms of Use for more detail.
Comments
hi,
I´m trying to do this backing up some days from a 7.1.1.2_1179 installation and importing them in a 8.1.0.1_4000 version.
the search data shows me no data (permissions are ok).
in the server_wrapper log I get this:
2017/08/03 19:16:55 | INFO | jvm 1 | Thu Aug 03 19:16:54 ART 2017|WARNING|qtp314441056-3649235|esecurity.ccs.comp.event.indexedlog.IndexedLogPartitionManager.getRestorablePartitionsFromLocal
2017/08/03 19:16:55 | INFO | jvm 1 | Directory name is not in a valid partition format: 20170731_FDEF7089-7621-1034-89AE-3440B5E05D82. This directory will be skipped.
2017/08/03 19:16:55 | INFO | jvm 1 | Thu Aug 03 19:16:54 ART 2017|WARNING|qtp314441056-3649235|esecurity.ccs.comp.event.indexedlog.IndexedLogPartitionManager.getRestorablePartitionsFromLocal
2017/08/03 19:16:55 | INFO | jvm 1 | Directory name is not in a valid partition format: 20170731_FDEF7089-7621-1034-8A03-3440B5E05D82. This directory will be skipped.

Any Ideas? Thanks!
I tried recreating the same setup and could see the eventdata partition has been restored and could search the events successfully in 8.1 sentinel server. These are few things worth checking it.
a. Make sure you have set the permission for the sub directories.
chown -R novell:novell
b. Refresh the UI to load the restored partitions under WeBUI–>Storage–>Events “Restored Data” and Select all the partitions and Click “Apply”.
c. Possible that particular date partition could have corrupted if you does not have any problem in restoring and searching other partitions events.
Top Contributors
Version history
Revision #:
1 of 1
Last update:
‎2017-07-17 16:15
Updated by:
 
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.