Absent Member.. JasonCantrell Absent Member..
Absent Member..
80 views

[OO Tip] Central Terracotta Database Corrupt and Service Unable to Start

Issue:

An error similar to the following appears in the %ICONCLUDE_HOME%/Clustering/terracotta/terracotta-data/server-logs/terracotta-server.log file:

 

2011-01-17 09:43:26,210 [WrapperStartStopAppMain] ERROR com.tc.server.TCServerMain - Thread:Thread[WrapperStartStopAppMain,5,main] got an uncaught exception. calling CallbackOnExitDefaultHandlers.

com.tc.objectserver.persistence.sleepycat.TCDatabaseException: Environment invalid because of previous exception: com.sleepycat.je.log.DbChecksumException: (JE 3.3.74) Location 0x12/0x7f4423 expected 1755551836 got 421290810

 

OR

 

2011-01-17 09:43:26,240 [WrapperStartStopAppMain] ERROR com.tc.logging.ThreadDumpHandler - Mon Jan 17 09:43:26 CST 2011

 

In addition please see the KCS article for further WARNINGS which might also follow this issue.

 

Solution:

This error is thrown when the Terracotta database becomes corrupted. The solution is to remove the corrupted Terracotta database, and allow Terracotta to rebuild a clean one.

 

1. Stop all Operation Orchestration related services across the entire cluster:

  • RSGridServer (The OO Clustering Service)
  • RSCentral (The OO Central Service)
  • RSJRAS (The OO Remote Access Service)
  • RSScheduler (The OO Scheduler)
  • RSCluster (The OO Load Balancing Component)

     Note: Include all Centrals, any standalone RAS, and any standalone Cluster components.

 

2. Backup, copy, or rename the current databases, which are found at %ICONCLUDE_HOME%/Clustering/terracotta/terracotta-data on every server running the RSGridServer service.

 

3. Delete every instance of the Terracotta database across the cluster by removing %ICONCLUDE_HOME%/Clustering/terracotta/terracotta-data from every server running the RSGridServer service.

 

4. Restart the OO services on the first node of the cluster, in the preferred order below:

  • RSGridServer (The OO Clustering Service)
  • RSCentral (The OO Central Service)
  • RSJRAS (The OO Remote Access Service)
  • RSScheduler (The OO Scheduler)
  • RSCluster (The OO Load Balancing Component)

5. Once the first node is up and healthy, bring up each additional cluster node in turn.

 

Please review the KCS article as it has a complete log entry for this error:

http://support.openview.hp.com/selfsolve/document/KM1042353

HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Labels (2)
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.