shell job failure

Hi All,

Host connection by shell job is failing with below error message.

2024-08-06 02:35:35,694  ERROR  [Request processor timer][CustomerId:1] (:) - Interrupting thread Process Results Thread1-Host Resources by Shell_CDN - CPU due to timeout of 120000 msec 
for root operation: com.hp.ucmdb.reconciliation.datain.operation.DataInAddOrUpdateData : [ID=1256956905] [Customer ID=1] [Changer=UCMDBDiscovery: Host Resources by Shell_CDN - CPU
Probe: WCHPMONIPRDW15] [Datas to update- 532] [Referenced Data- 0] addOrUpdate : [Objects For Update- node(14) file_system(62) process(1) file_system_export(40)
ip_service_endpoint(1) cpu(2) osuser(119) ip_address(13)] [Links For Update- containment(13) client_server(1) composition(225) usage(1) realization(40)] [Referenced Objects-] [Referenced Links-]

Regards,

Sridhara.

  • Suggested Answer

    0  

    Hello Sridhara,

    I saw this issue a lot of times. Most of the times it's about long running threads while trying to reconcile the result vector from Host Resources and Applications by shell.

    For this there are a few leads to work on. Most of the times the problem lies with the weak nodes from the result vector.

    \Bogdan

  • 0 in reply to   

    Hi Bogdan,

    Thanks for the information. Can you please let me know how we can identify weak nodes.

    Regards,

    Sridhara.

  • Verified Answer

    +2   in reply to 

    Weak nodes are nodes that have no information except an attached IP Address.  You'd need to look into the identification queries (most likely).  This timeout  will have a series of numbers that indicate times that was taken during reconciliation.  For example:

    2024-08-07 11:12:37,697 INFO [qtp2045224304-784][CustomerId:1] - [ID=1718230469] [Customer ID=1] [Changer=UCMDB: User:{UISysadmin###UCMDB},LoggedInUser:{sysadmin###UCMDB}] [total= 0.525 , manipulation= 0.019 , identification= 0.271 , identification.query= 0.202 , identification.match= 0.016 , merge= 0.097 , update= 0.040 , merge.loa= 0.010 , datain.lock= 0.001 , datain.lock.check= 0.001 , update.model.topology= 0.003 , update.model.post_validation= 0.020 , update.model.db= 0.000 ] [durations: identify= 0.282 dataIn= 0.110] [Ignore From Cmdb- 0 Ignore From Bulk- 0 Merge Operations- 0 Merged CIs- 0 Type Changes- 0 Max Topology Level- 1]

    In the above, identification took 0.271 seconds (pretty fast).  Post what your identifications were.  It could be weak nodes, or it could be missing indexes in the database.  You may need to work with your DBA to determine which queries are taking the longest, but the log message in the cmdb.reconciliation.audit.log will be invaluable in detecting this issue.

    Keith

    -- Hope this helps!

    Keith Paschal

    UCMDB Worldwide Support Lead