Highlighted
Absent Member.. Absent Member..
Absent Member..
2783 views

Data flow probe disconnection and slow processing

Jump to solution

Hi,

 

 

working with UD 10.01 on VMware windows 2008 , with data flow probe with 4 core cpu and 4 gb ram.and 60 gb application drive space.

 

 

I sterted discovery and as of now 1600 servers have been discovered.

 

But I observered DFProbe get disconnected several times. Sometimes it starts on its own but maximum time I have to start the service.  Also processing is very slow...

 

Please let me the factors which affects the performance  and how to troubleshoot where it is worng?

 

 

 

I pasted below some log info(wrapperprobegw1) as unable to add attachment due to some issue.

 

 

jvm 24   | #
jvm 24   | # A fatal error has been detected by the Java Runtime Environment:
jvm 24   | #
jvm 24   | #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000f6716561, pid=4852, tid=5996
jvm 24   | #
jvm 24   | # JRE version: 7.0_05-b06
jvm 24   | # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.1-b03 mixed mode windows-amd64 compressed oops)
jvm 24   | # Problematic frame:
jvm 24   | # C  [WMIdll.dll+0x16561]
jvm 24   |   JNI_OnUnLoad+0x8f91
jvm 24   | #
jvm 24   | # Core dump written. Default location: D:\hp\UCMDB\DataFlowProbe\bin\hs_err_pid4852.mdmp
jvm 24   | #
jvm 24   | # An error report file with more information is saved as:
jvm 24   | # D:\hp\UCMDB\DataFlowProbe\bin\hs_err_pid4852.log
jvm 24   | #
jvm 24   | # If you would like to submit a bug report, please visit:
jvm 24   | #   http://bugreport.sun.com/bugreport/crash.jsp
jvm 24   | # The crash happened outside the Java Virtual Machine in native code.
jvm 24   | # See problematic frame for where to report the bug.
jvm 24   | #
wrapper  | JVM exited unexpectedly.

 

 

 please suggest.

 

 

with regards,

Nick

 

 

 

 

 

 

 

 

 

 

0 Likes
1 Solution

Accepted Solutions
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

HI Nick,

This is a known issue described in http://support.openview.hp.com/selfsolve/document/FID/DOCUMENTUM_QCCR1H83741.

 

The fix is currently scheduled to be part of CP12 Update 5 (not released yet).

 

You could wait a bit, or open support case to get a private patch for the issue.

 

 

Regards
-Dmitry Gomel, PMP
Click the Like button at the bottom to say 'Thanks'.

View solution in original post

0 Likes
8 Replies
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

HI Nick,

This is a known issue described in http://support.openview.hp.com/selfsolve/document/FID/DOCUMENTUM_QCCR1H83741.

 

The fix is currently scheduled to be part of CP12 Update 5 (not released yet).

 

You could wait a bit, or open support case to get a private patch for the issue.

 

 

Regards
-Dmitry Gomel, PMP
Click the Like button at the bottom to say 'Thanks'.

View solution in original post

0 Likes
Highlighted
Absent Member.. Absent Member..
Absent Member..

Re: Data flow probe disconnection and slow processing

Jump to solution

Hi Dima,

 

Thanks for the information.

 

 

with regards,

Nick

 

 

0 Likes
Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

Dima,

 

I am experiencing an identical symptom and log entry in the WrapperProbeGW log

I am on CP 12 update 5

The change request in the article you linked is in the release notes for CP 12 update 5 with the description of "Fixed the issue of the probe_manager dummy process remaining after the probe service is finished."

 

I see the same error from the WrapperProbeGW log in the hs_err_pid log:


# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000f8756561, pid=368, tid=3540
#
# JRE version: 7.0_05-b06
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.1-b03 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [WMIdll.dll+0x16561]  JNI_OnUnLoad+0x8f91
#
# Core dump written. Default location: D:\hp\UCMDB\DataFlowProbe\bin\hs_err_pid368.mdmp


The current thread in the hs_err_pid logs is from a discovery job that uses WMI protocol. Is this an issue with WMIdll.dll and WMI discovery jobs.

 

Thank you

Rey Lejano

effectualsystems.com
0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

New problem related to HPCmdDLL was discovered. Defect could be found at HP SSO http://support.openview.hp.com/selfsolve/document/LID/QCCR1H88503.

The fix will be released as part of CP12 Update 8 (that will be released really soon).

For a hot fix please open support incident. You could metioned the thread here.

 

 

 

Regards
-Dmitry Gomel, PMP
Click the Like button at the bottom to say 'Thanks'.
0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

We are happy to announce that Content Pack 12 Update 8 (CP 12.08) is now available for download.

This update contains fixes for critical issues coming from our customers, normalization rules and SAI updates.

 

CP12 Update 8 can be installed on top UCMDB 10.01 CUP2(or higher CUP) and can be downloaded from here:

https://hpln.hp.com/node/11274/contentfiles/?dir=17230

 

New fixed version of HPCmdDll is included.

Regards
-Dmitry Gomel, PMP
Click the Like button at the bottom to say 'Thanks'.
Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

Dima,

 

I want to verify that HPCmdDLL is the same issue that is causing our probe to disconnect. Disconneciton only occurs during jobs that use the WMI protocol. Can you verify that this is a symptom?

 

Thank you,

Rey

Rey Lejano

effectualsystems.com
0 Likes
Highlighted
Acclaimed Contributor.. Acclaimed Contributor..
Acclaimed Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

Dear Rey,

HPCmdDll is only responsible for NTCMD jobs.  Not related to WMI...

Generally speaking, probe heartbeat coming along with all CIs coming from the job. In case system is busy by processing some data, the heartbeat will be waiting in line to be processed as well. UCMDB UI after a timeout will show probe as disconnected.

My suggestion is to check cmdb.reconciliation.audit.log. It prints 2 lines for every data-in operation (every bulk comming). Second line will show total time for operation. Wait for time the problem will be reproduced. Take timestamp, open the log and check if identification of data takes long time.

The problem most probably is there.

Regards
-Dmitry Gomel, PMP
Click the Like button at the bottom to say 'Thanks'.
0 Likes
Highlighted
Honored Contributor.. Honored Contributor..
Honored Contributor..

Re: Data flow probe disconnection and slow processing

Jump to solution

Dima,

 

A few probes disconnected a little after 22:36 and I looked in the cmdb.reconciliation.audit.log and I did not see any  identification taking long. Here's an excerpt of the cmdb.reconciliation.audit.log with a 45 min gap after 20:35
There are no corresponding entries in the cmdb.operation.log and error.log except ERROR - Failed send reply to Collectors in the error.log

 

2013-11-19 22:35:14,624 INFO  -  [ID=1567592029] [Customer ID=1] [Changer=UCMDBDiscovery: ProbeGW_Topology_Task_]                      [Datas to update-   13]
2013-11-19 22:35:14,811 INFO  -  [ID=1567592029] [Customer ID=1] [Changer=UCMDBDiscovery: ProbeGW_Topology_Task_]                      [total time= 0.186 model= 0.049]  [durations: identify= 0.103 dataIn= 0.027]
2013-11-19 23:20:29,555 INFO  -  [ID=1641659210] [Customer ID=1] [Changer=UCMDBDiscovery: ProbeGW_Topology_Task_]                      [Datas to update-   13] [Referenced Data-    0] addOrUpdate    :  [Objects For Update- nt(1) ip_address(1) discoveryprobegateway(2) domain(1) interface(1) discoveryprobemanager(1) ip_subnet(1)] [Links For Update- membership(2) composition(1) management(1) containment(1)] [Referenced Objects-] [Referenced Links-]
2013-11-19 23:20:30,833 INFO  -  [ID=1641659210] [Customer ID=1] [Changer=UCMDBDiscovery: ProbeGW_Topology_Task_]                      [total time= 1.278 model= 0.533]  [durations: identify= 0.561 dataIn= 0.178] [Ignore From Cmdb-  0 Ignore From Bulk-  0 Merge Operations-  0 Merged CIs-    0 Type Changes-    0 Max Topology Level- 1] [Datas to update-   13].

 

Could there be any other issues related to discovery jobs using the WMI protocol. This only happens during Host Apps by WMI and DB Connections by WMI.

 

Thank you

Rey Lejano

effectualsystems.com
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.