Lieutenant
Lieutenant
1624 views

Internal error: DbaXXXX functions after Update to DP 10.70

Jump to solution

After our update from DP to version 10.70, I keep getting the message Internal error: DbaXXXX functions during the backups.
In the event log of the Windows OS there is also a suitable entry regarding a crash of dbsm.exe.

Faulting application name: dbsm.exe, version: 10.70.210.0, time stamp: 0x5eb1a796
Faulting module name: ucrtbase.DLL, version: 10.0.14393.2990, time stamp: 0x5caeb96f
Exception code: 0xc0000409
Fault offset: 0x000000000006e00e
Faulting process id: 0x2ab0
Faulting application start time: 0x01d660b22724ec98
Faulting application path: E:\HPE_DP_BIN\OmniBack\bin\dbsm.exe
Faulting module path: C:\Windows\SYSTEM32\ucrtbase.DLL
Report Id: 65274ed1-cca5-11ea-80ee-0050568f4073
Faulting package full name:
Faulting package-relative application ID:

DP
Labels (1)
0 Likes
1 Solution

Accepted Solutions
Micro Focus Expert
Micro Focus Expert

Hello @Ravil_M,

Please reboot the CM and report back if this solves the problems. There is a possible problem with memory in 10.70. If the reboot cures the situation (for some time) ask support for OCTCR19Q932210_HF2.

Regards,
Sebastian Koehler

---
Please use the Like button below, if you find this post useful.

View solution in original post

15 Replies
Micro Focus Expert
Micro Focus Expert

Hello @Maddog1972

Try this: 

  1. Rename DebugLogger.librarytype and LVDataMoverComponent.librarytype from C:\ProgramData\OmniBack\Config\client\components\registry
  2. Restart Services
  3. Rerun command "omnidbutil -encrypt_user_pass -debug 1-300  userpass.txt" successful

Regards, 

Andres Fallas Salazar
Customer Support Engineer

If you find that this or any other post resolves your issue, please be sure to mark it as an accepted solution.
If you are satisfied with anyone’s response please remember to give them a LIKE by clicking on the bottom at the left of the post and show your appreciation.
0 Likes
Captain Captain
Captain

Andres,

i have same problem after upgrading to 10.70, but no one file (DebugLogger.librarytype or LVDataMoverComponent.librarytype) in the folder C:\ProgramData\OmniBack\Config\client\components\registry.

What we can do more for solving problem?

 

 

0 Likes
Micro Focus Expert
Micro Focus Expert

Hello @Ravil_M,

Check if the C:\ProgramData\OmniBack\Config\client\components\registry folder contains any *.librarytype files that has no valid reference to DLLs in C:\Program Files\OmniBack\bin\components folder.  Remove any *.librarytype file that has no corresponding *.dll.

Regards,
Sebastian Koehler

---
Please use the Like button below, if you find this post useful.
0 Likes
Captain Captain
Captain

Sebastian, hello.

I have not found any lone files in the folder "D:\ProgramData\OmniBack\Config\client\components\registry"

 

C:\>dir "D:\Program Files\OmniBack\bin\components\*.dll"
Volume in drive D is Новый том
Volume Serial Number is E679-E0FF

Directory of D:\Program Files\OmniBack\bin\components

06/04/2020 06:25 PM 4,785,416 ArrayProvider3PAR.dll
06/04/2020 06:23 PM 3,836,680 CasDiskComponent.dll
06/04/2020 06:26 PM 5,635,760 CDpDataMoverComponent.dll
06/04/2020 06:29 PM 7,821,064 dellemcunityarrayprovider.dll
06/04/2020 06:23 PM 4,311,304 DpBackendService.dll
06/04/2020 06:28 PM 6,803,632 DPHostingEnvironmentComponent.dll
06/04/2020 06:19 PM 3,426,480 DpSessionLogger.dll
06/04/2020 06:26 PM 5,543,088 DpSplitMirrorComponent.dll
06/04/2020 06:29 PM 7,762,096 netapparrayprovider.dll
06/04/2020 06:30 PM 3,534,512 ormapper.dll
06/04/2020 06:19 PM 3,044,616 PocoLogger.dll
06/04/2020 06:18 PM 2,786,568 ScaContainer.dll
06/04/2020 06:15 PM 2,313,648 SerializerBasic.dll
06/04/2020 06:13 PM 2,130,184 SerializerFwsTypes.dll
06/04/2020 06:19 PM 2,811,656 ServiceFactory.dll
06/04/2020 06:29 PM 9,682,184 vepalib_h3ccas.dll
06/04/2020 06:28 PM 6,772,488 vepalib_hyperv.dll
06/04/2020 06:25 PM 6,469,384 vepalib_vcd.dll
06/04/2020 06:30 PM 65,457,416 vepalib_vmware.dll
06/04/2020 06:30 PM 49,595,656 VmdkMountComponent.dll
20 File(s) 204,523,832 bytes
0 Dir(s) 959,466,872,832 bytes free

C:\>dir "D:\ProgramData\OmniBack\Config\client\components\registry"
Volume in drive D is Новый том
Volume Serial Number is E679-E0FF

Directory of D:\ProgramData\OmniBack\Config\client\components\registry

08/26/2020 11:12 AM <DIR> .
08/26/2020 11:12 AM <DIR> ..
08/10/2020 04:39 PM 342 ArrayProvider3PAR.librarytype
08/10/2020 04:39 PM 346 ArrayProviderNetApp.librarytype
08/10/2020 04:39 PM 351 ArrayProviderUnity.librarytype
08/10/2020 04:39 PM 339 CasDiskComponent.librarytype
08/10/2020 04:39 PM 346 CDpDataMoverComponent.librarytype
08/10/2020 04:39 PM 366 DpBackendService.librarytype
08/10/2020 04:39 PM 309 DPHostingEnvironmentComponent.librarytype
08/10/2020 04:39 PM 322 DpSessionLogger.librarytype
08/10/2020 04:39 PM 374 DpSplitMirrorComponent.librarytype
08/10/2020 04:39 PM 371 H3CCASVirtualEnvironmentProtector.librarytype
08/10/2020 04:39 PM 392 HyperVVirtualEnvironmentProtector.librarytype
08/10/2020 04:39 PM 314 ORMapper.librarytype
08/10/2020 04:39 PM 299 PocoLogger.librarytype
08/10/2020 04:39 PM 303 ScaContainer.librarytype
08/10/2020 04:39 PM 2,030 SerializerBasic.librarytype
08/10/2020 04:39 PM 3,840 SerializerFwsTypes.librarytype
08/10/2020 04:39 PM 315 ServiceFactory.librarytype
08/10/2020 04:39 PM 382 VCDVirtualEnvironmentProtector.librarytype
08/10/2020 04:39 PM 344 VmdkMountComponent.librarytype
08/10/2020 04:39 PM 371 VMwareVirtualEnvironmentProtector.librarytype
20 File(s) 12,056 bytes
2 Dir(s) 959,466,868,736 bytes free

0 Likes
Micro Focus Expert
Micro Focus Expert

Hello @Ravil_M,

Do you see this kind of errors when running a command like omnidb -session -latest, too? Are there recent hard errors in D:\ProgramData\OmniBack\server\db80\pg\log?

Regards,
Sebastian Koehler

---
Please use the Like button below, if you find this post useful.
0 Likes
Captain Captain
Captain

Sebastian, 

in the D:\ProgramData\OmniBack\server\db80\pg\log no errors - only "connection" and "disconnection".

Command "omnidb -session -latest"  working fine, no new errors in the OS system log.

 

In the debug.log exist messages:

2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":112] A.10.70 b210
MALLOC failed Size=18446744072703442944

2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":120] A.10.70 b210
MALLOC failed Size=18446744072703442944, errcode=8

 

I think the size of the requested segment is a little too big -  Size=18446744072703442944.

 

Currently used 72% of total memory on the CS - 14TB of 20TB.

 

 

0 Likes
Micro Focus Expert
Micro Focus Expert

Hello @Ravil_M,

Have you restarted the Cell Manager already? From which versions did you perform the upgrade?

Regards,
Sebastian Koehler

---
Please use the Like button below, if you find this post useful.
0 Likes
Captain Captain
Captain

Sebastian,

CS not restarted yet - some backups running.

Upgrade was from 10.50 to 10.70.

 

Some findings - "MALLOC failed" from debug.log match "Faulting application name: dbsm.exe, version: 10.70.210.0, time stamp: 0x5ed9637f" from Application Log (Windows)

debug.log

2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":112] A.10.70 b210
MALLOC failed Size=18446744072703442944

2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":120] A.10.70 b210
MALLOC failed Size=18446744072703442944, errcode=8

2020-08-26 12:30:29.295 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(omnis.corp.te.ru) .Network performance warning: took 0,843000 sec

2020-08-26 12:30:30.127 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(slm-bkp.corp.te.ru) .Network performance warning: took 0,826000 sec

2020-08-26 12:30:30.949 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(te-rz.corp.te.ru) .Network performance warning: took 0,820000 sec

2020-08-26 12:48:09.216 INET.1076.8468 ["/inet/inet_util.c $Rev$ $Date:: $:":2020] A.10.70 b210
[RxSmallFtp] peerHost:id-srk-02.corp.te.ru, peerProg:DBSM, event:Rejected:/barlists/veagent/id vm id-pyramid-01..04 (temporary)
[KeycloakRESTApiConnector::restMethodExecute] Response: 400 body:

{"error":"invalid_grant","error_description":"Invalid refresh token"}

2020-08-26 12:54:59.775 DBSM.9276.10212 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":112] A.10.70 b210
MALLOC failed Size=18446744072703442944

2020-08-26 12:54:59.775 DBSM.9276.10212 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":120] A.10.70 b210
MALLOC failed Size=18446744072703442944, errcode=8

 

Application log - see picture debug&Application.log.jpeg.

 

 

 

 

0 Likes
Captain Captain
Captain

Sebastian,

CS not restarted - some backups running.

Upgrade was from 10.50 to 10.70.

 

Some findings:

"MALLOC failed" from debug.log match "Faulting application name: dbsm.exe, version: 10.70.210.0, time stamp: 0x5ed9637f" from Application log (Windows)

 

debug.log

 


2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":112] A.10.70 b210
MALLOC failed Size=18446744072703442944

2020-08-26 12:23:23.743 DBSM.1124.7712 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":120] A.10.70 b210
MALLOC failed Size=18446744072703442944, errcode=8

2020-08-26 12:30:29.295 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(omnis.corp.te.ru) .Network performance warning: took 0,843000 sec

2020-08-26 12:30:30.127 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(slm-bkp.corp.te.ru) .Network performance warning: took 0,826000 sec

2020-08-26 12:30:30.949 OMNICC.7792.9332 ["/lib/ipc/ipc.c $Rev$ $Date:: $:":3934] A.10.70 b210
gethostbyaddr(te-rz.corp.te.ru) .Network performance warning: took 0,820000 sec

2020-08-26 12:48:09.216 INET.1076.8468 ["/inet/inet_util.c $Rev$ $Date:: $:":2020] A.10.70 b210
[RxSmallFtp] peerHost:id-srk-02.corp.te.ru, peerProg:DBSM, event:Rejected:/barlists/veagent/id vm id-pyramid-01..04 (temporary)
[KeycloakRESTApiConnector::restMethodExecute] Response: 400 body:

{"error":"invalid_grant","error_description":"Invalid refresh token"}

2020-08-26 12:54:59.775 DBSM.9276.10212 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":112] A.10.70 b210
MALLOC failed Size=18446744072703442944

2020-08-26 12:54:59.775 DBSM.9276.10212 ["/lib/cmn/safemem.c $Rev$ $Date:: $:":120] A.10.70 b210
MALLOC failed Size=18446744072703442944, errcode=8

 

Application.log - see picture.

0 Likes
Captain Captain
Captain
Sebastian, CS not restarted - some backups running.
Upgrade was from 10.50 to 10.70.
0 Likes
Captain Captain
Captain

Sebastian, some findings - "MALLOC failed" from debug.log match "Faulting application name: dbsm.exe, version: 10.70.210.0, time stamp: 0x5ed9637f" from Application log (Windows).

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.