Despina_1 Valued Contributor.
Valued Contributor.
1937 views

VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hello.

We have a Cell Manager in a Windows Server 2012 R2 with Data Protector 9.07 installed. Cell Manager is also the Backup Host for the VE integration.

We have some virtual machines (hyper-v) that we want to backup using the VE integration. In some of them we have oracle forms and reports server cluster installed and when we try to back them up we get the following error:

[Normal] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:31:43 πμ
 Volume Shadow Copies successfully created.

[Normal] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:31:43 πμ
 Exiting Shadow Copy Set creation phase.

[Major] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:31:44 πμ
[145:575]  Writer 'Microsoft Hyper-V VSS Writer' failed to prepare files for backup:
     Reported state: VSS_WS_FAILED_AT_PREPARE_SNAPSHOT
     Expected state: VSS_WS_WAITING_FOR_BACKUP_COMPLETE
     Failure code:  VSS_E_WRITERERROR_NONRETRYABLE

[Major] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:31:44 πμ
 Cannot perform backup of:
  '/Microsoft Hyper-V VSS Writer/Virtual Machines/Online\APPL26',
 which contains data in:
  C:\ClusterStorage\volume1\hyper-v\appl26\Virtual Machines\ED770C97-4775-4AFA-A64C-91DFDF7D9D5F.xml
  C:\ClusterStorage\volume1\hyper-v\appl26\Virtual Hard Disks\APPL26_C.VHDX
  C:\ClusterStorage\volume1\hyper-v\appl26\Virtual Hard Disks\APPL26_D.VHDX
  C:\ClusterStorage\volume1\hyper-v\appl26\Snapshots\49A6C42B-0EBF-462D-B2E0-362C5307FEA9.xml
  C:\ClusterStorage\volume1\hyper-v\appl26\Virtual Hard Disks\APPL26_C-AutoRecovery.avhdx
  C:\ClusterStorage\volume1\hyper-v\appl26\Virtual Hard Disks\APPL26_D-AutoRecovery.avhdx

[Major] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:31:44 πμ
 There is no data to be backed up.

[Major] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:32:20 πμ
 Backup failed.

[Normal] From: OB2BAR_VSSBAR@hyper11.xxx.gr "HyperV"  Time: 1/4/2018 8:32:22 πμ
 Deleting Volume Shadow Copies and releasing the volumes.

[Major] From: VEPALIB_HYPERV@dpsrv01 "HyperV"  Time: 1/4/2018 8:32:19 πμ
 VSS Agent finished with errors.

 

Cell Manager/Backup Host: dpsrv01 (Windows Server 2012 R2)

Physical server: hyper11 (Windows Server 2012 R2)

VM: efkaappl26 (Windows Server 2008 R2)

 

If we stop the oracle services, VSS has no problem and the backup runs without errors.

Does anyone have any idea on what to do?

Thanks

 

0 Likes
9 Replies
Micro Focus Expert
Micro Focus Expert

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hi Despina,

Could you please paste the errors from the Event logs: Hyper-V-VMMS (Hyper-V) / Application (Hyper-V and VM) at the time of the backup. 

Regards

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hello Despina_1,

Looking at the error

[145:575] Writer 'Microsoft Hyper-V VSS Writer' failed to prepare files for backup:
Reported state: VSS_WS_FAILED_AT_PREPARE_SNAPSHOT
Expected state: VSS_WS_WAITING_FOR_BACKUP_COMPLETE
Failure code: VSS_E_WRITERERROR_NONRETRYABLE


Error code VSS_E_WRITERERROR_NONRETRYABLE(0x800423F4) the failure is because of any particular VSS writer is not in a good state causing the snapshot process to fail and hence the backup fails.

This can be verified using the following command from an elevated command window.

vssadmin list writers

You might want to check your VSS writers - identify them.

Next one I will recommend - log a SW support case so that DP Support can work with you on this one. it looks like this will need an in-depth investigation.

Hope this helps

Rgds,

Sherman

DP Support

 

 

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hi Despina,

Can you check if the Oracle VSS Provider is enabled and started on this VM? This is a optional component to the Oracle database on Windows and might cause trouble if the VSS requester is waiting for it.

Regards,
Sebastian Koehler
---
Please use the Like button below, if you find this post useful.
0 Likes
Despina_1 Valued Contributor.
Valued Contributor.

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

@fbrenesPlease see the attached file for the logs.

@Sherman_S-DP_L3 We have already checked the writers and after the backup the Microsoft Hyper-V VSS Writer is in a "non-retryable error" state. We reset the status of the writer by restarting the "Hyper-V Virtual Machine Management" service and the same thing happens after the next backup

@Sebastian.Koehler There isn't an Oracle VSS Provider on the VM. We have only two providers: 'Hyper-V IC Software Shadow Copy Provider' and 'Microsoft Software Shadow Copy provider 1.0'

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hi Despina,

Thank you. Looking at the Event logs the issue seems to be coming from MS. I would recommend to check the latest patches installed in the VM and Hyper-V, please be aware the same issue has been already reported for other backup vendors for example:

https://social.technet.microsoft.com/Forums/windows/en-US/0b4a8b65-00ac-4185-afa2-5c783f79a462/vss-writer-failures-during-vm-backup-of-server-with-oracle-application-installed?forum=windowsbackup

And looking at the Events from your enviroment they match with the same symptoms.

I would try:

https://social.technet.microsoft.com/Forums/windows/en-US/f410e62d-a5ed-467c-921f-25cd38307bbc/not-all-of-the-shadow-volume-arrived-in-the-guest-operating-system?forum=winserverhyperv

And also install this patch:

https://support.microsoft.com/en-us/help/2996928/backup-task-fails-with-a-time-out-error-in-windows

If none of them fixes the issue then I would recommend to open a case with MS support.

Best Regards

 

0 Likes
Despina_1 Valued Contributor.
Valued Contributor.

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

@fbrenes Thanks for your help! We'll look into these and i'll let you know.

Regards

0 Likes
Despina_1 Valued Contributor.
Valued Contributor.

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

We tried everything - no success!

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

Hello @Despina_1,

@Despina_1 wrote:

If we stop the oracle services, VSS has no problem and the backup runs without errors.

Is the Oracle instance very busy creating a massive amount of archive log files or audit files keeping the disk busy? Maybe the instance is misconfigured for the workload.

Regards,
Sebastian Koehler

---
Please use the Like button below, if you find this post useful.
0 Likes
Despina_1 Valued Contributor.
Valued Contributor.

Re: VE backup and Oracle - Writer 'Microsoft Hyper-V VSS Writer' failed

We have some systems with Oracle Forms and Report Server (Oracle Fusion Middleware) in a standalone mode and we have no problem backing them up.  The problem lies within the systems that have an Oracle Fusion Middleware cluster installation and the reports are saved in a Windows cluster file share. These systems we cannot backup up and we get the error I mentioned in the first post. 

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.