Idea ID: 1681644

Cloud (Azure) Gateway support for Data Protector 10.10, running on RHEL 7.x.

Status : Delivered
over 2 years ago

Hello,

We have been using the Cloud (Azure) gateway to perform backups directly to Azure storage.  It seems to be working quite well for smaller backups, but every now and then, for large backups, we see an error that appears to be a communication interruption with Azure.  The exact error details are:

[Warning] From: BMA-NDMP@ucwdpbs001.wilm.ppdi.com "DP_Azure_Cloud_gw1 [GW 5233:0:541170802490011539]" Time: 1/14/2019 1:06:03 PM

Error in communication with cloud [UNKNOWN EXCEPTION], retrying.

It looks like data is still being sent to Azure as the 'counters' keep increasing, ie. Done - 245GB and counting, or Done (%) - 36%...37%...38%...etc.

After a while though, if this error occurs again, one or two more times, we then see the following errors:

[Major] From: BMA-NDMP@ucwdpbs001.wilm.ppdi.com "DP_Azure_NixOracle_Full_Clou_gw1 [GW 8923:2:541170802490011539]" Time: 1/13/2019 4:27:22 PM

[90:51] \\dpcloudtier.blob.core.windows.net\dp-nixoracle-full-backups/b9b411ac_5c3abdb5_7880_3323

 

Cannot write to device (Invalid arguments.)

 

[Major] From: BMA-NDMP@ucwdpbs001.wilm.ppdi.com "DP_Azure_NixOracle_Full_Clou_gw1 [GW 8923:2:541170802490011539]" Time: 1/13/2019 4:27:22 PM

[90:64] Cannot unload exchanger medium (Invalid arguments.)

After which the Media Agent is aborted, the medium is unloaded from the slot and the job stops, as "Completed/Failures".  Again, smaller backup jobs do not seem to have an issue and complete successfully.

I opened a case with tech. support and the first thing they stated was that the Platform Support Matrix shows that our installation (Cell Manager, DP 10.10 on RHEL 7.4) does not support the "Cloud (Azure) Device" support matrix.

I'm trying to request full support of the "Cloud (Azure) Device" for DP 10.10 running on RHEL 7.x).

Thanks,

Sven

Labels:

Devices
Operating System
Cloud
  • This finding seems to indicate a bug and the DP support ticket should help in resolving this.

    Thanks,

    Shishir


  • i think something is different in version 10.40 and cannot create a new blob if the 75 GB limit exceeed, because on a similar setup in version 09.08 i can see that more than on blobs are created if needed(see attached).

     

    This is from Data Protector documentation on azure device limits

    Cloud (Azure) device blob size limit
    A Data Protector medium is uploaded to Cloud (Azure) device as one or more blobs of data and
    metadata. Cloud (Azure) has a blob size limit of 195 GB, whereas Data Protector media has no upper
    size limit. However, to comply with this restrictions, one Data Protector medium can span multiple
    blobs, the maximum size of each being 75 GB. The exact amount of data stored per blob is a function of
    how compressible is the data.

  • do you know if there is any variable to keep the connection active?

  • yes i have a ticket running, the problem is for every test you have to wait some hours.

     

    i will try to enable the debug and check again.

  • Hello.

    We'd tested backups larger than single blobs and hadn't run into problems at the time. Perhaps it might be worth checking this again. You can also consider raising a support ticket, after generating debugs using circular logging to control disk usage.

    Thanks,

    Shishir