Dear All,

While testing the DP backup on S3, we encountered an error.

After enabling extended logging on S3, we saw that DP successfully creates a bucket,

initializes a session for multipart upload and uploads a file (piece).

Next, according to the protocol, it is necessary to transfer to the S3 (storage) side

a command to complete the multipart and an xml file, which should list all the downloaded

parts of the future file for its assembly.

The format of this xml file is defined by the Amazon S3 API specification at the link below:

docs.aws.amazon.com/.../API_CompleteMultipartUpload.html

In particular, it says that the xml file should contain information about the PartNumber

and ETAG of each uploaded part: “For each part in the list, you must provide the PartNumber

value and the ETag value that are returned after that part was uploaded.”

ETAG is a field containing the MD5 hash sum of the object, which the storage system returns

to the application after loading each part

Log analysis showed that Data Protector sends an xml file with an empty ETAG field:

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]: ([]uint8) (len=109 cap=512) {

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000000  3c 43 6f 6d 70 6c 65 74  65 4d 75 6c 74 69 70 61  |<CompleteMultipa|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000010  72 74 55 70 6c 6f 61 64  3e 0a 09 3c 50 61 72 74  |rtUpload>..<Part|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000020  3e 0a 09 09 3c 50 61 72  74 4e 75 6d 62 65 72 3e  |>...<PartNumber>|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000030  31 3c 2f 50 61 72 74 4e  75 6d 62 65 72 3e 0a 09  |1</PartNumber>..|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000040  09 3c 45 54 61 67 2f 3e  0a 09 3c 2f 50 61 72 74  |.<ETag/>..</Part|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000050  3e 0a 3c 2f 43 6f 6d 70  6c 65 74 65 4d 75 6c 74  |>.</CompleteMult|

Apr 18 12:43:37 hostname1 GW-hostname.domain [3995171]:  00000060  69 70 61 72 74 55 70 6c  6f 61 64 3e 0a           |ipartUpload>.|

That is, DP sends this xml file to the storage side:

<CompleteMultipartUpload>

<Part> <PartNumber>1</PartNumber>

<ETag/>

</Part>

</CompleteMultipartUpload>

--

This looks like a violation of the S3 protocol and the "complete multipart" operation fails.

Does anyone has an experience with the above functionality? Any ideas? 

Maybe it was done on purpose? Maybe there are some local (omnirc) or global (global)

parameters that allows you to enable for usage of the ETAG field?

Thank you!