Highlighted
Absent Member.
Absent Member.
149 views

Slow Lotus backups

Dear all,
I recently moved the backup system for a Domino cluster node (not MS cluster) from TSM to DataProtector with Lotus Integration.
I've no errors or warnings during backups (for both ArchiveLogs and DBs), BUT the performances of the backups are really really poor...
And when I say poor, I mean an average data transfer of about 3GB/hour. Considering that this machine creates 10GB of archive logs/day and have 400GB between DBs and NFSs... It's a mess...
The server is a Windows 2003 SP1 (patched) with 2x Xeon Hyperthreaded 2.7GHz and 4GB ram.
I've checked at network level and it's ok (2% bandwidth used on the 1Gbps lan card during backup, no errors at datalink level like duplex mismatch and so on).
The CPU load seems normal
The ram utilization seems normal too
The machine which is the "bridge" between italian network and tape drives (so called "Media Agent") has no problem of memory or network or cpu overhead (with 3 simultaneous backups the bandwidth used is about 20% of the 1Gbps lan card which receive the backup incoming traffic).
No errors in the logs of the machines...

Do you have some idea about this problem?

The Cell server is HP-UX DP 5.5 fully patched, the media server is W2003 5.5 fully patched (00359 for Core and 00365 Media agent).
The Domino has currently the latest version of DP6.0 agents (I tryed with them also), but with 5.5 agents it had exactly the same behaviour...

I attach here a small part of the log of yesterday archive log backup:

[Normal] From: BSM@itdcmaf9mil "Domino_MD0_LotusDomino_TransLog" Time: 21/07/2008 14.46.26
OB2BAR application on "itmilmd0" successfully started.

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 7/21/2008 2:45:34 PM
Lotus: Lotus Notes C API init succeeded.

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.34
Checking configuration
'util_notes.exe -chkconf -server:ITMILMD0'...

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.34
Checking of configuration succeeded.

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.35
Backing up archive logs...

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.35
Backing up file 'ArchiveLog:0088164' ...
File size: 64 MB

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.35
Backing up file 'ArchiveLog:0088165' ...
File size: 64 MB

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.35
Backing up file 'ArchiveLog:0088166' ...
File size: 64 MB

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.45.35
Backing up file 'ArchiveLog:0088167' ...
File size: 64 MB

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_02" Time: 21/07/2008 14.46.08
STARTING Media Agent "ESLA_lto2_02"

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_05" Time: 21/07/2008 14.46.08
STARTING Media Agent "ESLA_lto2_05"

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_05" Time: 21/07/2008 14.46.09
By: UMA@dp@/dev/rac/ESL_A
Loading medium from slot 136 to device Tape4:0:1:0

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_02" Time: 21/07/2008 14.46.28
By: UMA@dp@/dev/rac/ESL_A
Loading medium from slot 574 to device Tape1:0:0:1

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.47.19
Starting OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088167 "Lotus"

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.47.19
Starting OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088165 "Lotus"

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.47.22
Starting OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088166 "Lotus"

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.47.22
Starting OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088164 "Lotus"

Archived logId: 4406308 -1054511250 -1188556182 924 Extent 88167

Archived logId: 4406308 -1054511250 -1188556182 924 Extent 88165

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.52.34
Completed OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088167 "Lotus"

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.52.34
Backing up file 'ArchiveLog:0088168' ...
File size: 64 MB

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.52.34
Starting OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088168 "Lotus"

Archived logId: 4406308 -1054511250 -1188556182 924 Extent 88166

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.52.38
Completed OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088165 "Lotus"

Archived logId: 4406308 -1054511250 -1188556182 924 Extent 88164

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 14.52.38
Backing up file 'ArchiveLog:0088169' ...
File size: 64 MB

.
.
.
.
.
.
.
.

[Normal] From: OB2BAR_LDBAR@itmilmd0.eur.cchbc.com "ITMILMD0" Time: 21/07/2008 16.49.39
Completed OB2BAR Backup: itmilmd0:ArchiveLog_ITMILMD0:0088247 "Lotus"

[Normal] From: BSM@itdcmaf9mil "Domino_MD0_LotusDomino_TransLog" Time: 21/07/2008 16.50.43
OB2BAR application on "itmilmd0" disconnected.

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_02" Time: 21/07/2008 16.50.14
Tape1:0:0:1
Medium header verification completed, 0 errors found

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_02" Time: 21/07/2008 16.50.34
By: UMA@dp@/dev/rac/ESL_A
Unloading medium to slot 574 from device Tape1:0:0:1

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_02" Time: 21/07/2008 16.50.50
COMPLETED Media Agent "ESLA_lto2_02"

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_05" Time: 21/07/2008 16.51.01
Tape4:0:1:0
Medium header verification completed, 0 errors found

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_05" Time: 21/07/2008 16.51.21
By: UMA@dp@/dev/rac/ESL_A
Unloading medium to slot 136 from device Tape4:0:1:0

[Normal] From: BMA@itmilid1.eur.cchbc.com "ESLA_lto2_05" Time: 21/07/2008 16.51.37
COMPLETED Media Agent "ESLA_lto2_05"

[Normal] From: BSM@itdcmaf9mil "Domino_MD0_LotusDomino_TransLog" Time: 21/07/2008 16.52.40

Backup Statistics:

Session Queuing Time (hours) 0,00
----------------------------------------
Completed Disk Agents ........ 166
Failed Disk Agents ........... 0
Aborted Disk Agents .......... 0
----------------------------------------
Disk Agents Total ........... 166
========================================
Completed Media Agents ....... 2
Failed Media Agents .......... 0
Aborted Media Agents ......... 0
----------------------------------------
Media Agents Total .......... 2
========================================
Mbytes Total ................. 10674 MB
Used Media Total ............. 2
Disk Agent Errors Total ...... 0




Thanks to all, and best regards,
Alessandro Guarda
0 Likes
2 Replies
Highlighted
Absent Member.
Absent Member.

Re: Slow Lotus backups

Adding another question: the Lotus integration has this "Backup buffer size" option (default 32kB). How does it influence the agent behaviour?
In the documentation there is nothing about it and I would like to know what is this option for...

Thanks, Alex
0 Likes
Highlighted
Absent Member.
Absent Member.

Re: Slow Lotus backups

So. The performance problem has been solved.
One month aho I prepared a brand new installation of HP-UX 11.31 with DP 6.0 (fully patched) on a RX3600.
This will become the productive machine at the end of August 2008.
I moved the Domino client from current productive system to this one, and "magically" from 3GB/h on current DP environment it runs at more than 1GB/min on the new prod. env.
I'm really happy, also because this will be the next prod. environment.

In any case, I still don't understand what caused this issue.
And the question on the "Backup buffer size" is still open: yesterday I tryed to bring this value to 1024kB, 8192kB, and finally also to 65536kB and I didn't see any difference in the backups ran with these different values...

Thanks, Alex
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.