NDMP limitations

[George Mitchell]

We have started using NDMP backups for our new VNX array, and all seems to be working well.

However I found this gem in software notes:

The NDMP integration can handle backup sessions involving up to 20 million files if up to 10% of the total number of backed up files are directories, for an average directory name length of 25 characters, and average filename length of 10 characters. In such a case, the NDMP integration allocates up to 1.9 GB of system memory and 2.8 GB of disk space.

For optimal performance the recommended number of files and directories for an NDMP backup specification is 10 million. The default upper limit for the number of files for an NDMP backup specification is 5 million. To enable higher values, the OB2NDMPMEMONLY omnirc file variable must be set to 0.

If we have more than 20 million files and directories in a mount point what will happen?  I think we already exceed this: 19,018,766 Files / 1,963,250 Folders on /CIFS.

How would I split up the backup to work correctly or optimally?

[Sebastian.Koehler]

If you face this limitation you might need to setup multiple backup jobs starting in different root directories of the VNX. If /data_user_1 is your fs root, and you have multiple larger subdirs there, then you need to specify /data_user_1/folder_1 for example for a single backup job.

From the VNX perspective it might be worth to split the filesystems into smaller pieces to bypass this limitation and be able to backup a complete filesystem. Usually it will slice the volumes anyway and it does not matter if you have one big or 10 smaller filesystems.

Regards,
Sebastian