MSL Library "Device could not be accessed"

I am running DP9.07.   Just before upgrading from 9.06 to 9.07 our Data Protector is reporting a wierd error.  I can get it to happen even just by trying to format a tape.  What is wierd is that it happens... the tape does get formatted... then the message happens again.... and again...  Ideas how to debug?  Our CM is HP-UX 11.31.  The tape library is an MSL6060

Sample session:

[Normal] From: MSM@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:43:02 PM
 Media session 2016/09/20-109 started.

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:43:02 PM
 STARTING Media Agent "LTO_Drive_2"

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:43:02 PM
 => UMA@sapbck.ssmhc.com@/dev/rchgr/autoch3
 Loading medium from slot 3 to device /dev/rtape/tape36_BESTn

[Warning] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:51:07 PM
 The device "LTO_Drive_2" could not be opened("Device could not be accessed")

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:51:07 PM
 Starting the device path discovery process.

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:51:12 PM
 A new path(s) "/dev/rtape/tape36_BESTn" found for device "LTO_Drive_2" and stored.

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:51:12 PM
 Device path discovery process finished.

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:58:16 PM
 => UMA@sapbck.ssmhc.com@/dev/rchgr/autoch3
 Unloading medium to slot 3 from device /dev/rtape/tape36_BESTn

[Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2"  Time: 9/20/2016 1:58:28 PM
 COMPLETED Media Agent "LTO_Drive_2"

 

Tags:

  • This looks like a hardware releated issue or something on the SAN (if the MSL6060 uses a NSR). You should check NSR logs and perform a drive assesment tests. Anyone swapped FC ports on the SAN switches? Is the drive working properly after the new path was discovered and stored?

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler

  • I am in the office Tuesday and was planning on doign the swap thing.  No wiring or connections have changed.  Only going from 9.06 to 9.07.  The CM hasn't had anything but the DP patches applied.

    I'll post the results of the swap when I get it done,

  • This was a question not an action plan. Sorry for being not more clear. :) If nothing changed something might be broken (NSR or drive). Please check logs as recommended before changing anything. I guess you will see some SCSI errors in the NSR log regarding one of the drives.

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler

  • Finally got response from HP.  They are having me run diagnostics using LTT.  I had the storage guy look at the fiber connections and logs on the switch and he said there were no issues.

    More mysteries.

    Thanks for the replies.

  • Was this a one-time event? Is the drive working properly after the new path was discovered and has been stored by Data Protector? Was the Drive Assessment test successful?

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler


  • wrote:

    Was this a one-time event? Is the drive working properly after the new path was discovered and has been stored by Data Protector? Was the Drive Assessment test successful?

    No, still fails.  In fact, it tries to find the missing drive when I use it, syas it finds it, the drive operation (no matter what I choose) fails.  I can repeat this over and over on the drive.

  • Please confirm that the serial number and drive index stored in Data Protector is correct (Device & Media context). Then choose the correct path (the one with the green tick) from the SCSI address dropdown and replace the existing path. If this fails share devbra -dev output and a screenshot of the drive configuration page.

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler

  • The drive shows it is OK from the command line AND inside of DP.

    Regardless of it being 'green' it still can't find it... finds it.. .then fails.

    Exch HP:MSL6000 Series Path: "/dev/rchgr/autoch3" SN: "3G49LRT62Y21"

    Description: HP StorageWorks MSL 6000 Series

    Revision: 0520 Flags: 0x0006 Slots: 58 Drives: 4

    Drive(s) SN:

    "MXP2J04Z1S"

    "MXP3F010VE"

    "MXP7G00Q2E"

    "MXP2J00QU8"

    Tape HP:Ultrium 2-SCSI Path: "/dev/rtape/tape32_BESTn" SN: "MXP7G00Q2E"

    Description: HP LTO drive

    Revision: F6AW Device type: lto [13] Flags: 0x0001

    Tape HP:Ultrium 2-SCSI Path: "/dev/rtape/tape28_BESTn" SN: "MXP2J04Z1S"

    Description: HP LTO drive

    Revision: F6AW Device type: lto [13] Flags: 0x0001

    Tape HP:Ultrium 2-SCSI Path: "/dev/rtape/tape36_BESTn" SN: "MXP3F010VE"

    Description: HP LTO drive

    Revision: F6AW Device type: lto [13] Flags: 0x0001

    Tape HP:Ultrium 2-SCSI Path: "/dev/rtape/tape35_BESTn" SN: "MXP2J00QU8"

    Description: HP LTO drive

    Revision: F6AW Device type: lto [13] Flags: 0x0001

    Here is a session I just ran:

    [Normal] From: MSM@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:31:58 PM

    Media session 2016/09/28-92 started. 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:32:03 PM

    STARTING Media Agent "LTO_Drive_2" 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:32:03 PM

    => UMA@sapbck.ssmhc.com@/dev/rchgr/autoch3

    Loading medium from slot 2 to device /dev/rtape/tape36_BESTn 

    [Warning] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:40:08 PM

    The device "LTO_Drive_2" could not be opened("Device could not be accessed") 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:40:08 PM

    Starting the device path discovery process. 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:40:14 PM

    A new path(s) "/dev/rtape/tape36_BESTn" found for device "LTO_Drive_2" and stored. 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:40:14 PM

    Device path discovery process finished. 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:47:20 PM

    => UMA@sapbck.ssmhc.com@/dev/rchgr/autoch3

    Unloading medium to slot 2 from device /dev/rtape/tape36_BESTn 

    [Normal] From: MMA@sapbck.ssmhc.com "LTO_Drive_2" Time: 9/28/2016 12:47:54 PM

    COMPLETED Media Agent "LTO_Drive_2"

    ============================================================================

    0 media out of 1 successfully initialized.

    ============================================================================

  • Are you using SCSI reserve/release on the drive (Adanced Options)? If yes, have you tried to power cycle the drive/library? Maybe there is a bma or rma process hanging on any Media Agent hosts that has access to this drive.

    Can you share the NSR eventlog entries at the time when you try to access the drive. The rescan is just a "last resort" and not our issue.

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler


  • wrote:

    Are you using SCSI reserve/release on the drive (Adanced Options)? If yes, have you tried to power cycle the drive/library? Maybe there is a bma or rma process hanging on any Media Agent hosts that has access to this drive.

    Can you share the NSR eventlog entries at the time when you try to access the drive. The rescan is just a "last resort" and not our issue.

    Please use the Accept Solution button next to my post and assign a KUDO (thumbs up icon) if this works for you.

    Regards,
    Sebastian Koehler


    Not sure about the Advanced options or 100% where to look,  The main (CM) server has also been bounced more than a few times.  Our storage guy looked at the switch logs today and said there were no errors (he was on vcation! How dare he!). HPE wants me to run some LTT stuff on the drive now.  Just have to wait for a quiet moment went NONE of the drives are used.

    I'll post the results.

    -Vince