Anonymous_User Absent Member.
Absent Member.
270 views

Replica stuck in Move State


I am in the process of merging a number of trees at remote locations
into one central tree. Completed the first 2 sites with no problems and
they were 2 of the larger directories. The third was a single server
tree and it did the merge fine but now after moving the OU to a
different O using the same process that I did with the other 2 sites the
replica is now locked in the Move state after 30+ hrs and I can't get it
to release.

Mix of OES 11 & NW 6.5 SP8 servers

NDStrace on the Master Replica server shows

MoveTree – move for OU1.O.TREE, contacted server .server01.OU2.O.Tree,
Process failed, no such entry (-601)
Also shows Sync to server failed, no access (-672)

DSRepair shows the following

Command Line Options: -RD -M
Initiated by user: .admin.O.TREE.

/****************************************************************************/
Repair utility for Novell eDirectory 8.8 - 8.8 SP7 v20704.00, DS
20704.00.
Repairing Local Database
Start: Monday, July 15, 2013 20:12:21 Local Time

** All disk amounts are approximations **
Disk space currently available: 42999 MB
->DSRepair may need to use: 630 MB
->Disk space remaining after operation: 42999 MB

Current transaction ID is 736033 (0xb3b21). Allowed limit of transaction
is 4294959104 (0xffffe000)
Physical Check
Repair Trees - Scan Values
Repair Trees - Sorting Values
Repair Trees - Scan Entries
Repair Trees - Sorting Entries
Repair Trees - Check Values
Total Objects in Database: 11435
Total Objects in Schema : 2730
Total External References: 1
Total Objects in Replicas: 8701
Schema Check


Repairing objects in a replica
Start: Monday, July 15, 2013 20:12:21 Local Time

Total objects in partition - T= TREE : 5553
Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(3000)
Repairing objects - done(4000)
Repairing objects - done(5000)
Found OBT_MOVED
Attribute 51e3fa76, Obituary
Object ID: 0000b97f, DN: OU=OU1.O=O1.T= TREE
Repairing objects - done(5553)

Total Objects = 5553, UNKNOWN class objects = 0, Total Values = 165970

Total objects in partition - OU=OU1.O=O1.T= TREE: 3148

Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(3000)
Repairing objects - done(3148)

Total Objects = 3148, UNKNOWN class objects = 0, Total Values = 85543
Total partition and object events rejected during repair - 1
Repairing single object:

Object ID: 00008038, [Pseudo Server]

Total Objects = 1, UNKNOWN class objects = 0, Total Values = 116
Total partition and object events rejected during repair - 1
Total partition and object events rejected during repair - 1
Finish: Monday, July 15, 2013 20:12:26 Local Time
Total repair time: 0:00:05
Checking stream syntax files
Repair process completed, total errors found = 0


It appears the move was successful as all the objects are in the correct
place and seem to be functional. I am just unable to complete the
process.


--
kbent
------------------------------------------------------------------------
kbent's Profile: https://forums.netiq.com/member.php?userid=5469
View this thread: https://forums.netiq.com/showthread.php?t=48192

Labels (1)
0 Likes
8 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State

How about try the following on all servers that were in the replica rings
and then post the output?

Code:
----------
ndsrepair -E
----------

I assume you took the merging-in trees down to one replica-holder before
the merge, but is that a valid assumption?

Good luck.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State


Yes there was only one master Replica before the merge.

ndsrepair -E results

/****************************************************************************/
Repair utility for Novell eDirectory 8.8 - 8.8 SP7 v20704.00, DS
20704.00.
Start: Monday, July 15, 2013 20:48:18 Local Time
Retrieve replica status

Partition: .[Root].
Replica: .BRHS1.BRHS.AVRSB ********** ******** -603
Replica: .AVRSB01.OPS.AVRSB 07-15-2013 09:37:45
Server: CN=BRHS1.OU=BRHS.O=AVRSB 07-15-2013 20:48:08 -672
Remote
Object: [Root]

Partition: .BRHS.AVRSB
Replica: .AVRSB01.OPS.AVRSB 07-15-2013 20:48:17
All servers synchronized up to time: 07-15-2013 20:48:17
Finish: Monday, July 15, 2013 20:48:18 Local Time


--
kbent
------------------------------------------------------------------------
kbent's Profile: https://forums.netiq.com/member.php?userid=5469
View this thread: https://forums.netiq.com/showthread.php?t=48192

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State

A -672 is 'ACCESS_DENIED' and in the context of a report synchronization
status it typically means that one of the servers really holds a replica
(and shows up in the replica ring) and the other one thinks it does, but
is not in the replica ring, and must be cleaned out. I am guessing that
the AVRSB01 server is the real one that should hold all replicas, but if
BPHS1 should also hold all replicas then we should probably proceed with
some caution. For now, run the following on both servers and post back
the output indicating which server ran each command:

Code:
----------
ndsstat -r
----------

Also, on which server was that ndsrepair command executed? Please run the
same command on the other server and post its output.

Good luck.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State


ndsstat -r results from AVRSB01 (ndsrepair results from same server)

/****************************************************************************/
Repair utility for Novell eDirectory 8.8 - 8.8 SP7 v20704.00, DS
20704.00.
Start: Monday, July 15, 2013 20:48:18 Local Time
Retrieve replica status

Partition: .[Root].
Replica: .BRHS1.BRHS.AVRSB ********** ******** -603
Replica: .AVRSB01.OPS.AVRSB 07-15-2013 09:37:45
Server: CN=BRHS1.OU=BRHS.O=AVRSB 07-15-2013 20:48:08 -672
Remote
Object: [Root]

Partition: .BRHS.AVRSB
Replica: .AVRSB01.OPS.AVRSB 07-15-2013 20:48:17
All servers synchronized up to time: 07-15-2013 20:48:17
Finish: Monday, July 15, 2013 20:48:18 Local Time

BRHS1 is a NW 6.5 server so that command doesn't work on it.

AVRSB01 is the Master Replica server. BRHS1 is the only server in the
location that was merged. The OU is it in is still a separate partition
and it has a Replica of that partition on the AVRSB01 server. BRHS1
says it has no replica reporting in dsrepair but at the Tree level it
shows a R/W for the BRHS partition is on the BRHS server.


--
kbent
------------------------------------------------------------------------
kbent's Profile: https://forums.netiq.com/member.php?userid=5469
View this thread: https://forums.netiq.com/showthread.php?t=48192

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State

Before doing anything too exciting in this two-server tree, get a backup
of each DIB. If these are VMs, you could use snapshots to do this (stop
eDir, get a snapshot, start eDir) or else you can use 'dsrepair' on
NetWare and the ndsrc.pl (shameless plugin) script from the CoolSolutions
site for the Linux box. Try running these two commands at the same time
if possible. On NetWare, run it like this:

Code:
----------
dsrepair -rc
----------

On Linux, run it like this (
https://www.novell.com/communities/node/1129/ndsrc ) and follow the
prompts, usually by pressing [Enter] several times and that's it:

Code:
----------
ndsrc.pl
----------

Okay, so you now have backups just in case. The next part involves
telling the real server (AVRSB01) that the other server (BRHS1) is not
really in the replica ring for the [root] partitions. AVRSB01 is the
Linux system so we'll use ndsrepair for this little bit of surgery. Start
by loading ndsrepair in Advanced mode for partition editing:

Code:
----------
ndsrepair -P -Ad
----------

Choose the appropriate partition ([root], so probably number '1').

Choose to view the replica ring (10).

Choose the listing for the soon-to-be-removed server (probably '2', but
verify that).

Choose the option to 'Remove this server from the replica ring' and then
follow the prompts until done. See if the output from your Report
Synchronization Status (on both boxes) then clears up. If anything goes
wrong you'll need to restore the DIBs and try again, or call Novell/NetIQ
for support.

Good luck.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State


Ok That is complete and it removed the BRHS1 server from the ring.

AVRSB01:~ # ndsstat -r

[1] Instance at /etc/opt/novell/eDirectory/conf/nds.conf:
AVRSB01.OU=OPS.O=AVRSB.AVRSB-TREE


Replicas of Partition : .T=AVRSB-TREE.
Server Name Replica Type Replica State
Agent Revision Agent Version

..CN=AVRSB01.OU=OPS.O=AVRSB.T=AVRS... Master Move State 0
20704.00 eDirectory for Linux x86_64 v8.8 SP7 [DS]


Replicas of Partition : .OU=BRHS.O=AVRSB.T=AVRSB-TREE.
Server Name Replica Type Replica State
Agent Revision Agent Version

..CN=AVRSB01.OU=OPS.O=AVRSB.T=AVRS... Master Move State 1
20704.00 eDirectory for Linux x86_64 v8.8 SP7 [DS]


I am still seeing the Replica stuck in Move state and the following on
dstrace

MoveTree - move .BRHS.AVRSB.AVRSB-TREE. moving to .AVRSB.AVRSB-TREE..
MoveTree - contacting server .iSCSI01.OPS.AVRSB.AVRSB-TREE..
MoveTree - move for .BRHS.AVRSB.AVRSB-TREE., contacted server
..iSCSI01.OPS.AVRSB
..AVRSB-TREE., Process failed, no such entry (-601).
MoveTree - end notify for .BRHS.AVRSB.AVRSB-TREE., Process succeeded.
Sync - Partition .BRHS.AVRSB.AVRSB-TREE. All processed = YES
Skulk Partition - change cache entry count 1661 for
..BRHS.AVRSB.AVRSB-TREE.
2013/07/16 9:46:52 Start partition sync .BRHS.AVRSB.AVRSB-TREE.
state:[81], typ
e:[0].
MoveTree - start notify for .BRHS.AVRSB.AVRSB-TREE..
MoveTree - move .BRHS.AVRSB.AVRSB-TREE. moving to .AVRSB.AVRSB-TREE..
MoveTree - contacting server .iSCSI01.OPS.AVRSB.AVRSB-TREE..
MoveTree - move for .BRHS.AVRSB.AVRSB-TREE., contacted server
..iSCSI01.OPS.AVRSB
..AVRSB-TREE., Process failed, no such entry (-601).
MoveTree - end notify for .BRHS.AVRSB.AVRSB-TREE., Process succeeded.
Sync - Partition .BRHS.AVRSB.AVRSB-TREE. All processed = YES

Thanks for your help so far.


--
kbent
------------------------------------------------------------------------
kbent's Profile: https://forums.netiq.com/member.php?userid=5469
View this thread: https://forums.netiq.com/showthread.php?t=48192

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State

What's this?

iSCSI01.OPS.AVRSB.AVRSB-TREE

Is it working? Is it in replica rings? If an old/decommissioned server
that still has an object in the tree, delete that object, but if not,
perhaps this needs to be cleaned up first.

Good luck.
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Replica stuck in Move State


That is just another server that is in the tree.
There is no replica on it and it is not doing anything yet so I just
removed it from the tree and I can add it back in later.
As soon as I removed it the error went away and the replica has switched
back to the ON state.

Things look all better now! Thanks so much for your help.


--
kbent
------------------------------------------------------------------------
kbent's Profile: https://forums.netiq.com/member.php?userid=5469
View this thread: https://forums.netiq.com/showthread.php?t=48192

0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.