Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner
3243 views

-626 ERR_ALL_REFERRALS_FAILED

Hello

Running eDirectory 9.0.4 v40006.34 on SLES11SP4.
Together with IDM 4.6.1

I'm seeing stuff like this in the IDM driver trace for a Null driver
while it's working:

Channel: Subscriber
Status: Retry
Message: Code(-9011) eDirectory returned an error indicating that
the operation should be retried later: novell.jclient.JCException:
getEffectivePrivileges -626 ERR_ALL_REFERRALS_FAILED

If I do a ndsrepair -T when I see that message then it says:

ERROR: Could not find a net address for this server - Error : -626

After a couple of retries everything is back to normal, then a couple of
minutes later I see the same message.

Has anybody seen this before?

Thanks.
-alekz

Labels (1)
23 Replies
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

On 2019-04-03 02:44, brucetimberlake wrote:
>
> brucetimberlake;2492947 Wrote:
>> We are also now seeing these ALL_REFERRALS_FAILED errors under IDM load
>> now, whereas with eDir 8/IDM 4.5 getting them was a sign of actual
>> issues with the environment. We are not seeing any replication delays,
>> and ndsrepair -E and -T are all clear. I plan to address it with our PSE
>> after the holidays and will post back any findings.

>
> A follow-up: we have not seen these messages in a while, which means
> that (a) the problem is somehow fixed (we're now running eDir 9.1.3 and
> IDM 4.7.1) or (b) we have not yet had sufficient load to trigger the
> problem. End of term is coming up in a month or so, which should cause
> a fair amount of load. I'll keep an eye on things and follow up again if
> anything interesting happens.
>
>

Keep us updated, I'm seeing this on another customer during load 😞

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

alekz;2492927 wrote:
On 2018-08-13 17:46, dgersic wrote:
>
> I had SR # 101165677501 open for a client with this same issue. We ended
> up abandoning the SR as unresolved, but the symptoms sound the same.
>
> I had it happening with a JDBC driver doing a resync all objects on the
> subscriber. I had it happening with a MAD driver on the publisher
> writing to a (large) object.
>
> We wandered around the "high value count" thing for a while. IMHO,
> that's not an answer, nor related to this actual issue.
>
>

I just created a new text driver that does a XML export and the -626 is
back when exporting thousands of objects. With just a few it works fine.

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.


That's interesting, but not surprising. It sounds similar to the symptoms I saw with the JDBC driver doing a resync on the subscriber.

Open an SR and reference mine. It has long since been closed, but having more reports of the same thing may help get somebody looking at it.
Highlighted
Captain
Captain

Hi Everyone,

we are also facing exactly the same issue with IDM 47 apps (this error is thrown in catalina.out) and on a loopback driver logs.

Can someone help the root cause of this issue and how this can be resolved.

Thanks in advance.

Regards,
dk
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

dkdng;2493048 wrote:
Hi Everyone,

we are also facing exactly the same issue with IDM 47 apps (this error is thrown in catalina.out) and on a loopback driver logs.

Can someone help the root cause of this issue and how this can be resolved.

Thanks in advance.

Regards,
dk


We here in the forums don't know what the cause is. Open an SR and let support know that you're also having this problem.
Highlighted
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Leaving here as reference. Saw this at a customer.

Loopback driver handling requests giving  Message: Code(-9006) The driver returned a "retry" status indicating that the operation should be retried later. Detail from driver: Code(-9011) eDirectory returned an error indicating that the operation should be retried later: novell.jclient.JCException: createEntry -626 ERR_ALL_REFERRALS_FAILED<application>DirXML</application>

This particular server was having issues on NCP because of another server's migration and port change from 524 to 1524. As soon as the port was opened, the events resumed.

Support mentioned that when a server goes into a bad address cache, NCP operations will fail which results in other operations not being allowed.

 

 

--
Smile, IT confuses people!
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

I opened a SR and got the question regarding the high value count as well :=)
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

It is good to know that Alekz has a tool to handle high values.

But the basic symptom being described there is issues syncing to much data/running out of resources

You can also check the thread count via ndstrace -c as the docs explain here:

https://www.netiq.com/documentation/edirectory-92/edir_tuning/data/bqmh9c0.html

 

We had the 626 error with running out of threads.  And have seen similar issues with high value counts (DirXML-EntitlementResult, i am looking at you!)

 

Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

Hello,

Still seeing this on eDirectory 9.2.2.

We have increased max threads to 512 but it doesn't help.

This are the threads settings in ndsconfig:

ndsconfig get | grep threads
n4u.server.max-threads=512
n4u.server.idle-threads=8
n4u.server.start-threads=24

Output from ndstrace -c threads when I've seen it happen:

Thread Pool Information
Summary : Spawned 1000, Died 935
Pool Workers : Idle 39, Total 65, Peak 101
Ready Work : Current 0, Peak 422, maxWait 1341246 us
Sched delay : Min 3 us, Max 59121403 us, Avg: 2 us
Waiting Work : Current 21, Peak 25

 

Highlighted
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

What does iMonitor tell you? Had this again today because of one server unwilling to replicate. Had to clear some obits and restart sync. Started working again after server caught up.

--
Smile, IT confuses people!
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

iMonitor shows no errors, replication is working fine.
Highlighted
Micro Focus Frequent Contributor
Micro Focus Frequent Contributor

Sorry, took a while to get back to this.
Have you checked the replica attribute on the partitions, that the server list and addresses are correct?
Also, check for a hosts.nds file, maybe there's wrong addresses in there.
--
Smile, IT confuses people!
Highlighted
Knowledge Partner Knowledge Partner
Knowledge Partner

Hi,

Have an SR open about this  at the moment.

It is not related to the things you mention, they are OK.

This only happens during heavy load at this particular customer.

 

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.