-626 ERR_ALL_REFERRALS_FAILED

[alekz]

Hello

Running eDirectory 9.0.4 v40006.34 on SLES11SP4.
Together with IDM 4.6.1

I'm seeing stuff like this in the IDM driver trace for a Null driver
while it's working:

Channel: Subscriber
Status: Retry
Message: Code(-9011) eDirectory returned an error indicating that
the operation should be retried later: novell.jclient.JCException:
getEffectivePrivileges -626 ERR_ALL_REFERRALS_FAILED

If I do a ndsrepair -T when I see that message then it says:

ERROR: Could not find a net address for this server - Error : -626

After a couple of retries everything is back to normal, then a couple of
minutes later I see the same message.

Has anybody seen this before?

Thanks.
-alekz

[aburgemeister]

Care to post the trace leading up to this?

Does this box hold real replicas o the whole tree as it probably should?
I presume so or the error would not be transient.

How is your IP address stuff setup for this box? I presume statically,
and not just via BootP or something but actually set statically within Yast.


--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

If you want to send me a private message, please let me know in the
forum as I do not use the web interface often.

[John_Walker]

If you are seeing intermittent -626 errors and an eDirectory server becomes slow or unresponsive during that time, it could likely be due to high valued attributes. If you suspect this to be the case, run a high valued attributes report in iMonitor. If any are reported at 20,000+ values, please contact support.

[alekz]

On 2018-04-03 00:04, John Walker wrote:
>
> If you are seeing intermittent -626 errors and an eDirectory server
> becomes slow or unresponsive during that time, it could likely be due to
> high valued attributes. If you suspect this to be the case, run a high
> valued attributes report in iMonitor. If any are reported at 20,000+
> values, please contact support.
>
>
Hello

Max values is 12 500 for a group.
I've checked the driver logs and they only show this error during a
period when I migrated ~8000 users through a driver. So I'll let it be
for now.

<status event-id="0" level="retry">Code(-9011) eDirectory returned an
error indicating that the operation should be retried later:
novell.jclient.JCException: getEffectivePrivileges -626
ERR_ALL_REFERRALS_FAILED</status>

[aburgemeister]

On 04/05/2018 06:32 AM, alekz wrote:
> On 2018-04-03 00:04, John Walker wrote:
>>
>> If you are seeing intermittent -626 errors and an eDirectory server
>> becomes slow or unresponsive during that time, it could likely be due to
>> high valued attributes. If you suspect this to be the case, run a high
>> valued attributes report in iMonitor. If any are reported at 20,000+
>> values, please contact support.

I've only ever seen a -625, not a -626, in this case. Maybe it can happen
either way.

> Max values is 12 500 for a group.
> I've checked the driver logs and they only show this error during a period
> when I migrated ~8000 users through a driver. So I'll let it be for now.
>
> <status event-id="0" level="retry">Code(-9011) eDirectory returned an
> error indicating that the operation should be retried later:
> novell.jclient.JCException: getEffectivePrivileges -626
> ERR_ALL_REFERRALS_FAILED</status>

Is it safe to assume that the server running the driver holds a writable
copy of the entire tree? Perhaps adding all replicas was forgotten.

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

If you want to send me a private message, please let me know in the
forum as I do not use the web interface often.

[alekz]

On 2018-04-05 15:38, ab wrote:
> On 04/05/2018 06:32 AM, alekz wrote:
>> On 2018-04-03 00:04, John Walker wrote:
>>>
>>> If you are seeing intermittent -626 errors and an eDirectory server
>>> becomes slow or unresponsive during that time, it could likely be due to
>>> high valued attributes. If you suspect this to be the case, run a high
>>> valued attributes report in iMonitor. If any are reported at 20,000+
>>> values, please contact support.
>
> I've only ever seen a -625, not a -626, in this case. Maybe it can happen
> either way.
>
>> Max values is 12 500 for a group.
>> I've checked the driver logs and they only show this error during a period
>> when I migrated ~8000 users through a driver. So I'll let it be for now.
>>
>> <status event-id="0" level="retry">Code(-9011) eDirectory returned an
>> error indicating that the operation should be retried later:
>> novell.jclient.JCException: getEffectivePrivileges -626
>> ERR_ALL_REFERRALS_FAILED</status>
>
> Is it safe to assume that the server running the driver holds a writable
> copy of the entire tree? Perhaps adding all replicas was forgotten.
>
Yes it's the master.
The DirXML status messages have the level="retry" so the events goes
through eventually.

[dsapanidis]

Has anyone figured out why this is happening?
I have the same, intermittent, error message when I'm running a scheduled job in Null driver on ~130K objects.
The job reads a couple of single valued attributes from an object and writes a single valued status if required.

When eDir 9.0.4 is restarted the scheduled job runs without errors. This particular job runs every night and after 2 - 4 days the error appears again.

[dgersic]

I had SR # 101165677501 open for a client with this same issue. We ended up abandoning the SR as unresolved, but the symptoms sound the same.

I had it happening with a JDBC driver doing a resync all objects on the subscriber. I had it happening with a MAD driver on the publisher writing to a (large) object.

We wandered around the "high value count" thing for a while. IMHO, that's not an answer, nor related to this actual issue.

[alekz]

On 2018-08-13 17:46, dgersic wrote:
>
> I had SR # 101165677501 open for a client with this same issue. We ended
> up abandoning the SR as unresolved, but the symptoms sound the same.
>
> I had it happening with a JDBC driver doing a resync all objects on the
> subscriber. I had it happening with a MAD driver on the publisher
> writing to a (large) object.
>
> We wandered around the "high value count" thing for a while. IMHO,
> that's not an answer, nor related to this actual issue.
>
>
I just created a new text driver that does a XML export and the -626 is
back when exporting thousands of objects. With just a few it works fine.

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

[brucetimberlake]

We are also now seeing these ALL_REFERRALS_FAILED errors under IDM load now, whereas with eDir 8/IDM 4.5 getting them was a sign of actual issues with the environment. We are not seeing any replication delays, and ndsrepair -E and -T are all clear. I plan to address it with our PSE after the holidays and will post back any findings.

[dgersic]

brucetimberlake;2492947 wrote:
We are also now seeing these ALL_REFERRALS_FAILED errors under IDM load now, whereas with eDir 8/IDM 4.5 getting them was a sign of actual issues with the environment. We are not seeing any replication delays, and ndsrepair -E and -T are all clear. I plan to address it with our PSE after the holidays and will post back any findings.

Interesting. Get an SR open and get your PSE working on it. I'm no longer working with the client where I saw this, but I'm still curious to see what happens with this.

[brucetimberlake]

brucetimberlake;2492947 wrote:
We are also now seeing these ALL_REFERRALS_FAILED errors under IDM load now, whereas with eDir 8/IDM 4.5 getting them was a sign of actual issues with the environment. We are not seeing any replication delays, and ndsrepair -E and -T are all clear. I plan to address it with our PSE after the holidays and will post back any findings.

A follow-up: we have not seen these messages in a while, which means that (a) the problem is somehow fixed (we're now running eDir 9.1.3 and IDM 4.7.1) or (b) we have not yet had sufficient load to trigger the problem. End of term is coming up in a month or so, which should cause a fair amount of load. I'll keep an eye on things and follow up again if anything interesting happens.

[alekz]

On 2019-04-03 02:44, brucetimberlake wrote:
>
> brucetimberlake;2492947 Wrote:
>> We are also now seeing these ALL_REFERRALS_FAILED errors under IDM load
>> now, whereas with eDir 8/IDM 4.5 getting them was a sign of actual
>> issues with the environment. We are not seeing any replication delays,
>> and ndsrepair -E and -T are all clear. I plan to address it with our PSE
>> after the holidays and will post back any findings.
>
> A follow-up: we have not seen these messages in a while, which means
> that (a) the problem is somehow fixed (we're now running eDir 9.1.3 and
> IDM 4.7.1) or (b) we have not yet had sufficient load to trigger the
> problem. End of term is coming up in a month or so, which should cause
> a fair amount of load. I'll keep an eye on things and follow up again if
> anything interesting happens.
>
>
Keep us updated, I'm seeing this on another customer during load 😞

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

[dgersic]

alekz;2492927 wrote:
On 2018-08-13 17:46, dgersic wrote:
>
> I had SR # 101165677501 open for a client with this same issue. We ended
> up abandoning the SR as unresolved, but the symptoms sound the same.
>
> I had it happening with a JDBC driver doing a resync all objects on the
> subscriber. I had it happening with a MAD driver on the publisher
> writing to a (large) object.
>
> We wandered around the "high value count" thing for a while. IMHO,
> that's not an answer, nor related to this actual issue.
>
>
I just created a new text driver that does a XML export and the -626 is
back when exporting thousands of objects. With just a few it works fine.

--
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

That's interesting, but not surprising. It sounds similar to the symptoms I saw with the JDBC driver doing a resync on the subscriber.

Open an SR and reference mine. It has long since been closed, but having more reports of the same thing may help get somebody looking at it.

[dkdng]

Hi Everyone,

we are also facing exactly the same issue with IDM 47 apps (this error is thrown in catalina.out) and on a loopback driver logs.

Can someone help the root cause of this issue and how this can be resolved.

Thanks in advance.

Regards,
dk