ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins. Read more for important details.
ALERT! The community will be read-only starting on April 19, 8am Pacific as the migration begins.Read more for important details.
Commander
Commander
2307 views

LDAP failure

Randomly ldap and ndsd will crash on our SuSe server
when I try to restart ldap '/opt/novell/eDirectory/sbin/nldap -u' '-l'
I get the message "failed to start / stop / restart ldap"
trying to reboot ndsd hangs up
I have to kill the ndsd process or it will hang the server on reboot
ultimately having to reboot the server

in the message log this is what it is showing:
Mar 15 01:00:00 server-name kernel: [6522184.893958] NSS: BGComp next start timer will go off in 86400 seconds at 00:00:00
Mar 15 01:00:00 server-name kernel: [6522184.893969] NSS: BGComp scanning started by background timer (nss /CompressionDailyCheckStartingHour).
Mar 15 01:00:00 server-name kernel: [6522184.893974] NSS: BGComp scanning completed.
Mar 15 01:00:00 server-name kernel: [6522184.893975] NSS: BGComp queued requests now complete.
Mar 15 01:00:01 server-name /usr/sbin/cron[28619]: (root) CMD (/etc/init.d/watchdog -e >/dev/null 2>&1)
Mar 15 01:00:01 server-name /usr/sbin/cron[28621]: (root) CMD (/opt/novell/eDirectory/bin/ndsrepair -U)
Mar 15 01:00:05 server-name sshd[28682]: Connection closed by 127.0.0.1 [preauth]
Mar 15 01:00:06 server-name nsmeventd[3098]: EM: Unhandled error in poll loop: basic_string::_S_construct null not valid
Mar 15 01:00:06 server-name nsmeventd[3098]: EM: Unhandled error: Unhandled exception requiring restart of event monitor function.
Mar 15 01:00:06 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81) Can't contact LDAP server
Mar 15 01:00:06 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81) Can't contact LDAP server
Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]: GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while trying to find group FDNs with scope=base for cn=UNIX Workstation - server-name,o=sd
Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]: GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while trying to find group FDNs with scope=base for cn=UNIX Workstation - server-name,o=sd
Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]: GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while trying to find group FDNs with scope=base for cn=UNIX Workstation - server-name,o=sd
Mar 15 01:00:09 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81) Can't contact LDAP server
Mar 15 01:00:09 server-name volmnd[5092]: Pinging EFLs
Mar 15 01:00:11 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81) Can't contact LDAP server
Mar 15 01:00:14 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81) Can't contact LDAP server

LSB_VERSION="core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64"
Novell Open Enterprise Server 2015 (x86_64)
VERSION = 2015.1
PATCHLEVEL = 1
NAME="SLES"
VERSION="11.4"
VERSION_ID="11.4"
PRETTY_NAME="SUSE Linux Enterprise Server 11 SP4"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:11:4"
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 4

NetIQ eDirectory 8.8 SP8 (20812.10)


var/opt/novell/edirectory/log

Mar 15 01:00:02 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
ndsd: ../../../../apache-log4cxx-0.10.0/src/main/cpp/aprinitializer.cpp:40: log4cxx::helpers::APRInitializer::APRInitializer(): Assertion `stat == 0' failed.
Labels (1)
Tags (3)
0 Likes
9 Replies
Knowledge Partner Knowledge Partner
Knowledge Partner

On 03/15/2018 11:34 AM, mcclary wrote:
>
> Randomly ldap and ndsd will crash on our SuSe server
> when I try to restart ldap '/opt/novell/eDirectory/sbin/nldap -u' '-l'
> I get the message "failed to start / stop / restart ldap"
> trying to reboot ndsd hangs up
> I have to kill the ndsd process or it will hang the server on reboot
> ultimately having to reboot the server


How long has this been going on? Any recent upgrades/updates or other
configuration changes on the eDirectory or OES side of things?

How old is this OES box?

OES is interesting because the OES pieces like NSS depend on eDirectory
working in order to get user information, trustees, etc. "Local" users
like 'admin' also reside in eDirectory, so in order for NSS to work LUM
(namcd) needs to work, and it works by connecting to eDirectory, sometimes
locally but often enough pointing to some external system at least as a
backup.

> in the message log this is what it is showing:
> Mar 15 01:00:00 server-name kernel: [6522184.893958] NSS: BGComp next
> start timer will go off in 86400 seconds at 00:00:00
> Mar 15 01:00:00 server-name kernel: [6522184.893969] NSS: BGComp
> scanning started by background timer (nss
> /CompressionDailyCheckStartingHour).
> Mar 15 01:00:00 server-name kernel: [6522184.893974] NSS: BGComp
> scanning completed.
> Mar 15 01:00:00 server-name kernel: [6522184.893975] NSS: BGComp queued
> requests now complete.
> Mar 15 01:00:01 server-name /usr/sbin/cron[28619]: (root) CMD
> (/etc/init.d/watchdog -e >/dev/null 2>&1)
> Mar 15 01:00:01 server-name /usr/sbin/cron[28621]: (root) CMD
> (/opt/novell/eDirectory/bin/ndsrepair -U)
> Mar 15 01:00:05 server-name sshd[28682]: Connection closed by 127.0.0.1
> [preauth]
> Mar 15 01:00:06 server-name nsmeventd[3098]: EM: Unhandled error in poll
> loop: basic_string::_S_construct null not valid
> Mar 15 01:00:06 server-name nsmeventd[3098]: EM: Unhandled error:
> Unhandled exception requiring restart of event monitor function.
> Mar 15 01:00:06 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81)
> Can't contact LDAP server
> Mar 15 01:00:06 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81)
> Can't contact LDAP server


What is this nsmeventd service, out of curiosity? Do you have something
like Storage Manager on this machine? It should not matter for eDirectory
starting, but it apparently wants to us LDAP and, as you reported, the
LDAP interface is not up yet.

> Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]:
> GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while
> trying to find group FDNs with scope=base for cn=UNIX Workstation -
> server-name,o=sd
> Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]:
> GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while
> trying to find group FDNs with scope=base for cn=UNIX Workstation -
> server-name,o=sd
> Mar 15 01:00:06 server-name /usr/sbin/namcd[4513]:
> GetGIDsGroupListNumberOfGroupsOfWS: Error [81] in LDAP search while
> trying to find group FDNs with scope=base for cn=UNIX Workstation -
> server-name,o=sd


Does this particular Unix Workstation object exist in eDirectory at the DN
specified above? LUM/namcd wants it. If it does, but namcd (via
/etc/nam.conf) is configured to point to the local service and the local
LDAP service is not up, that may explain the inability to find it.

> Mar 15 01:00:09 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81)
> Can't contact LDAP server
> Mar 15 01:00:09 server-name volmnd[5092]: Pinging EFLs
> Mar 15 01:00:11 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81)
> Can't contact LDAP server
> Mar 15 01:00:14 server-name nsmeventd[3098]: EM: ldapssl_start_tls: (81)
> Can't contact LDAP server


More Storage Manager looking stuff.

> LSB_VERSION="core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64"
> Novell Open Enterprise Server 2015 (x86_64)
> VERSION = 2015.1
> PATCHLEVEL = 1
> NAME="SLES"
> VERSION="11.4"
> VERSION_ID="11.4"
> PRETTY_NAME="SUSE Linux Enterprise Server 11 SP4"
> ID="sles"
> ANSI_COLOR="0;32"
> CPE_NAME="cpe:/o:suse:sles:11:4"
> SUSE Linux Enterprise Server 11 (x86_64)
> VERSION = 11
> PATCHLEVEL = 4
>
> NetIQ eDirectory 8.8 SP8 (20812.10)
>
> var/opt/novell/edirectory/log
>
> Mar 15 01:00:02 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8
> v20811.03 Successfully loaded
> ndsd:
> ../../../../apache-log4cxx-0.10.0/src/main/cpp/aprinitializer.cpp:40:
> log4cxx::helpers::APRInitializer::APRInitializer(): Assertion `stat ==
> 0' failed.


Are these the only two lines in that whole directory of files, or even in
ndsd.log? I would expect a lot more, and it may be useful to even see
successes from before the current problem happened to see what "normal"
looks like for this server.

Can you, if you give it a bit of time, get eDirectory to load at all? If
not, we should probably ignore LDAP and focus on that.

Did this server hold replicas of any/all partitions before things started
going haywire?

If eDirectory starts and it is just the LDAP bits that do not work, you
can watch those startup using the following commands as 'root':


ldapconfig set 'LDAP Screen Level=all'
ndstrace

set dstrace=nodebug
dstrace +time +tags +ldap +init
dstrace file on
set dstrace=*r
unload nldap
load nldap

#wait for a few seconds

dstrace file off
quit


Please post the resulting /var/opt/novell/eDirectory/log/ndstrace.log file
here for review.

--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

If you want to send me a private message, please let me know in the
forum as I do not use the web interface often.
0 Likes
Commander
Commander

How long has this been going on? Any recent upgrades/updates or other
configuration changes on the eDirectory or OES side of things?

This has been going on, on and off for about a year now.. will happen super randomly, no recent upgrades / updates or other configuration changes
NDSREPAIR -U comes back with no errors


How old is this OES box?
The VM is about 3 years old

What is this nsmeventd service, out of curiosity? Do you have something
like Storage Manager on this machine? It should not matter for eDirectory
starting, but it apparently wants to us LDAP and, as you reported, the
LDAP interface is not up yet.

I am not positive waht nsmeventd service is, I am fairly new to this all.

Does this particular Unix Workstation object exist in eDirectory at the DN
specified above? LUM/namcd wants it. If it does, but namcd (via
/etc/nam.conf) is configured to point to the local service and the local
LDAP service is not up, that may explain the inability to find it.

Yes the object does exist



Are these the only two lines in that whole directory of files, or even in
ndsd.log? I would expect a lot more, and it may be useful to even see
successes from before the current problem happened to see what "normal"
looks like for this server.

No here are the lines from the log from the last couple months
Dec 29 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Dec 29 09:36:01 About to stop NetIQ eDirectory server on host: wgsd-master
Dec 29 09:36:01 Got SIGINT/SIGTERM signal, server shutting down
Dec 29 09:36:03 Unloading SSNCP...
Dec 29 09:36:03 SecretStore NCP Plugin Unloaded Successfully
Dec 29 09:36:03 NetIQ PKI Services unloaded
Dec 29 09:36:03 SPM DClient closed
Dec 29 09:36:07 NDS iMonitor for NetIQ eDirectory 8.8 SP8 v20811.01 shutdown successfully.
Dec 29 09:36:12 Unloading SSLDP...
Dec 29 09:36:12 SecretStore LDAP Plugin Unloaded Successfully
Dec 29 09:36:12 SecretStore LDAP Extension Handler Unloaded
Dec 29 09:36:12 LDAP Agent for NetIQ eDirectory 8.8 SP8 (20811.09) stopped
Dec 29 09:36:12 Information: SNMP Trap Server for NetIQ eDirectory 8.8.8 v20810.18 stopped.

Dec 29 09:36:12 GAMS closed
Dec 29 09:36:15 MASV closed
[ -- DHost Logging STOPPED Fri Dec 29 09:37:56 2017 -- ]
Dec 29 09:37:56 Shutdown NCPServer
Dec 29 09:37:56 Shutdown NCPServer ... beginning check for packets in process
Dec 29 09:37:59 DSDeregisterSignalHandler failed. err=-5997.
Dec 29 09:37:59 ... NCPServer halted
Dec 29 09:37:59 SASL closed
Dec 29 09:36:01 Stopped NetIQ eDirectory server on host: wgsd-master .
Dec 29 09:40:33 About to start NetIQ eDirectory server on host: wgsd-master.
Dec 29 09:40:34 Path of NetIQ eDirectory configuration file /etc/opt/novell/eDirectory/conf/nds.conf
Dec 29 09:40:35 NCPServer: Have 1 interfaces in conf file
Dec 29 09:40:35 Init NCPServer
Dec 29 09:40:35 DSRegisterSignalHandler failed. err=-5997.
Dec 29 09:40:35 Host process for NetIQ eDirectory 8.8 SP8 v20812.20 successfully started
Dec 29 09:40:35 DHLog: file size 1048576
[ -- DHost Logging STARTED Fri Dec 29 09:40:35 2017 -- ]
Dec 29 09:40:35 MASV Init called
Dec 29 09:40:35 Mandatory Access Control Service Version: 8.8.8.10 started
Dec 29 09:40:35 NMAS Server Version:8.8.8.10 Build:20170328 started
Dec 29 09:40:35 SPM DClient Version:8.8.8.10 Build:20170328 started
Dec 29 09:40:35 MASV Init called
Dec 29 09:40:35 MASV already initialized.
Dec 29 09:40:35 Opening NCPServer
Dec 29 09:40:35 NCP Server name wgsd-master
Dec 29 09:40:40 volume SYS has matching attribute
Dec 29 09:40:40 volume NSMPROXYHOME has matching attribute
Dec 29 09:40:40 CertMutual Method LSM Version:2837 Build:20131206 loaded
Dec 29 09:40:40 CertMutual Method Proxy LCM Version:2837 Build:20131206 loaded
Dec 29 09:40:41 Simple Password Method LSM Version:2837 Build:20131206 loaded
Dec 29 09:40:41 Simple Password Method Proxy LCM Version:2837 Build:20131206 loaded
Dec 29 09:40:41 Digest MD5 LSM Version:2837 Build:20131206 loaded
Dec 29 09:40:41 Digest MD5 Proxy LCM Version:2837 Build:20131206 loaded
Dec 29 09:40:41 GAMS Init called
Dec 29 09:40:41 Graded Authentication Management Service Version: 8.8.8.10 started
Dec 29 09:40:41 Information: SNMP Trap Server for NetIQ eDirectory 8.8.8 v20810.18 started.

Dec 29 09:40:41 NDS iMonitor for NetIQ eDirectory 8.8 SP8 v20812.04 started successfully.
Dec 29 09:40:41 NMAS Server Version:8.8.8.10 Build:20170328 started
Dec 29 09:40:41 SPM DClient already started (2)
Dec 29 09:40:41 Loading SecretStore Server...
Dec 29 09:40:41 NetIQ SecretStore Service Version 8.8.8.10 Loaded Successfully
Dec 29 09:40:42 NetIQ PKI Services Started Successfully
Dec 29 09:40:42 PKIHealth.log in directory: /var/opt/novell/eDirectory/log/
Dec 29 09:40:42 Loading SecretStore LDAP Transport Plugin...
Dec 29 09:40:42 NetIQ SecretStore LDAP Plugin Version 8.8.8.10 Loaded Successfully.
Dec 29 09:40:42 SecretStore LDAP Extension Handler Loaded Successfully
Dec 29 09:40:42 LDAP Agent for NetIQ eDirectory 8.8 SP8 (20812.10) started
Dec 29 09:40:42 SASL Version:8.8.8.10 Build:20170328 started
We are in DHost.
Dec 29 09:40:33 Started NetIQ eDirectory server on host: wgsd-master
NetIQ JClient 2.08.1200-2.8.1200. (c) 2013 NetIQ Corporation and its affiliates. All Rights Reserved.
Dec 29 09:41:00 Loading SecretStore NCP Transport Plugin...
Dec 29 09:41:00 NetIQ SecretStore NCP Plugin Version 8.8.8.10 Loaded Successfully.
Dec 30 01:00:02 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Dec 30 01:00:41 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Dec 31 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Dec 31 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 01 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 01 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 02 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 02 01:00:38 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 03 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 03 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 04 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 04 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 05 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 05 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 06 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 06 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 07 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 07 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 08 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 08 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 09 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 09 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 10 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 10 01:00:45 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 11 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 11 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 12 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 12 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 13 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 13 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 14 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 14 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 15 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 15 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 16 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 16 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 17 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 17 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 18 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 18 01:00:38 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 19 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 19 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 20 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 20 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 21 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 21 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 22 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 22 01:00:46 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 23 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 23 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 24 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 24 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 25 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 25 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 26 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 26 01:00:45 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 27 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 27 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 28 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 28 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 29 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 29 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 30 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 30 01:00:45 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Jan 31 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Jan 31 01:00:32 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 01 01:00:02 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 01 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 02 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 02 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 03 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 03 01:00:41 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 04 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 04 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 05 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 05 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 06 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 06 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 07 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 07 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 08 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 08 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 09 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 09 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 10 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 10 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 11 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 11 01:00:48 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 12 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 12 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 13 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 13 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 14 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 14 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 15 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 15 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 16 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 16 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 17 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 17 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 18 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 18 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 19 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 19 01:00:38 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 20 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 20 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 21 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 21 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 22 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 22 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 23 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 23 01:00:48 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 24 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 24 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 25 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 25 01:00:37 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 26 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 26 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 27 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 27 01:00:41 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Feb 28 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Feb 28 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 01 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 01 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 02 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 02 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 03 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 03 01:00:49 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 04 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 04 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 05 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 05 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 06 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 06 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 07 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 07 01:00:47 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 08 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 08 01:00:38 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 09 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 09 01:00:34 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 10 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 10 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 11 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 11 01:00:47 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 12 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 12 01:00:33 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 13 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 13 01:00:36 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 14 01:00:01 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
Mar 14 01:00:35 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully unloaded
Mar 15 01:00:02 Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03 Successfully loaded
ndsd: ../../../../apache-log4cxx-0.10.0/src/main/cpp/aprinitializer.cpp:40: log4cxx::helpers::APRInitializer::APRInitializer(): Assertion `stat == 0' failed.



Can you, if you give it a bit of time, get eDirectory to load at all? If
not, we should probably ignore LDAP and focus on that.
edirectory will not load at all, we have to kill the process as it does not stop / start / restart the terminal will just hand so we have to kill the process. If we do not kill the process we on reboot the VM will hang and we have to remote on the esx host to reboot the VM

Did this server hold replicas of any/all partitions before things started
going haywire?

Yes it has held all replicas since day 1
0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

K, let's get the ndstrace output, but in two ways; first, let's see which
modules are loaded right now:


ndstrace -c modules


Next, run the commands from my last post and then put the output in the
ndstrace.log file here. For simplicity, here they are again:


ldapconfig set 'LDAP Screen Level=all'
ndstrace

set dstrace=nodebug
dstrace +time +tags +ldap +init
dstrace file on
set dstrace=*r
unload nldap
load nldap

#wait for a few seconds

dstrace file off
quit


Please post the resulting /var/opt/novell/eDirectory/log/ndstrace.log file
here for review.


--
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below.

If you want to send me a private message, please let me know in the
forum as I do not use the web interface often.
0 Likes
Commander
Commander

:~ # ndstrace -c modules

[1] Instance at /etc/opt/novell/eDirectory/conf/nds.conf: wgsd-master.O=SD.WGSD-TREE

dsr Not Loaded
xdasauditds Not Loaded
snmpinst Not Loaded
repair Not Loaded
pkiinst Not Loaded
password-plugin Not Loaded
nmasldap Running
nmasinst Not Loaded
ndsinfo Not Loaded
ndsclone Not Loaded
ncpns Not Loaded
merge Not Loaded
lsss Running
ldapxs Running
lburp Running
krbpwd Not Loaded
jvmload Not Loaded
dxldap Running
dxevent Running
dstrace Not Loaded
dsi Not Loaded
dsbk Not Loaded
backupcr Running
vrdim Running
ssncp Running
pkiserver Running
sasl Running [ nldap ]
sss Running [ ssldp ssncp ]
ssldp Running [ nldap ]
spmdclnt Running
nmas Running [ sasl ]
embox Running
imon Running
nldap Running
hconserv Running
snmp Running SNMP Trap Server for NetIQ eDirectory 8.8.8
gams Running
niciext Running
httpstk Running [ nds hconserv imon embox ]
nds Running
masv Running [ gams ]
dsloader Running [ httpstk hconserv nldap ]
dhlog Running
ncpengine Running
ndsd Running NetIQ eDirectory 8.8 SP8 Host Environment

Working on figuring out User FDN after I enter in "ldapconfig set 'LDAP Screen Level=all'" it asks me for user fdn
0 Likes
Commander
Commander

We are planning on patching these servers over spring break since eDirectory 8.8 SP8 Patch 11 instrumentation for OES2015 SP1 came available yesterday
https://www.novell.com/support/kb/doc.php?id=3426981
0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

why do you schedule an unattended full repair every night? the error messages which get thrown by storage manager fall (at least in the log provided) in the timeframe the repair runs, so they don't surprise too much.
could you post the repair log?
are you running some sort of virus scanner?
how big is the DIB?
how much free diskspace do you have?
would you mind to retain from the scheduled repairs (as you shouldn't do these anyway)?
If you like it: like it.
0 Likes
Commander
Commander

why do you schedule an unattended full repair every night? the error messages which get thrown by storage manager fall (at least in the log provided) in the timeframe the repair runs, so they don't surprise too much.
This is how it was set up when I came into the environment
could you post the repair log?
/****************************************************************************/
Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03, DS 20812.20.

** Automated Repair Mode **
Repairing Local Database
Start: Thursday, March 15, 2018 01:00:02 Local Time


/****************************************************************************/
Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03, DS 20812.20.
Start: Thursday, March 15, 2018 13:30:04 Local Time
Retrieve replica status

Partition: .[Root].
Replica: .wgsd-master.SD 03-15-2018 13:30:01
Replica: .iprint.SD 03-15-2018 13:29:58
Replica: .WGSD-APPS.SD 03-15-2018 13:30:02
Replica: .WGSD-LOGIN.SD 03-15-2018 13:29:41
All servers synchronized up to time: 03-15-2018 13:29:41

/****************************************************************************/
Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03, DS 20812.20.

** Automated Repair Mode **
Repairing Local Database
Start: Thursday, March 15, 2018 13:33:01 Local Time

** All disk amounts are approximations **
Disk space currently available: 5376 MB
->DSRepair may need to use: 313 MB
->Disk space remaining after operation: 5376 MB

Current transaction ID is 422567001 (0x192fdc59). Allowed limit of transaction is 4294959104 (0xffffe000)
Creating Temporary Files

Physical Check

Repair Trees - Scan Values

Repair Trees - Sorting Values

Repair Trees - Scan Entries

Repair Trees - Sorting Entries

Repair Trees - Check Values

Total Objects in Database: 16466
Total Objects in Schema : 3775
Total External References: 1
Total Objects in Replicas: 12686
Schema Check


Repairing objects in a replica
Start: Thursday, March 15, 2018 13:33:11 Local Time

Total objects in partition - T=WGSD-TREE : 10609
Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(3000)
Repairing objects - done(4000)
Repairing objects - done(5000)
Repairing objects - done(6000)
Repairing objects - done(7000)
Repairing objects - done(8000)
Repairing objects - done(9000)
Repairing objects - done(10000)

ERROR: Modification time was incorrect, it has been updated
Object ID: 0000b0b3, DN: CN=NSMProxy.O=SD.T=WGSD-TREE

Repairing objects - done(10609)

Total Objects = 10609, UNKNOWN class objects = 0, Total Values = 241238

Total objects in partition - CN=driverset.O=SD.T=WGSD-TREE : 23

Repairing objects - done(23)

Total Objects = 23, UNKNOWN class objects = 0, Total Values = 479

Total objects in partition - OU=Students.O=SD.T=WGSD-TREE : 2054

Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(2054)

Total Objects = 2054, UNKNOWN class objects = 0, Total Values = 98323
Repairing single object:

Object ID: 00008013, [Pseudo Server]

Total Objects = 1, UNKNOWN class objects = 0, Total Values = 110
Temporary DIB set replacing NDS working DIB set.

Checking stream syntax files
Repair process completed, total errors found = 1

** Automated Repair Mode **
Repairing Server Network Addresses
Start: Thursday, March 15, 2018 13:33:37 Local Time
****************************************************************
This operation searches IPX, SLP, and DNS tables, if
available, to validate network address attributes of
NCP_SERVERS, and the referrals on replica objects.
If the information on DNS/DHCP tables or the /etc/hosts file is
incorrect, then we can neither guarantee proper validation of network
addresses, nor replica referral updates. This is particularly true
if the system uses unregistered DHCP addresses or the information
in the HOSTS file is incorrect or outdated
****************************************************************


Checking server: .wgsd-mail.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524

Checking server: .iprint.SD
Checking server address in Replica ID : 4, .[Root].
Checking server address in Replica ID : 4, .driverset.SD
Checking server address in Replica ID : 4, .Students.SD

Checking server: .WGSD-APPS.SD
Checking server address in Replica ID : 2, .[Root].
Checking server address in Replica ID : 2, .driverset.SD
Checking server address in Replica ID : 2, .Students.SD

Checking server: .WGSD-LOGIN.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 1, .[Root].
Checking server address in Replica ID : 1, .driverset.SD
Checking server address in Replica ID : 1, .Students.SD

Checking server: .wgsd-master.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 5, .[Root].
Checking server address in Replica ID : 5, .driverset.SD
Checking server address in Replica ID : 5, .Students.SD

** Automated Repair Mode **
Repairing replica ring
Start: Thursday, March 15, 2018 13:33:37 Local Time

Replica Ring for replica: .[Root].
Remote server's local ID: 00008040
Remote server's replica root ID: 00008042
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000803f
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008010
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008019
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .driverset.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 000195c6
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 00019595
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 0000c88c
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 0000b2ae
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .Students.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 00008126
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000811b
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008194
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008267
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

** Automated Repair Mode **
Finish: Thursday, March 15, 2018 13:33:37 Local Time
Total repair time: 0:00:36

/****************************************************************************/
Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03, DS 20812.20.

** Automated Repair Mode **
Repairing Local Database
Start: Thursday, March 15, 2018 13:34:03 Local Time

** All disk amounts are approximations **
Disk space currently available: 5316 MB
->DSRepair may need to use: 313 MB
->Disk space remaining after operation: 5316 MB

Current transaction ID is 422567262 (0x192fdd5e). Allowed limit of transaction is 4294959104 (0xffffe000)
Creating Temporary Files

Physical Check

Repair Trees - Scan Values

Repair Trees - Sorting Values

Repair Trees - Scan Entries

Repair Trees - Sorting Entries

Repair Trees - Check Values

Total Objects in Database: 16466
Total Objects in Schema : 3775
Total External References: 1
Total Objects in Replicas: 12686
Schema Check


Repairing objects in a replica
Start: Thursday, March 15, 2018 13:34:12 Local Time

Total objects in partition - T=WGSD-TREE : 10609
Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(3000)
Repairing objects - done(4000)
Repairing objects - done(5000)
Repairing objects - done(6000)
Repairing objects - done(7000)
Repairing objects - done(8000)
Repairing objects - done(9000)
Repairing objects - done(10000)
Repairing objects - done(10609)

Total Objects = 10609, UNKNOWN class objects = 0, Total Values = 241356

Total objects in partition - CN=driverset.O=SD.T=WGSD-TREE : 23

Repairing objects - done(23)

Total Objects = 23, UNKNOWN class objects = 0, Total Values = 486

Total objects in partition - OU=Students.O=SD.T=WGSD-TREE : 2054

Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(2054)

Total Objects = 2054, UNKNOWN class objects = 0, Total Values = 98325
Repairing single object:

Object ID: 00008013, [Pseudo Server]

Total Objects = 1, UNKNOWN class objects = 0, Total Values = 111
Temporary DIB set replacing NDS working DIB set.

Checking stream syntax files
Repair process completed, total errors found = 0

** Automated Repair Mode **
Repairing Server Network Addresses
Start: Thursday, March 15, 2018 13:34:40 Local Time
****************************************************************
This operation searches IPX, SLP, and DNS tables, if
available, to validate network address attributes of
NCP_SERVERS, and the referrals on replica objects.
If the information on DNS/DHCP tables or the /etc/hosts file is
incorrect, then we can neither guarantee proper validation of network
addresses, nor replica referral updates. This is particularly true
if the system uses unregistered DHCP addresses or the information
in the HOSTS file is incorrect or outdated
****************************************************************


Checking server: .wgsd-mail.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524

Checking server: .iprint.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 4, .[Root].
Checking server address in Replica ID : 4, .driverset.SD
Checking server address in Replica ID : 4, .Students.SD

Checking server: .WGSD-APPS.SD
Checking server address in Replica ID : 2, .[Root].
Checking server address in Replica ID : 2, .driverset.SD
Checking server address in Replica ID : 2, .Students.SD

Checking server: .WGSD-LOGIN.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 1, .[Root].
Checking server address in Replica ID : 1, .driverset.SD
Checking server address in Replica ID : 1, .Students.SD

Checking server: .wgsd-master.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 5, .[Root].
Checking server address in Replica ID : 5, .driverset.SD
Checking server address in Replica ID : 5, .Students.SD

** Automated Repair Mode **
Repairing replica ring
Start: Thursday, March 15, 2018 13:34:40 Local Time

Replica Ring for replica: .[Root].
Remote server's local ID: 00008040
Remote server's replica root ID: 00008042
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000803f
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008010
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008019
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .driverset.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 000195c6
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 00019595
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 0000c88c
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 0000b2ae
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .Students.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 00008126
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000811b
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008194
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008267
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

** Automated Repair Mode **
Finish: Thursday, March 15, 2018 13:34:40 Local Time
Total repair time: 0:00:37

/****************************************************************************/
Repair utility for NetIQ eDirectory 8.8 - 8.8 SP8 v20811.03, DS 20812.20.

** Automated Repair Mode **
Repairing Local Database
Start: Friday, March 16, 2018 01:00:01 Local Time

** All disk amounts are approximations **
Disk space currently available: 5786 MB
->DSRepair may need to use: 313 MB
->Disk space remaining after operation: 5786 MB

Current transaction ID is 422632359 (0x1930dba7). Allowed limit of transaction is 4294959104 (0xffffe000)
Creating Temporary Files

Physical Check

Repair Trees - Scan Values

Repair Trees - Sorting Values

Repair Trees - Scan Entries

Repair Trees - Sorting Entries

Repair Trees - Check Values

Total Objects in Database: 16465
Total Objects in Schema : 3775
Total External References: 1
Total Objects in Replicas: 12686
Schema Check


Repairing objects in a replica
Start: Friday, March 16, 2018 01:00:07 Local Time

Total objects in partition - T=WGSD-TREE : 10609
Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(3000)
Repairing objects - done(4000)
Repairing objects - done(5000)
Repairing objects - done(6000)
Repairing objects - done(7000)
Repairing objects - done(8000)
Repairing objects - done(9000)
Repairing objects - done(10000)
Repairing objects - done(10609)

Total Objects = 10609, UNKNOWN class objects = 0, Total Values = 241334

Total objects in partition - CN=driverset.O=SD.T=WGSD-TREE : 23

Repairing objects - done(23)

Total Objects = 23, UNKNOWN class objects = 0, Total Values = 479

Total objects in partition - OU=Students.O=SD.T=WGSD-TREE : 2054

Repairing objects - done(1000)
Repairing objects - done(2000)
Repairing objects - done(2054)

Total Objects = 2054, UNKNOWN class objects = 0, Total Values = 98297
Repairing single object:

Object ID: 00008013, [Pseudo Server]

Total Objects = 1, UNKNOWN class objects = 0, Total Values = 112
Temporary DIB set replacing NDS working DIB set.

Checking stream syntax files
Repair process completed, total errors found = 0

** Automated Repair Mode **
Repairing Server Network Addresses
Start: Friday, March 16, 2018 01:00:33 Local Time
****************************************************************
This operation searches IPX, SLP, and DNS tables, if
available, to validate network address attributes of
NCP_SERVERS, and the referrals on replica objects.
If the information on DNS/DHCP tables or the /etc/hosts file is
incorrect, then we can neither guarantee proper validation of network
addresses, nor replica referral updates. This is particularly true
if the system uses unregistered DHCP addresses or the information
in the HOSTS file is incorrect or outdated
****************************************************************


Checking server: .wgsd-mail.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524

Checking server: .iprint.SD
Checking server address in Replica ID : 4, .[Root].
Checking server address in Replica ID : 4, .driverset.SD
Checking server address in Replica ID : 4, .Students.SD

Checking server: .WGSD-APPS.SD
Checking server address in Replica ID : 2, .[Root].
Checking server address in Replica ID : 2, .driverset.SD
Checking server address in Replica ID : 2, .Students.SD

Checking server: .WGSD-LOGIN.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 1, .[Root].
Checking server address in Replica ID : 1, .driverset.SD
Checking server address in Replica ID : 1, .Students.SD

Checking server: .wgsd-master.SD
Address Type = TCP, data[6] = xxx.xxx.xxx.xxx:524
Address Type = UDP, data[6] = xxx.xxx.xxx.xxx:524
Checking server address in Replica ID : 5, .[Root].
Checking server address in Replica ID : 5, .driverset.SD
Checking server address in Replica ID : 5, .Students.SD

** Automated Repair Mode **
Repairing replica ring
Start: Friday, March 16, 2018 01:00:33 Local Time

Replica Ring for replica: .[Root].
Remote server's local ID: 00008040
Remote server's replica root ID: 00008042
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000803f
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008010
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008019
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .driverset.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 000195c6
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 00019595
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 0000c88c
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 0000b2ae
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

Replica Ring for replica: .Students.SD
Remote server's local ID: 00008040
Remote server's replica root ID: 00008126
Remote server name is: .WGSD-LOGIN.SD
OK - Authenticated to server
Remote server's local ID: 00008091
Remote server's replica root ID: 0000811b
Remote server name is: .WGSD-APPS.SD
OK - Authenticated to server
Remote server's local ID: 0000804d
Remote server's replica root ID: 00008194
Remote server name is: .iprint.SD
OK - Authenticated to server
Remote server's local ID: 0000801b
Remote server's replica root ID: 00008267
Remote server name is: .wgsd-master.SD
OK - Authenticated to server

** Automated Repair Mode **
Finish: Friday, March 16, 2018 01:00:33 Local Time
Total repair time: 0:00:32

are you running some sort of virus scanner?
No virus scanner

how big is the DIB?
8GB

how much free diskspace do you have?
:~ # df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/system-root 50G 45G 3.6G 93% /
udev 7.8G 96K 7.8G 1% /dev
tmpfs 7.8G 452K 7.8G 1% /dev/shm
/dev/sda1 493M 47M 421M 10% /boot
admin 4.0M 0 4.0M 0% /_admin
SP1-Overlay-x86_6400521


would you mind to retain from the scheduled repairs (as you shouldn't do these anyway)?
I do not mind refraining from doing the scheduled repairs, as I stated above this is how the environment was set up when I came into it.
0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

First, you really shouldn't have unattended fulls scheduled these days. there have been times when some people did this by routine, but that's been on NetWare 4.x. That being said: please de-schedule these repairs.
Apart from that it seems that the last time you've been hit by the issue was seconds after the start of ndsrepair, BEFORE it could even write its diskspace assumptions to its log. Facing this, i'd strongly opt expanding the root partition an an 8GB DIB with 3.6GB of free space is a ticking bomb. And PLEASE give the box more diskspace BEFORE trying to patch it...
take care,
MB
If you like it: like it.
0 Likes
Knowledge Partner Knowledge Partner
Knowledge Partner

I can see, that you have IDM installed on this box

dxldap Running
dxevent Running


Do you have any "stopped" (but "enabled") drivers?
If you have it, what size of your driver cashe?

I had an issue with start/restart eDirectory, when one of the drivers had huge amounts of events in the queue. Driver didn't start automatically (manual mode)...
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.