In the first three parts of this article series,
I started listing error codes I found when implementing a new Active Directory driver.
I have done this before for other drivers, like the JDBC driver. You can read that series at:
As I continued working through the process of deploying a new set of drivers, I found even more errors than I had covered in the first three articles, so here comes part 4 of a continuing series.
As always, I encourage any one else who wants too, to contribute this style of article. The more error codes we get published, with the actual text of the error code, a suggestion of what was going on, and how you resolved it, the easier it will be for others who encounter it to resolve it for themselves.
Right now, there are not enough articles about Identity manager driver error codes out there, so these are my attempts to rectify that situation.
I have been collecting errors for a bunch of other drivers and hope to have enough for articles on the eDirectory to eDirectory, PeopleSoft, and GroupWise drivers soon enough.
Please help contribute any errors you may have dealt with. Lets make Google searching more useful on this topic!
Anyway, on with the errors:
Case typo in Schema map.
<nds dtdversion="1.1" ndsversion="8.7">
<source>
<product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\AMERICAS-AD" version="3.5.3">AD</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<status event-id="Xidv1#20090421173802#1#1" level="error" text1="schema violation" type="app-general">
<message>Attribute 'Company' is not in the application schema</message>
<xds-path>/nds/input/add[@event-id='Xidv1#20090421173802#1#1'][@class-name='user'][@src-dn='\ACME-LAB-IDV\Acme\Users\abmaint'][@dest-dn='CN=Maint. Dept AB,ou=Users,ou=Ames,ou=West,dc=americas,dc=acme,dc=corp'][@class-name='user']</xds-path>
</status>
</output>
</nds>
This is an easy error to make. The good news is very easy to fix. The basic gist of it, is that Schema in Active Directory has interesting case patterns. First character is always lower case. In eDirectory that is not a requirement. Thus it is possible to have a typo in your Schema Map object for the driver that mismatches the case.
What makes this even more interesting, is that the driver looks like it is using LDAP to talk to Active Directory, on the shim side. You can see this on the Remote Loader trace, when it writes out the changes to Active Directory. It looks like an LDAP modify. Additionally, as you will see in the next error, if it cannot write to Active Directory, you get LDAP error codes.
Well it turns out that if you use an LDAP browsing or modifying tool, (like Gowars' LDAP Browser/Editor which is no longer publicly available, Softerra's LDAP Browser which is commercial or even just ICE in ConsoleOne or iManager) you will see that LDAP does not care about case in schema names. But the Active Directory driver does!
Very annoying. Trivial to fix, go into your Schema Map rule, and check to make sure you have no typos for this attribute, in this case, Company with an upper case C, should really be company, with a lower case c, on the Application schema side of the rule.
You can read the error message pretty easily, but since it includes an XPATH reference, I think it is worth parsing it, to better understand it.
Obviously the line:
<status event-id="Xidv1#20090421173802#1#1" level="error" text1="schema violation" type="app-general">
Then the <message> node is even better as it says "Attribute 'Company' is not in the application schema". Perfect! How much more direct can you get than that!
Finally you get an <xds-path> node trying to describe the document where the error was found.
<xds-path>/nds/input/add[@event-id='Xidv1#20090421173802#1#1'][@class-name='user'][@src-dn='\ACME-LAB-IDV\Acme\Users\abmaint'][@dest-dn='CN=Maint. Dept AB,ou=Users,ou=Ames,ou=West,dc=americas,dc=acme,dc=corp'][@class-name='user']</xds-path>
This actually returns a nice complex XPATH statement that selects the specific doc. Lets parse it out.
/nds/input/add means select the <add> under <input> under <nds> (like a file system path) node. Then there is a set of predicates, that are unnecessarily complex:
[@event-id='Xidv1#20090421173802#1#1'][@class-name='user'][@src-dn='\ACME-LAB-IDV\Acme\Users\abmaint'][@dest-dn='CN=Maint. Dept AB,ou=Users,ou=Ames,ou=West,dc=americas,dc=acme,dc=corp'][@class-name='user']
Each of these is like saying, the <add> node, where, event-id is the specific value, and the class-name is user, and the src-dn is the specified one, and the dest-dn is the specified one, (not sure why we get a second class-name is user though). In reality, almost any single or pair of those predicates (The test inside [] are called predicates in this case) would probably be sufficient to uniquely identify the event. But the driver programmatically returns some predicate overkill.
One case of LDAP UNWILLING TO PERFORM:
[04/21/09 14:19:51.115]:AMERICAS-AD :
<nds dtdversion="1.1" ndsversion="8.7">
<source>
<product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\AMERICAS-AD" version="3.5.3">AD</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<add-association dest-dn="\ACME-LAB-IDV\Acme\Users\abmaint" dest-entry-id="34068" event-id="Xidv1#20090421181949#1#1">bd479552542e2e4baa0983e2ebf36459</add-association>
<status level="error" type="driver-general">
<message>Password set failed.</message>
<ldap-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">
<client-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">Unwilling To Perform</client-err>
<server-err>0000052D: SvcErr: DSID-031A11E5, problem 5003 (WILL_NOT_PERFORM), data 0</server-err>
<server-err-ex win32-rc="1325"/>
</ldap-err>
</status>
<status event-id="Xidv1#20090421181949#1#1" level="success"/>
</output>
</nds>
In this case, the error is not really as useful as the previous example. It is just saying whatever it was asked to do, it is unwilling to do. Usually because it is an illegal operation.
Thus to troubleshoot, you have look above the error to find the originating document being processed, that generates this error.
I did not clip out that document, but as it happens, I know it was a password set event. There was something wrong with the users password, that did not match the Active Directory password complexity rule, and thus Active Directory refused to set the password, with this error.
You will see this type of error for all sorts of different things that violate the rules that Active Directory is enforcing, but possibly could get sent from an Identity Manager driver. Thus this is a very generic, you did something wrong in what you sent, error.
To figure it out, look at the document being sent and look for errors.
Password errors are hard to track down, since the contents are usually shown as <-content-suppressed-> nodes in the trace (which is a GOOD thing!) but you can retrieve them if you decide to allow it, using a tool like Jim Willeke's Universal Password Diagnostic tool Password Information Tool
I have written a couple of articles on using his tool, since it is such a great and useful tool! You can read those at:
This event was on an event, which you can see, since there is an operation at the beginning of the document. This shows that we are on the Publisher channel, returning from the driver shim.
To further show this general case error, here is another example, where there was a different reason for UNWILLING TO PERFORM.
First the Engine side of the UNWILLING TO PERFORM error:
<nds dtdversion="1.1" ndsversion="8.7">
<source>
<product asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\AMERICAS-AD" version="3.5.3">AD</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<status event-id="Xidv1#20090424184121#1#1" level="error" type="driver-general">
<ldap-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">
<client-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">Unwilling To Perform</client-err>
<server-err>00000032: SvcErr: DSID-031A11E5, problem 5003 (WILL_NOT_PERFORM), data 0</server-err>
<server-err-ex win32-rc="50"/>
</ldap-err>
</status>
</output>
</nds>
Note that this time, the win32-rc is 50, whereas in the first case it was 1325. If someone knows where to find the definition for these error codes, from the Windows side of the house, that would be awesome and make troubleshooting this entire class of problems much much easier!
Then we can look at the Remote Loader side of UNWILLING TO PERFORM to try and figure out what exactly the complaint is about:
DirXML: [04/24/09 14:42:30.64]: Loader: XML Document:
DirXML: [04/24/09 14:42:30.64]: <nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.6.0.4294">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<modify cached-time="20090424184337.053Z" class-name="group" event-id="Xidv1#20090424184337#1#1" qualified-src-dn="O=Acme\OU=DistLists\CN=GlobalDistro" src-dn="\ACME-LAB-IDV\Acme\DistLists\GlobalDistro" src-entry-id="34073" timestamp="1240598617#2">
<association state="associated">49b1b8477b177e46a16587ba840ec38a</association>
<modify-attr attr-name="groupType">
<remove-all-values/>
<add-value>
<value>2</value>
</add-value>
</modify-attr>
</modify>
</input>
</nds>
DirXML: [04/24/09 14:42:30.64]: ADDriver: parse command
className group
destDN
eventId Xidv1#20090424184337#1#1
association 49b1b8477b177e46a16587ba840ec38a
DirXML: [04/24/09 14:42:30.64]: ADDriver: parse modify class = group
DirXML: [04/24/09 14:42:30.64]: ADDriver: association
DirXML: [04/24/09 14:42:30.64]: ADDriver: 49b1b8477b177e46a16587ba840ec38a
DirXML: [04/24/09 14:42:30.64]: ADDriver: modify-attr
DirXML: [04/24/09 14:42:30.64]: ADDriver: remove-all-values
DirXML: [04/24/09 14:42:30.64]: ADDriver: add-value
DirXML: [04/24/09 14:42:30.64]: ADDriver: value
DirXML: [04/24/09 14:42:30.64]: ADDriver: 2
DirXML: [04/24/09 14:42:30.64]: ADDriver: ldap_modify group CN=GlobalDistro,OU=Groups,OU=AMES,OU=West,DC=americas,DC=acme,DC=corp
LDAPMod operations:
delete attribute groupType
add attribute groupType
>> 2
DirXML: [04/24/09 14:42:30.64]: Loader: subscriptionShim->execute() returned:
DirXML: [04/24/09 14:42:30.64]: Loader: XML Document:
DirXML: [04/24/09 14:42:30.64]: <nds ndsversion="8.7" dtdversion="1.1">
<source>
<product version="3.5.3" asn1id="" build="20080229_143300" instance="\ACME-LAB-IDV\Acme\Drivers\IDM\AMERICAS-AD">AD</product>
<contact>Novell, Inc.</contact>
</source>
<output>
<status level="error" type="driver-general" event-id="Xidv1#20090424184337#1#1">
<ldap-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">
<client-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">Unwilling To Perform</client-err>
<server-err>00000032: SvcErr: DSID-031A11E5, problem 5003 (WILL_NOT_PERFORM), data 0</server-err>
<server-err-ex win32-rc="50"/>
</ldap-err>
</status>
</output>
</nds>
DirXML: [04/24/09 14:42:30.64]:
DirXML Log Event -------------------
Driver = \ACME-LAB-IDV\Acme\Drivers\IDM\AMERICAS-AD
Thread = Subscriber Channel
Object = \ACME-LAB-IDV\Acme\DistLists\GlobalDistro
Level = error
Message = <ldap-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">
<client-err ldap-rc="53" ldap-rc-name="LDAP_UNWILLING_TO_PERFORM">Unwilling To Perform</client-err>
<server-err>00000032: SvcErr: DSID-031A11E5, problem 5003 (WILL_NOT_PERFORM), data 0</server-err>
<server-err-ex win32-rc="50"/>
</ldap-err>
In this case, we can see that the event was for a Group object in Active Directory, and I was trying to change the groupType attribute. This was for handling groups changing types from Security groups in Active Directory to Distribution lists, Global, Domain, or Universal.
On the Active Directory side, you can change what the MMC snapin allows you to change. But I was trying to mimic that from the eDirectory side, where a Group Type might change in eDirectory (really not going to happen, but I was trying to be consistent) and synchronize that to Active Directory, such that group type there would change as well. Looks like this specific change was not allowed.
A couple of points of interest in dissecting the error from the Remote Loader side of the house:
You see the document that comes in from the engine:
<nds>
<source>
<product version="3.6.0.4294">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<modify cached-time="20090424184337.053Z" class-name="group" event-id="Xidv1#20090424184337#1#1" qualified-src-dn="O=Acme\OU=DistLists\CN=GlobalDistro" src-dn="\ACME-LAB-IDV\Acme\DistLists\GlobalDistro" src-entry-id="34073" timestamp="1240598617#2">
<association state="associated">49b1b8477b177e46a16587ba840ec38a</association>
<modify-attr attr-name="groupType">
<remove-all-values/>
<add-value>
<value>2</value>
</add-value>
</modify-attr>
</modify>
</input>
</nds>
Then you see the shim (running in the Remote Loader in this case, which is probably one of the better ways to handle it) try and parse the XDS document into commands it can send to Active Directory. The first line is literally: Parse command.
DirXML: [04/24/09 14:42:30.64]: ADDriver: parse command
className group
destDN
eventId Xidv1#20090424184337#1#1
association 49b1b8477b177e46a16587ba840ec38a
DirXML: [04/24/09 14:42:30.64]: ADDriver: parse modify class = group
DirXML: [04/24/09 14:42:30.64]: ADDriver: association
DirXML: [04/24/09 14:42:30.64]: ADDriver: 49b1b8477b177e46a16587ba840ec38a
DirXML: [04/24/09 14:42:30.64]: ADDriver: modify-attr
DirXML: [04/24/09 14:42:30.64]: ADDriver: remove-all-values
DirXML: [04/24/09 14:42:30.64]: ADDriver: add-value
DirXML: [04/24/09 14:42:30.64]: ADDriver: value
DirXML: [04/24/09 14:42:30.64]: ADDriver: 2
DirXML: [04/24/09 14:42:30.64]: ADDriver: ldap_modify group CN=GlobalDistro,OU=Groups,OU=AMES,OU=West,DC=americas,DC=acme,DC=corp
LDAPMod operations:
delete attribute groupType
add attribute groupType
>> 2
DirXML: [04/24/09 14:42:30.64]: Loader: subscriptionShim->execute() returned:
You can see it decide this is a modify event for a group:
DirXML: [04/24/09 14:42:30.64]: ADDriver: parse modify class = group
With a Remove all values node:
DirXML: [04/24/09 14:42:30.64]: ADDriver: modify-attr
DirXML: [04/24/09 14:42:30.64]: ADDriver: remove-all-values
Then an Add-value of 2.
DirXML: [04/24/09 14:42:30.64]: ADDriver: add-value
DirXML: [04/24/09 14:42:30.64]: ADDriver: value
DirXML: [04/24/09 14:42:30.64]: ADDriver: 2
Then it generates the ldap_modify command it will send:
DirXML: [04/24/09 14:42:30.64]: ADDriver: ldap_modify group CN=GlobalDistro,OU=Groups,OU=AMES,OU=West,DC=americas,DC=acme,DC=corp
Finally it starts to do the LDAP Modify operation:
LDAPMod operations:
delete attribute groupType
add attribute groupType
>> 2
After which it fails, and returns the error which I won't reproduce again.
Remote Loader side of a missing Tree CA certificate:
DirXML: [02/13/09 11:50:55.22]: Loader: Waiting for DirXML to connect on 'TCP server socket, port 8090, address localhost, using SSL'...
DirXML: [02/13/09 11:50:55.28]:
DirXML Log Event -------------------
Thread = Subscriber Channel
Level = fatal
Message = Error initializing connection to DirXML: SSL library initialization error: error:02001002:system library:fopen:No such file or directory
DirXML: [02/13/09 11:50:55.38]: Loader: This instance is running as a service. Use the service control manager to unload this instance.
DirXML: [02/13/09 11:50:55.38]: Loader: Waiting for DirXML to connect on 'TCP server socket, port 8090, address localhost, using SSL'...
DirXML: [02/13/09 11:50:56.94]:
DirXML Log Event -------------------
Thread = Subscriber Channel
Level = fatal
Message = Error initializing connection to DirXML: SSL library initialization error: error:02001002:system library:fopen:No such file or directory
This is pretty typical of a getting started with a driver error. In this case, the trusted root CA's public key was not available to the Remote Loader. But the driver wanted to use SSL in its configuration.
Easy enough, export the Public key of the Organizational CA object. This is used so that the SSL connection will trust the Root Certificate Authority that signed the private key of the certificate being used for SSL. This is like when your browser hits a page that uses SSL but does not like something about the certificate.
You will usually see three possible errors.
Certificate date is expired (or in the future, even funnier!).
Certificate claims to be for IP address or IP name X, but is coming from host Y.
The Root CA that signed this cert is unknown.
Firefox and Internet Explorer come with a stack of well known trusted root CA's like Verisign, Thwate, etc. Turns out it is many millions of dollars to get added to that list, so that everybody in the world knows about you. Your cheap little tree CA that comes withe eDirectory, is not in that list. Alas.
There are a couple of approaches to handling this in SSL. Jim Willeke's Universal Password Diagnostic tool (http://www.novell.com/communities/node/1290/password-information-tool) takes a simple approach. Trust any CA it sees, since you have to be an admin, and you should be certain where you are trying to query, and it is horribly unlikely someone is trying to fake you out when you do this. Thus if you do not provide a trusted keystore with the CA included, it still works.
Gowar's LDAP Browser/Editor asks as it connects, and you can accept once, always, or never.
The bidirectional drivers from Omnibond (AS400 aka Midrange, Mainframe, Linux/Unix, and Scripting) are much smarter and the Remote Loader executable for those platforms just ask you to point at any server in the tree via LDAP over SSL, and gets the certificate for you.
The Windows Remote Loader from Novell expects you to do the work, instead of taking the trivial approach that the bidirectional drivers take. (I have a Bugzilla open on this issue now, for about 2 years for the Windows remote loader. Seems like a simple fix that makes life easier for everybody. Oh well). In this case, I had forgotten to copy in the B64 file, with the Tree CA's public key.
Easy enough to fix.
If you want to read more about Remote Loaders in Identity Manager, you can look at this article I wrote on the topic:
The Many Faces of Remote Loaders in IDM
DirXML Log Event -------------------
Driver: \NE_IDN\ACME\ACME-NEIDN-DriverSet\ACME-AD
Channel: Subscriber
Object: \NE_IDN\ACME\Center\CIS\SND\wardusers
Status: Error
Message: Code(-9010) An exception occurred: javax.mail.internet.AddressException: Illegal address in string ``''
at javax.mail.internet.InternetAddress.(InternetAddress.java:64)
at com.novell.nds.dirxml.util.mail.SendMail.generateEMail(SendMail.java:96)
at com.novell.nds.dirxml.engine.rules.DoSendEmail.apply(DoSendEmail.java:189)
at com.novell.nds.dirxml.engine.rules.ActionSet.apply(ActionSet.java:311)
at com.novell.nds.dirxml.engine.rules.DirXMLScriptProcessor.applyRules(DirXMLScriptProcessor.java:268)
at com.novell.nds.dirxml.engine.Subscriber.processAdd(Subscriber.java:1033)
at com.novell.nds.dirxml.engine.Subscriber.processModify(Subscriber.java:1173)
at com.novell.nds.dirxml.engine.Subscriber.processEvent(Subscriber.java:946)
at com.novell.nds.dirxml.engine.Subscriber.processEvents(Subscriber.java:833)
at com.novell.nds.dirxml.engine.Driver.submitTransaction(Driver.java:549)
at com.novell.nds.dirxml.engine.DriverEntry.submitTransaction(DriverEntry.java:1009)
at com.novell.nds.dirxml.engine.DriverEntry.processCachedTransaction(DriverEntry.java:893)
at com.novell.nds.dirxml.engine.DriverEntry.eventLoop(DriverEntry.java:720)
at com.novell.nds.dirxml.engine.DriverEntry.run(DriverEntry.java:525)
at java.lang.Thread.run(Thread.java:534
One of things I like doing in Identity Manager is reporting via email. While I could just trace out the error message, or send an Audit event, often the client does not have Audit configured to collect the events, or I do not want to spend the time reading through trace. I would rather be proactively notified via email.
I was sending an email message, using the Send Email token (do-send-email) in Policy, based on a specific event, and I got this error. Turns out, trying to send to a bad (blank in this case) email address throws an error. Not specifically an Active Directory driver, but it happens I called the token from an Active Directory driver.
You can see that Identity Manager is using a Java class for internet email, javax.mail.internet that Identity Manager calls from its function com.novell.nds.dirxml.util.mail.SendMail
On a side note, there are two types of Send Email tokens. There is the Send Email, and the Send Email from Template.
Using a Template has lots of benefits, specifically that you can make it look pretty once, and then every email you send afterwards using it, leverages that work. I.e. Use it as a Template. If you use the plain send email token, you have to build the entire email message every time.
That's about it for this set of Active Directory driver error messages. I still have more, so expect to see a part 5 one of these days!
As always, I will say it again. If you have collected any messages like these, please write and submit an article. If you do not feel up to writing it, and it is an interesting error, you can always send it to me, and I would be willing to write it up and post it! The more the merrier! Get involved!
Support
Documentation
Learning Services
Partner Programs