Identity Manager Troubleshooting Toolkit
Getting started with Novell Identity Manager, or actually any Identity Management product can be quite a task. Beyond learning the core engine of the system, and how it processes rules, you need to learn a fair bit about each connected system. Every system has its foibles, and alas, you need to learn quite a bit of detail at a level that most administrators do not really care about.
At the engine level there are a lot of resources available.
The Novell Support Forum's are excellent for Identity Manager. You can find them via a web interface at http://forums.novell.com or via NNTP in a news reader like Thunderbird or even the GroupWise client at nntp://forums.novell.com.
The forums of interest are:
The various fora (I am pretty sure, like data, fora is the plural of forum (Singular of data is of course datum) has excellent content, and searching it can be very useful.
In addition to Cool Solutions, there are a number of tools you really need to know about in order to be really effective at developing an Identity Management solution.
First up is one the most useful tools around, DirXML Association Modifier (DAModifier). A consultant at Novell Consulting in Hungary wrote it, and it is just plain great!
It is a Windows 32 bit application, and needs Client 32 installed to provide the NCP connection to eDirectory. But once that requirement is met, you can select the tree you are interested in, since you are probably logged in to at least two trees, the driver in the tree you are interested, and finally start to look at and manipulate DirXML-Association values.
You can Query the tree for all objects with a DirXML-Association, or with a DirXML-Association of say state 4. Because the DirXML-Association attribute is of schema syntax Path, it is a structured attribute with three components, getting this information in a simple LDAP query is harder than it looks. You can do it via XPATH pretty straight forwardly, if you need to, check out this article on that topic: Using XPATH to examine Association values
If you want to write a rule in Identity Manager's DirXML Script to try and do it all inside Identity Manager, you could look at the series I wrote on toolkit rules, where I explain how you can trigger a rule that goes out, looks for something in the tree, processes it, and does something with it, finally emailing a report back to you of what was done. I quite like these rules, they can be quite helpful.
- Toolkit Rules in Identity Manager Part 1
- Toolkit Rules in Identity Manager Part 2
- Toolkit Rules in Identity Manager Part 3
- Toolkit Rules in Identity Manager Part 4
- Example of a Use for a Toolkit Rule in Identity Manager
- Another Toolkit Rule Use Example, Bad Attribute Value Cleanup
However, having a Win32 application that is designed to do these tasks and do them well is great.
You can clean up your tree by removing all DirXML-Associations of state 2 for example. Or if you are testing in your development lab, you can easily remove all association values for a specific driver, or all of them trivially.
Truly a key tool to have on your Bat-utility belt!
Universal Password Diagnostic Tool:
Next up, is UP (Universal Password) troubleshooting.
The best UP troubleshooting is by Jim Willeke:
I wrote an article on what you ought to see when using this tool, and some of the common errors it can return, you can read it here:
Examples of Jim Willeke's Dump UP Tool
Two key tips:
- Make sure you have a JVM. If you do not, you can use the one in the ConsoleOne directory! (Works!)
- Use Keytool to get your Tree cert (in B64 aka PEM format) into the Java keystore for it to work.
Syntax is something like:
keytool -import -alias newcert -file U:\data\sysmaint\edir\tools\willeke-tools\DumpUPtool\rootcert.der -keystore U:\data\sysmaint\edir\tools\willeke-tools\DumpUPtool\MyKeystore
-alias needs a new alias per Cert imported, usually non issue.
Path to files should match your locations. You can specify the name and location of the keystore, then when you call his tool, you tell it which keystore to use. I think password on the cert in the keystore (makes you do it) should be either "default" or "changeit"
Once you get this tool working, you can specify a user and check the UP status. If Password Policy allows your connected user to read Passwords, then it will show you the password in clear text. Not a security hole, you had to allow it first. It is not on by default and you have to be an admin level user to enable it.
I used to use a BAT file like:
java -jar DumpPasswordInformation.jar -Z SSL -A -h %1 -p 636 -D cn=admin,o=acme -w %2 -b %3 -e path\to\keystore
So that I call the bat with as
DumpUP.bat IpOfServer PasswordToBindWith DNofUserToLookAt
But in the most recent versions released, Jim W. added some very useful new features, and reminded me of something important about keystores.
First off it turns out that you do not need to specify a keystore at all. If you do not, the tool decides to accept any certificate that it receives as if it knew all about the signing certificate authority. After all, if you are running this tool, you pretty much need to be an admin, and as such you have the admins credentials, so not really trusting the Tree Certificate Authority in advance probably does not really matter.
This makes it very much easier to use, as connections just plain work, without any keystore fiddling.
Thus you can get rid of the -e for keystore part and end up with this:
java -jar DumpPasswordInformation.jar -Z SSL -A -h %1 -p 636 -D cn=admin,o=acme -w %2 -b %3
Then the next thing he added was the ability to specify an attribute name (default is cn) and a value for it to match, say the users CN that you are interested. These are the -S and -V options. You need to use both, and actually you need -A as well, so that it will actually search the tree.
Thus you can say:
java -jar DumpPasswordInformation.jar -Z SSL -A -h %1 -p 636 -D cn=admin,o=acme -w %2 -V %3 -S cn -A
This would then get called with:
DumpUP.bat IpOfServer PasswordToBindWith CNofUserToLookAt
Thus you could say, geoffc, as CNofUserToLookAt, instead of having to type "cn=geoffc,ou=acme,dc=com" which is much nicer!
This is hugely powerful when you are troubleshooting password synchronization issues across multiple trees which are connected via Identity Manager. You want to be sure that the passwords and states in all the trees are the same. The -S -V -A switches make it much easier, since you do not have to specify the DN for different trees, since rarely are all the trees of the same design and layout. Now you can just tell the tool to search in each tree. It does a typical LDAP search for the -S value (cn by default) being equal to the -V value (that you provide as the CN).
When I have to do that, I make a batch file that has an entry for each tree with the appropriate administrator DN's and passwords. Then I call it with just the user name I am tracking, and I get to see the current password, and state of the user in each tree. I can very quickly see if there are any inconsistencies in the password synchronization. Saves me huge amounts of time!
To make it even better, Jim W. also just added some really cool extra bits of info to the output, if you select -E for extra info. Now you get to see if the account is disabled, intruder locked out, password expiration time, users login time, and if available, intruder lockout reset time.
Now in one command you can see the password state of the user (UP enabled, same as NDS, same as Simple Password, and they are not intruder locked, nor account disabled, and have a password expiration time in the future) and decide for yourself why they might not be able to login. All the major usual causes are there in the output for you to see.
Even if you disable the ability to retrieve the password, all the rest of the information is very useful.
The next tool you need on your tool belt is some kind of an LDAP browser. This is because you will want to test passwords for logins, or you will want to do searches to see how the data looks, and an LDAP browser makes this very easy.
There used to be a Java based tool called LBE for LDAP Browser/Editor from a guy name Gowars. The links to the site have gone the way of the dodo, (which my friends from Mauritius is not their fault, they blame the extinction of the dodo on the Dutch hundreds of years ago! This was in response to my calling them dodo killers). I have a copy I keep as do many others, but I have not seen the author offer it out anywhere else. If you happen to find an active link to it, please let me know as I like pointing people at it!
One of the amusing things you can use it for, is to make a Java keystore! It turns out that the first time you browse to an LDAP tree with it, and the Tree CA that signed the SSL certificate used in the connection is not known, it asks if you want to accept it just this once, never, or forever. If you choose Always, then it actually adds it to its own Java keystore, which just being a file you could reuse somewhere else, if needed.
However LDAP searches have one benefit over Identity Manager query token searches. They can have a filter, which can lead to more efficient returned values.
For example, in my toolkit rules examples, it would be really nice to query for objects that have a certain value, since those are the ones we need to fix. However, the Query token does not support filtering in this manner, so instead, we query for all objects of that class in the tree, and then loop through the values returned, testing in each case for the certain values. eDirectory is much more efficient at doing that filtering via an LDAP query than running through a loop to do it would be, especially from a memory consumed perspective.
So much so, that when Lothar Haeger was writing his Password Notification Driver for Identity Manager, (
IDM 3.5 Password Notification Service Driver or http://www.brummelhook.com/dirxml.html) he decided that since there could be thousands of objects in a tree, but only a small number are of interest, querying for all the thousands, then throwing away the ones that are not interesting was just not efficient enough. Instead he implemented an ECMA Script function that can do an LDAP Search, using an appropriate filter. In his case as well, the memory requirements to hold a node set of every object in the tree could quickly get problematic, as a node takes up about 10K or so, (You can see where I got that number from in these articles:
Reading and Displaying the Value of Java Heap in Identity Manager Rules
Another Node Set Size Discussion
More thoughts on the size of a node set in Identity Manager ) and you can see how fast the default 64 Meg Java heap allocation in Identity Manager could get eaten up and cause other drivers problems.
It is quite amazing how far you can get with the default 64 Meg memory allocation for Java heap space in Identity Manager, but there is an entire class of problems that just need lots of memory to resolve. However, day to day operations can get by quite well on the defaults.
There are other good LDAP Browsers out there, like Apache Directory Studio, which is an Eclipse based LDAP tool. http://directory.apache.org/studio/
What I like the most about this tool is that it will export your search results as LDIF, CSV, or as an Excel formatted file. This is very useful when you need to play with the data and look through it. Converting an LDIF into the format a spreadsheet can use is not so easy. Nice to have that built in.
Softerra has a nice LDAP Browser as well http://www.ldapadministrator.com/ but they expect you to buy it. I do not see any great value in their tool over the two free alternatives.
I am sure there are more tools you use every day in your Identity Manager toolkit, if you have any, write an article, or if you like, comment and point me at it, and I might write about it as well!