Common Mistakes Newcomers to IDM Make - Part 1

A recent post in the support forums, makes me realize that an article listing common 'rookie' mistakes would be a big help for people starting in IDM.

First mistake, not using the Support Forums! Seriously, Google searches will return lots of great hits from the Forums, or else from Cool Solutions, so one of the big mistakes is not using the available resources. You are likely not the first person to run into an issue, though it does still happen.

The forums are available with a web interface at or via a Newsreader (like Thunderbird, or even GroupWise) over nntp://

Lots of experienced people post messages in the Identity Manager Engines and drivers forums, and can answer some pretty tough questions, or even simple ones.

Second mistake is not reading some important stuff. The documentation is useful, but it is not formatted as a how do I get started with Identity Manager document.

There is a ton of background you need to know, and some of it is quickly summed up in these few articles, which if you read NOTHING else, I highly suggest you read:

David Gersic did an excellent series on what the parts of the fishbone diagram in IDM mean:

Once you understand where things happen in the system, you need to understand how to watch them in action. Fernando Frietas, a nice guy at Novell Support wrote this excellent series on how to read DStrace for IDM:

I have more things you should read if you want to get deeper into it, in this forums article:

Since I am writing this, let me plug my personal collection of articles, 280 so far, sorted by many topics, at:'s_personal_collection

Now onto common mistakes that are not lack of action, but rather mistakes made (Kind of like the debate over the Commerce Clause in the US right now).

Default DN formatting:

The engine, internally, will use backslash format for DN's (Distinguished Names). Thus if you are doing anything that requires a DN, that stays within the engine, use backslash notation. That is \TREE-Name\O-Name\Ou-Name\Object-Name and so on. If you have a leading backslash, you need a tree name, if you start without it, you can start at the O level, or whatever the root of your tree is.

If you are working in another system, then you probably need to consider using the target systems notation, which might be LDAP or something else.

However, lots of tokens can convert from one format to another, specifically ParseDN, which is designed for this (as a note for a moment later, this is a verb token so needs something act upon). However, going from backslash format, you cannot go directly to LDAP format, since needed information is missing. (is that tree most node, a dc=com or O=com? Cannot tell from backslash format). There are again ways to get what you need, but that is beyond the scope of this article. Resolve Token is one approach, else, you can XPATH out from a query the @qualified-src-dn. (I won't explain that here, but can if someone is interested).

Verb Tokens:

If you are in Argument Builder, like in a Set Local Variable, where you build the string, you will notice on the right hand side in Designer tokens are grouped into two groups, the top half are Nouns (like Source Attribute, Global Configuration Variable) which return a value. The bottom half are designated as Verbs, which implies they perform an action upon something. Things like Map, Parse XML, or Replace All, need something (typically) a noun to act upon.

Thus very commonly someone tries to use a Map or Replace All token, and looking through trace, it does nothing. Well they forgot to nest some kind of data, in the form of a noun token underneath it.

The Map token makes the example clear. So a Map token needs a reference to the mapping table, the source field, the destination field, and maybe a default value to return. Well what gets sent in, for the source field? Well whatever you nested underneath it. That might be Destination Attribute value (Be careful if it is multivalued, I am not sure what it will do) or maybe a local variable, like current-node (Which is the current node of the loop you are in, more on this later).

Usually this is a slap your head moment as rookies learn this, since in hindsight, of course it makes sense. But the interface is not the clearest until you have seen it done once.

You nest things by using the up and down arrow to move your tokens around, and you will see that tokens gets nested into things that nest, or just move as appropriate. Try it, it is obvious in about 10 seconds once you try.

Time Conversions:

If you Google for time conversions, you likely will find calls to Java.time() which will work fine, but then you have to construct everything yourself. There are new tokens in IDM 3.5 and higher, Time which gives the current time in whatever format you need (And thus is a noun as it provides data) and Convert Time which takes a source value in a specified format, and converts it to the destination format (And thus is a verb, and needs something nested under it to provide that source data).

Once you use the time tokens, you will never go back to the Java.time functions if nothing else, because there is a Test button in Designer. You can build as crazy a pattern as you need, bit by bit, or pick some very common patterns (like CTIME, FILETIME, LONG-DATE), and then hit test to be sure it looks right, using this moment in time as the sample data.

If you cannot remember if it is MM for minutes, or MM for month in that pattern, it does not matter! The UI in Designer lets you pick it from a menu and it is trivial to build whatever pattern you need.

On a side note, to include literal strings in a pattern use `String` (back tick), which is nice and helpful.

Oh and it does time zone conversions really nicely as well. Again, letting you easily

Built in variables:

This one comes from Shon Vella, one of the authors of much of the engine code, is about the current-node, current-value, and current-op local variables.

If you are inside a For-Each loop, then you are looping over the values inside a nodeset. For each iteration of the loop, if you watch in trace it will say something like:
Action: do-for-each(arg-node-set(token-xpath("$attrib/value"))).
Token Value: {<value> @timestamp = "1212618948#17" @type = "int"}.
Arg Value: {<value> @timestamp = "1212618948#17" @type = "int"}.
Performing actions for local-variable(current-node) = <value> @timestamp = "1212618948#17" @type = "int".

Your for each loop over the nodeset (which is an XPATH in this case), turns out to have only a single value, as shown by the Token Value line.

But it is the Performing action for local-variable(current-node), and it shows the node it is equal too, which is interesting. What that means is, say your nodeset has 12 values, all strings. Then inside your loop, you would want to pick apart the string, say it is delimited by # signs. Then you could do your string manipulation on the current-node local value, which is the current value of the loop.

I was trying VERY hard not to say "current value" until that last moment, because current-value is VERY different and VERY specific to really only one use case (As far as I know), for use inside a Reformat Operation Attribute token.

The Reformat Op Attr token is very special and amazingly powerful because it works on the <attr> nodes in the return from a <query> event. It works on the <add> events which look like <add-attr>, and on the <modify> case, where the nodes look more like <modify-attr> with <add-value>, <remove-value>, or <remove-all-values>. Writing this yourself would be a lot of work, and not always work right. This token does a great job and you can read more in this article: Reformat Operation Attribute

Now, what if there is a multivalued attribute you need to reformat all values? Well the current-value local variable is great, since it lets you pretend you are in a loop over all the values, and apply this logic to all of them. Very confusing from the docs, and rarely properly understood.

The current-op attribute is nothing short of freaky, since you can read from it, and get a XML nodeset copy of the current operation document. But you can actually set it, and change the current event document on the fly. I am still wrapping my head around how to use this, but it is very freaky!

There are couple of great and useful GCV's that are automatically available, like which will return the current drivers DN in slash format, and more. They are actually nicely listed in the docs.

If you look at the Variable selector in Designer, you will often see a couple of tabs. (Variables can be policy or driver scoped, and inside policy scoped, some tokens have error variables they put data into. You can explore them through the variable selector, since the documentation on them stinks. (I have submitted comments on the need to improve the docs about this already).

Setting up an AD driver and SSL:

Probably the most common driver people start with is the Active Directory driver. At some levels this is simple, at others there is annoying complexity.

I think Aaron Burgemeister (who also works in NTS, and posts very often in the forum, so you know you are getting quality help) did a great job of working through the various issues in his series of articles, so to save repetition, just read his stuff:

Biggest issue is the complexity between where SSL is used in this driver. To simply state it, between the engine and the remote loader, you need SSL. This is set like all other drivers, with a kmo="Cert Name" in the connect string (in iManager, but in Designer it has its own line).

There is another SSL possibility, which is encrypting the connection from the Remote Loader to the Domain Controller, when your RL is not running on a DC. This is entirely different and relies on a Certificate Authority running in your Active Directory tree.

Aaron nicely handles this in his articles. Also the issue of what should be in the Authentication context. This is most surprising, for the most common configuration, a Remote Loader running on a DC, the Auth Context should be BLANK (!) and the auth ID should be Domain\LoginName.

Aaron also notes that a common SSL issue is only setting up half of it. On the engine side, you specify the KMO (Key Material Object) name of the certificate object in eDirectory (minus the space dash servername (I.e. SSL CertificateIP - Server22 in eDirectory should be entered into Designer/iManager as SSL CertifticateIP (minus the " - Server22" part). On the Remote Loader side you specify a file that contains the Public key of the tree CA (Certificate Authority). For Windows that is a Base64 encoded version (B64) and for Unix remote loaders it is DER format, which is the binary format (sometimes known as PEM format).

Thus the engine knows the private key of the cert, from the KMO object and on the remote loader side, it knows to trust it, since it trusts the CA that signed it via the B64 or DER file you provided.

I have more ideas, and people in the forums are providing me with more ideas for common problems, so maybe I will work on a series of these articles. If you think of something please let me in the forums, or as a comment response to this article.


How To-Best Practice
Comment List
  • current-value is in the docs as being for reformat-op-attr only, at least it is today:
  • Good argument against script-policies (or pro in case of job protection ;-)
  • Good question: because I would like to manipulate the current operation in ECMAScript (even adding/removing operation attributes)... I would like to have all logic in 1 ECMAScript function call
  • So one of the advantages of IDM as compared to other products is in fact Policy.

    What is even nicer is that for folk who do not like using Policy, doing everything in Java or ECMA is an option.

    However, you get yourself into a case where you are ignoring much of the built in functionality that does a good job , to reinvent whatever you need, as you need it.

    I think in terms of supportability if you want to develop a solution that is supportable, going off and doing everything in ECMA of Java has a real cost for the next guy who has to support it.

    doing it in Policy makes it harder to hide stuff in non-obvious ways. But as you note, it is flexible enough to support all the various approach.

    PS: You could always clone by XPATH your ECMA results to the current document if you wanted to as well as using current-op. Regardless, current-op is a simpler to read approach and still pretty freaky to me!
  • Pieter, why bother converting to JSON? It's very simple to deal with the native XML. You can freely mix ECMA, Policy and, Java code while using the same DOM data.
  • Hi Geoffrey!

    This paragraph grabbed my attention:

    "The current-op attribute is nothing short of freaky, since you can read from it, and get a XML nodeset copy of the current operation document. But you can actually set it, and change the current event document on the fly. I am still wrapping my head around how to use this, but it is very freaky!"

    Yes, indeed. It's freaky. But it also opens new perspectives. Let me explain...

    As a developer I really really hate to write policies: it's slow in terms of development speed (click-hell in Designer), verbose (if you leave Designer and write policies in the XML editor), difficult to remember the tag names... Other people I talk with, share the same opinion: it would be a lot easier if one could just simply implement a Java Class and use that as a policy (the main drawback I see is non-trivial deployment: installing JARs).

    XML Policies vs. (theoretical) Java Policies (these are my opinions and perhaps do not reflect other people's experiences :-))

    • [+] It's a domain language. Anyone working with IdM understands it

    • [+] No need for compiling, installing JARs

    • [-] Slow development

    • [-] Verbose (even for doing the simplest things)

    • [-] Less flexible: restricted number of nouns/verbs

    But now getting to the point of my comments:

    As a rather newbie Novell IdM Developer I recently got my hands dirty on ECMAScript and (for me) it is a lot more convenient to work with. E.g. complex validation & transformation of attributes etc...
    That one paragraph takes my ideas to a whole new level: why not simply write the whole policy in ECMAScript?

    With ECMAScript you could get the best of the two worlds (Java and Policies):


    • People with a developer background can get stuff done quick

    • Much less verbose

    • No need for compiling

    • Flexible

    • Easier testing (?)


    • Less readable for non-developers (but imho non-technical people should keep their hands off
    • policies)

    Building Blocks:

    • A Policy Rule that transforms the NDS Document to a String

    • An ECMAScript call within a Policy Rule with the String as an argument for the function call. This script would then perform the following actions:

    • Call an ECMA function containing the policy logic

    • Transform JSON back to serialized XML

    • Return the serialized XML

    • A Policy Rule that replaces the current operation with the returned (serialized) XML. This is what you were talking about, I think

    Good or bad idea? Am I missing something? Other comments?