Novell Identity Manager has a number of usable languages for managing and converting events.

It all started with XSLT (XML Stylesheet language) which was useful, but nowadays has been severely eclipsed by the primary language, DirXML Script. There are many reasons for this, but perhaps the best reason is the ease of troubleshooting and working through issues with DirXML Script, compared with the XSLT.

In DirXML Script, if you enable tracing, each rule along the way shows the before document, the after document, and each token shows in the trace, as it executes. Thus you can see what happens quickly, no need to enable specific debug code, it is just there built in. On top of that, you can turn tracing on or off at the driver level quickly. Just change the value of the DirXML-TraceLevel attribute in eDirectory on the driver object. Does not actually matter how you do it (via Console One, LDAP, iManager, Designer, or even dxcmd), and it happens live. This is great as there is a definite performance penalty to running with trace on. But it is worth it most of the time. Then with Identity Manager 3.5 a new option on ever token came around, the ability to disable tracing individual tracing.

On top of all that DirXML Script is the direction Novell is moving with the product, and DirXML Script is considered to actually be faster than the XSLT implementation!

For more information on reading and understand DSTrace you can read the following excellent articles:

• The Many Faces of DSTRACE

• Capturing and Reading Novell Identity Manager Traces

• Comprehending IDM Traces - Part 1

• Comprehending IDM Traces - Part 2

Both DirXML Script and XSLT inherit an additional language, XPATH, the XML Path language, which turns out to be quite powerful, often in ways you would not expect!

However XPATH is probably the hardest aspect of Novell's Identity Manager product for most people to understand. I have been working on articles that work through interesting examples of XPATH to try and provide some real world examples that may make it more understandable. Here is what I have written so far, specifically about XPATH:

XPATH Concepts:

• Some thoughts on XPATH in Novell Identity Manager

• XPATH and the context node

• XPATH and math

• Using String Compares in XPATH Statements

• Another attempt at explaining the XPATH Context Node

• Example walk through of using XPATH in Identity Manager

XPATH Cool tips:

• Using XPATH to examine Association values

• Cool tricks using XPATH on nodesets

• Using Global Configuration Values in XPATH

• Using XPATH to Get the Position of a Node in a Node Set

• XPATH to do schema mapping rule

On top of that, I often find myself spending a lot of time explaining XPATH in other articles like in this article on the Attribute tokens (Attribute, Source Attribute, Operational Attribute, Destination Attribute) where I also list the XPATH equivalents to the existing tokens:

• The different attribute options in Identity Manager

I have been working on a series of articles taking apart other default driver configurations, most recently the Compliance Management Platform (CMP) version of some common drivers, and tracking them at:

Detailed driver walk through collection

The SAP HR and SAP Business Logic drivers in the CMP versions have all sorts of interesting XPATH in them, and you can read these specific articles for some of those details:

• SAP HR CMP Integration Driver Walkthrough - Part 5

• SAP HR CMP Integration Driver Walkthrough - Part 8

Recently someone in the forums asked a question about whether it makes more sense to use a very large map table or some other structure from a performance concern, when you have several hundred entries in such a data structure.

My initial answer was, use the Mapping table, since it is ready to go, built, works, supported, and one imagines optimized somewhat for what it is doing. Besides, it is probably doing some kind of XPATH like approach behind the scenes.

The definitive answer was then given by Shon Vella, as:

Mapping tables are much more efficient than the equivalent XPATH for large tables because they are placed in a HashMap, so for a table of size N an XPath lookup would take up to N comparisons (on average N/2) (plus the XML navigation overhead which is also significant), whereas the table lookup would usually take at most log(N) comparisons with very little overhead, e.g. where N = 1000, XPath would take up to 1000 comparisons (on average 500), the table lookup would usually take about 3 comparisons.

Ya, what he said!

However by then it was too late. I wanted to know, can you use XPATH to read a mapping table? Regardless of the answer I figured the exercise would be fun. Turns out I was correct it was an interesting example to work through, and I was able to do all the testing in Designer, using Simulator exclusively. I recently ran into an issue where I was working in Designer 4 and a bug in Identity Manager 3.6.1 has been fixed in IDM 4, and is included in Designer 4. So my code worked fine in Designer's Simulator but did not work for real. However this is pretty rare.

When working on XPATH issues, I usually suggest, having a sample XML document that you will be working with often makes visualizing XPATH issues much easier. To start with, we need to understand the XML for a mapping table object.

<mapping-table>
	<col-def name="source" type="nocase"/>
	<row>
		<col>add</col>
		<col>chicken</col>
		<col>red</col>
		<col>before</col>
	</row>
	<col-def name="dest" type="nocase"/>
	<row>
		<col>modify</col>
		<col>cow</col>
		<col>yellow</col>
		<col>after</col>
	</row>
	<row>
		<col>move</col>
		<col>duck</col>
		<col>green</col>
		<col>inbetween</col>
	</row>
	<row>
		<col>delete</col>
		<col>pigeon</col>
		<col>blue</col>
		<col>female</col>
	</row>
	<row>
		<col>rename</col>
		<col>lamb</col>
		<col>purple</col>
		<col>male</col>
	</row>
	<col-def name="colour" type="nocase"/>
	<col-def name="gender" type="nocase"/>
</mapping-table>

I made up the above table. There are four columns, and four rows of data. The columns names in order are:

source
dest
colour
gender

For reasons I do not understand Designer will place the <col-def> nodes all over the place in ways I would not have expected.

So the trick is that the XML DTD is kind of weak (to my mind) since it is positional. That is, you need to figure out what position each col-def is first, to get the names of the comments. Then in the row data for column 2 (dest) is the 2nd <col> node inside any particular <row> fragment.

For clarity I have edited tables in the XML view by hand to lump all the <col-def> nodes together and found that Designer seems to reorder them back later, so I gave up on it.

First things first, we need to load the Mapping Table into a nodeset variable, named MAP. This would be a case of XML Serialize the source attribute DirXML-Data into a node set variable. For my example, since I wanted to work entirely in Simulator, I set MAP equal to XML Serialize of the above XML text.

Next we need to load the column names and positions.

I used the following variables.

SRC for the source column name
DEST for the destination column name
SRC-VAL for the value passed in to be the source value.

Lets make a COLUMNS variable also a nodeset, so we can select via XPATH in it, to hold the list of <col-def> nodes.

If I were going to use this in a driver, I would probably make the MAP and COLUMNS variables driver scoped, and before loading or reloading them, I would test to see if they have a value, if not load them. This way you only ever waste the time and effort loading them a single time per driver start.

To set local variable COLUMNS I used an XPATH of $MAP/mapping-table/col-def initially, and that did generate a nodeset of the <col-def> nodes. However I later found out that my code that determined positions really expected a parent node, so first I set local variable COLUMNS to the nodeset of XML Serialize the text <payload/> as that inserts a parent node.

<do-set-local-variable name="COLUMNS" scope="policy">
	<arg-node-set>
		<token-xml-parse>
			<token-text xml:space="preserve"><payload/></token-text>
		</token-xml-parse>
	</arg-node-set>
</do-set-local-variable>

Then instead of using set local variable to use XPATH to select the <col-def> nodes, I used the same XPATH to clone by XPATH expression token to copy the data into the COLUMNS local variable. (My target XPATH was of course $COLUMNS/payload)

<do-clone-xpath dest-expression="$COLUMNS/payload" src-expression="$MAP/mapping-table/col-def"/>

That says that copy the <col-def> nodes in the nodeset in the MAP variable, under the <mapping-table> node (our src-expression) into the dest-expression location of $COLUMNS/payload.

That leaves you with a nodeset that looks like:

<payload>
	<col-def name="source" type="nocase"/>
	<col-def name="dest" type="nocase"/>
	<col-def name="colour" type="nocase"/>
	<col-def name="gender" type="nocase"/>
</payload>

Using XPATH to get data out of that is pretty easy. We know to talk about XML attributes like the name="colour" part, we address it as to select it.

Well I thought using that to get the positional value of the source column is easy.

It would help to use a couple of attributes to hold the positions of the source and destination columns.

SRC-POSITION = $COLUMNS/position(col-def[name=$SRC])
DEST-POSITION = $COLUMNS/position(col-def[name=$DEST])

That is, in the $COLUMNS variable, use the position() function to get the position value of the col-def whose name is the $SRC value. Well turns out position() does not work that way so instead use the approach I wrote about in this article:

• Using XPATH to Get the Position of a Node in a Node Set

Which I actually got from a posting at Stack Overflow at:

• http://stackoverflow.com/questions/226405/find-position-of-a-node-using-xpath

It sort so of seems silly that you cannot say position(path/to/nodeset) in XPATH and get back a value, but you can say col[position()=2] to get the second instance of <col> node in the current context. I am sure there is a perfectly reasonable reason why this is so, but it would be nice to have!

What that article reminded me was to try this approach instead:

SRC-POSITION= count($COLUMNS/payload/col-def[@name=$SRC]/preceding-sibling::*) 1
DEST-POSITION= count($COLUMNS/payload/col-def[@name=$DEST]/preceding-sibling::*) 1

That is, find the <col-def> node in the COLUMNS variable, under the <payload> node, and then all its preceding siblings. Count the number of nodes, and add one. That is how many nodes are there before this one, and then add one to get this ones position. This is when I realized I needed a parent node to get this to work. Thinking about it, maybe I could have used preceding:: instead of preceding-sibling:: without the parent node and it would have worked. However, this was a simple solution that makes it easier to read and understand.

When I had the position variables all working, then the following XPATH works to take the input source column (SRC), output destination column (DEST), and an input source value (SRC-VAL), from which we had calculated the SRC-POSITION and DEST-POSITION as above, and the following returns the text node, (or true if used in a condition test):

$MAP/mapping-table/row[col[$SRC-POSITION]=$SRC-VAL]/col[position()=$DEST-POSITION]

What that says is, look in the MAP variable, under the <mapping-table> node, in a <row> node, whose <col> node, at a particular position, is equal to the SRC-VAL we passed in. Then, in that selected <row> node, return the <col> node, at the position we specified for the destination node. This is the power of predicates in XPATH. Select the <row> which meets certain conditions (the right position), and then get the <col> node under that selected <row> that meets a different condition.

This helps if you understand that in XPATH col[1] is the short form for col[postion()=1]. That is, return the <col> node that is first. (or second, of Nth). You can see that for the SRC-POSITION I used the short form, but for DEST-POSITION I used the longer form. Both ways work.

Since we have preset SRC-POSITION and DEST-POSITION to be integers (the result of our count() function is an integer) this gets us the <col> nodes position inside the <row> nodeset fragment.

But you can replace the $DEST-POSITION with the code that calculates it ( $COLUMNS/position(col-def[name=$DEST]) ) and the same for the SRC-POSITION to get it all in one token if you want to blow someones brain! These are like those Perl signatures you sometimes see people use. If you parse the Perl expression you get something cute or funny. Well this is one of those examples.

$MAP/mapping-table/row[col[position()=count($COLUMNS/payload/col-def[@name=$SRC]/preceding-sibling::*) 1]=$SRC-VAL]/col[position()=(count($COLUMNS/payload/col-def[@name=$DEST]/preceding-sibling::*) 1)]

Or this would work too a bit simpler, since col[1] is the same as col[position()=1]

$MAP/mapping-table/row[col[count($COLUMNS/payload/col-def[@name=$SRC]/preceding-sibling::*) 1]=$SRC-VAL]/col[(count($COLUMNS/payload/col-def[@name=$DEST]/preceding-sibling::*) 1)]



I would probably NOT do it all in one step, since debugging it is painful that way, but it does work that way if you would like.

You still need to load up MAP, build the COLUMNS variable, but once that is done the single XPATH step should work to select what you need.

This was a fun example to work through, and I am sure there are more. Often it is interesting to see how much work it would take to reimplement new tokens from IDM 3.5, 3.6 or 4 in older IDM versions. This could be used in an IDM 3.0 environment relatively easily.

I will post my full rules sample code below, if you want to play with it:

<rule>
	<description>XPATH Equiv to Map token</description>
	<comment name="author" xml:space="preserve">Geoffrey Carman</comment>
	<comment name="version" xml:space="preserve">1</comment>
	<comment name="lastchanged" xml:space="preserve">Nov 17, 2010</comment>
	<conditions>
		<and/>
	</conditions>
	<actions>
		<do-set-local-variable name="MAP" scope="policy">
			<arg-node-set>
				<token-xml-parse notrace="true">
					<token-text xml:space="preserve"><?xml version="1.0" encoding="UTF-8"?><mapping-table>
	<col-def name="source" type="nocase"/>
	<row>
		<col>add</col>
		<col>chicken</col>
		<col>red</col>
		<col>before</col>
	</row>
	<col-def name="dest" type="nocase"/>
	<row>
		<col>modify</col>
		<col>cow</col>
		<col>yellow</col>
		<col>after</col>
	</row>
	<row>
		<col>move</col>
		<col>duck</col>
		<col>green</col>
		<col>inbetween</col>
	</row>
	<row>
		<col>delete</col>
		<col>pigeon</col>
		<col>blue</col>
		<col>female</col>
	</row>
	<row>
		<col>rename</col>
		<col>lamb</col>
		<col>purple</col>
		<col>male</col>
	</row>
	<col-def name="colour" type="nocase"/>
	<col-def name="gender" type="nocase"/>
</mapping-table></token-text>
				</token-xml-parse>
			</arg-node-set>
		</do-set-local-variable>
		<do-set-local-variable name="COLUMNS" scope="policy">
			<arg-node-set>
				<token-xml-parse>
					<token-text xml:space="preserve"><payload/></token-text>
				</token-xml-parse>
			</arg-node-set>
		</do-set-local-variable>
		<do-clone-xpath dest-expression="$COLUMNS/payload" src-expression="$MAP/mapping-table/col-def"/>
		<do-set-local-variable name="SRC" scope="policy">
			<arg-string>
				<token-text xml:space="preserve">colour</token-text>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="DEST" scope="policy">
			<arg-string>
				<token-text xml:space="preserve">gender</token-text>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="SRC-VAL" scope="policy">
			<arg-string>
				<token-text xml:space="preserve">green</token-text>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="SRC-POSITION" scope="policy">
			<arg-string>
				<token-xpath expression="count($COLUMNS/payload/col-def[@name=$SRC]/preceding-sibling::*) 1"/>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="DEST-POSITION" scope="policy">
			<arg-string>
				<token-xpath expression="count($COLUMNS/payload/col-def[@name=$DEST]/preceding-sibling::*) 1"/>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="RESULT" scope="policy">
			<arg-string>
				<token-xpath expression="$MAP/mapping-table/row[col[count($COLUMNS/payload/col-def[@name=$SRC]/preceding-sibling::*) 1]=$SRC-VAL]/col[(count($COLUMNS/payload/col-def[@name=$DEST]/preceding-sibling::*) 1)]"/>
			</arg-string>
		</do-set-local-variable>
		<do-set-local-variable name="RESULT" scope="policy">
			<arg-string>
				<token-xpath expression="$MAP/mapping-table/row[col[$SRC-POSITION]=$SRC-VAL]/col[position()=$DEST-POSITION]"/>
			</arg-string>
		</do-set-local-variable>
	</actions>
</rule>