With Novell Identity Manager 4.0 there are a number of new features available. You can read more about those features in these articles:

• What has changed between Identity Manager versions - Part 1

• What has changed between Identity Manager versions - Part 2

• What has changed between Identity Manager versions - Part 3

• More on what's new in IDM 4

There are four new drivers, two are for new connected system support (Salesforce.com and Sharepoint) and two are used as service drivers that are needed for the Reporting module.

These two drivers are the Managed System Gateway (MSG) driver, and Data Collection Service (DCS) driver, the first of which you can read about in this series of articles:

• Trying to understand the Managed System Gateway driver in IDM 4 - Part 1

• Trying to understand the Managed System Gateway driver in IDM 4 - Part 2

• Trying to understand the Managed System Gateway driver in IDM 4 - Part 3

• Trying to understand the Managed System Gateway driver in IDM 4 - Part 4

• Trying to understand the Managed System Gateway driver in IDM 4 - Part 5

The Data Collection Service driver, is the second half of that pairing, and will be the topic of this series. Both these drivers are meant to enable the Reporting module to get enough information about the system to report upon it. The MSG driver is focused more on providing information about how the drivers are configured, heck it even tries to infer the matching rule criteria by reading the rules out of the objects, and the DCS driver is focused more on collecting events about objects for storage in the Reporting database.

Like the MSG driver the DCS driver uses some cache variables that it builds at startup and when queried, returns from the cache using a very neat system, that I spent some time describing in the previous series of articles.

I think it would be helpful, since this is a brand new driver that really no one has any insight into, to pick apart the policies to try and understand what it is trying to do. I think this is a worthwhile endeavor for all the drivers in Identity Manager, and am trying to encourage others to do the same and link to them in this Wiki page: http://wiki.novell.com/index.php/Detailed_driver_walk_through_collection

Lets start with the Input Transform. First up there is a Registration cache that is being maintained by the driver. Before that actually starts, there is a test for a Global Configuration Variable (GCV) that controls whether this cache is used or not. Then a test to be sure the driver is ready to respond.

This test is important, since the driver startup process uses events in XDS documents (Novell's XML dialect) to complete the startup. When you use rules that fire on most any condition, you can start firing while the driver is still initializing and is not yet ready to process events. This is also a pitfall that many beginners to IDM will fall into, when they try to block certain classes or events in say an Event Transformation policy, and have a too indiscriminate Veto policy. If you are not careful, it is possible to veto the startup events, which is REALLY hard to troubleshoot, and looks inexplicable. You can read more about this issue in this article: Avoiding Startup Vetos with Scoping Rules

I ran into this issue when I was trying to get a Schema mapping rule working, basically all in XPATH. (You can read about that approach in this article: Avoiding Startup Vetos with Scoping Rules ). I needed to load the Schema Map object into a variable, and sometimes my rule worked, and other times it did not. I finally tracked it down to it depending on when it fired, before or after the driver finished starting up.

The way you test for this is to check for the object class of the driver object. There is a GCV dirxml.auto.driverdn that is available for all drivers and returns the current drivers full DN in backslash format. This means you can always refer to the current driver easily. On a side note, one of the few new engine side things in IDM 4 is a new GCV of this type, driver.auto.localserverDN which returns the DN of the server this instance of the driver is running on, in LDAP format. I had wanted just such a feature for a long time and was pleasantly surprised to see it show up without any fanfare, nor was it in the documentation. (I submitted a comment, and hope they add in the information soon, if not already).

If the driver has finished starting, then you will get a value, otherwise you get an empty response document, which you can test for, using the Regular Expression equality test (shown as match="something" in the underlying DirXML Script) for .* (period asterisk) which means any number of characters (the period), of any character class (the asterisk).

Then because there are certain events that might cause the cache to need to be recalculated, say some of the information it relies was seen to change in the Identity Vault, we will see later that the Subscriber channel, Event Transformation policy set has a rule to dirty the cache, which it should not do, if this driver is in the midst of updating the cache, nor should this driver update the cache at the moment the cache is being flagged as dirty. This is a problem since the Publisher and Subscriber channels run in separate threads, so they can both run at the same time independent (mostly) of each other.

To handle this they use a mutex variable, that they try to grab, when it comes time to refresh the cache, giving the other thread plenty of time to complete. I am not sure if they have testing to show that three tries of one second (thus three seconds total wait time) is sufficient in all cases, or they just figured if something is locked for three seconds in IDM you have bigger issues. Regardless, three seconds is a lifetime in IDM. I have contrived cases where a rule may run for hours, but usually that is due to poor coding, and I can later optimize it. Brute force can be fun, but slow sometimes! I had a reporting rule I wrote that initially ran over 3 hours before I killed it. Once I spent some time optimizing it, with the help of Father Ramon, I was able to get it down to 2 minutes, which was pretty cool.

You can read more about that approach in this series of articles, where I talk about toolkit rules, which are usually quite a bit slower, and tips and tricks to optimize their performance:

• Toolkit Rules in Identity Manager Part 1

• Toolkit Rules in Identity Manager Part 2

• Toolkit Rules in Identity Manager Part 3

• Toolkit Rules in Identity Manager Part 4

• Example of a Use for a Toolkit Rule in Identity Manager

• Another Toolkit Rule Use Example, Bad Attribute Value Cleanup

Now if the mutex is grabbed, it is time to build the registration cache. First the policy gets some information from the MSG driver, since that drivers DN is stored in a GCV on this driver. It reads back the GUID, DirXML-ConfigInfo, DirXML-ShimAuthServer and DirXML-JavaModule.

Next it starts up the cache driver scoped variable, by XML Serializing in a <cache/> node. Then it loops through the returned MSG drivers. Now in principle there should be only one, but that just means the loop would only ever loop a single time. I am not sure if they are concerned that there might be more than one, nor am I sure what the consequences of two MSG drivers would be since each MSG driver will manage all the drivers in a driver set.

Next it sets some variables with the data returned by the query, for the GUID, the DirXML-JavaModule, and the others. Some of these have to be handled specially, like the GUID, which is returned as a base 64 encoded string. However the representation of that data is not usually done in simple decoded fashion, rather there are three specific approaches, and this driver uses an ECMA function out of the Advanced Java Class ECMA library object to convert it.

es:guid2string($current-node/attr[@attr-name='GUID']/value/text())

The Advanced Java Class was something Novell Consulting put together over the years by building functionality that was missing into their own Java class. It turns out that much of that functionality (but not all of it) got bundled back into the product,. For example there are a bunch of date and time conversion functions that are no longer needed since the Time() and Convert Time tokens were added in IDM 3.5. But other functions are still useful. A while back Shon Vella (his name is in the ECMA as the author) rewrote the functionality into ECMA functions, since distributing a Java class to go on each engine server is a bit harder than including an object in a Package now! With Packages as well an update is much easier, since a driver restart makes the new ECMA take affect, but for a Java file change, it takes a restart of eDirectory, a much bigger deal.

Next it looks at a GCV, drv.registration.msgw.override.address, which allows you to override the address of the MSG server instead of reading the value from the MSG servers configuration attributes. If there is no value, (I.e. The address is not being overridden) then it does a check to see if the MSG servers configuration has an IP address specified. If it does, it uses XPATH to get the value with this statement:

$driverParams//definition[@name='ip-address']/value

This means find any (//) <definition> node in the driverParams variable, whose XML attribute name is ip-address, and return the value. Then it uses the Split() token to cut apart that value, based on commas (since you can have this driver listen on more than one IP address, if the server were multi homed. Tests that the first value (XPATH predicate of [1] which is short hand for [position()=1] ) has a count() of more than 0, in other words there really is a value, and then uses the first value.

If there is no value there, it tries another approach, by checking the DirXML-JavaModule to infer if the driver is running local in the eDirectory instance, or as a Remote Loader, since it can run both ways. However, when running as a Remote Loader, you specify a Java class of com.novell.nds.dirxml.remote.driver.DriverShimImpl in which case, it knows to look at the connection parameters from the DirXML-ShimAuthServer and checks if it starts with the string REMOTE (using the XPATH starts-with($driverAuthSrv,"REMOTE(") ) and if not, errors out, if it does find such a value, it selects it with the following slightly hard to follow XPATH of:

substring-before(substring-after($driverAuthSrv, 'hostname='), ' ')

What it is doing is simple, the string we are after looks something like:
REMOTE(hostname=Some.Server.Name kmo="SSL CertificateIP port=8090) and we want to get out the hostname value. Well that's easy, grab everything after the hostname= and before the next space. Thus the XPATH does in stages, the inner function is to substring after the hostname= string, and then the outer function does a substring from that, everything before the space. Isn't XPATH fun?

Then it traces out the end condition it got to. Either the it got an address from the remote loader connection or two of the error cases we discussed.

Next, it has to do the else case, where the driver is running local, (not using the remote loader shim class) and in that case, it reads back from the NCP Server object the Network Address attribute with the DN based on the new GCV in IDM 4, dirxml.auto.localserverdn, which returns the LDAP DN of the server object the current instance is running on. Since it is returned in LDAP format, it is Parse DN'ed back to backslash format. This is stored as a nodeset, since there are many possible network address values on a given NCP Server, which you can see in the MSG driver series where it builds a cache of such values for each server. It uses the following XPATH to try and get the IP address the eDirectory server is listening on:

$ncpServer/attr[@attr-name='Network Address']/value[component[@name='netAddrType']/text()=9]/component[@name='netAddr']/text()"

Looking in the ncpServer local variable, for the <attr attr-name="Network Address"> node, whose <value> node has a component named netAddrType, whose text value is 9, then get the component netAddr from that node, as a string value. In other words, the netAddrType of 9 means TCP/IP address, and then netAddr is the actual address.

At this point it has read back the IP address successfully or else failed for a variety of reasons, but no without giving a really good effort in the process.

I think that is a good point to take a break as there is a lot more to this rule to work through, so stay tuned for part 2 of this series.