What's new in IDM 4.5 - Part 2

I think digging in and seeing what is new in releases of Identity Manager is a useful thing. The high level What's New that the vendor provides is helpful, but rarely covers the level of detail I am interested in.

With IDM 4.5 there is a TID https://www.novell.com/support/kb/doc.php?id=7016414 that lists all the bugs tagged as fixed in IDM 4.5. I thought it would be interesting to pick out ones I wanted to talk about and discuss what the issue is for each. This way you can get a better feel for what is new in this release.

730251 Driver-eDirectory 64 bit integers are not getting synchronized in the publisher channel

eDirectory has a limitation in how it handles time. It uses a 32 bit integer syntax (Time syntax) that stores a count of seconds since Jan 1, 1970. That is 4 billion or so available seconds. At 86.400 seconds a day that runs out in 2109, but if you want to indicate dates earlier than 1970 you need to use it signed, which means only 2 billion seconds which runs out in 2037. Yay Y2K37 bugs to come. This is a pretty common time format, in fact it is often known as UNIXTIME format as well as CTIME.

Microsoft in Active Directory use a format called FILETIME which is a 64 bit integer counter that counts 100 nanosecond intervals since the year 1601. This runs out several thousand years from now.

So first step in fixing the issue was getting eDirectory to support 64 bit integers. Once you support them you have to sync them and it turns out that there was an issue when they came in because they come in octet string and were not being properly converted.

Now this does not mean eDirectory is using 64 bit time, but it means the infrastructure to do it is there. The switch to a new time syntax will be tricky as it will break lots of older things when it happens. I for one look forward to the day when we can switch over.

833117 Driver-eDirectory eDir Bi-Directional, Attribute in Filter set to Ignore on Sub will not sync on Pub

This is a bug I worked on at a client. I discussed the issue in these articles:

But basically the changelog facility you install on the remote eDirectory to collect events is really dxevent from the IDM engine (which is why it cannot coexist with IDM on the same server). If you watch the driver trace you would see that while the <init-params> node specifies the Publisher and Subscriber filter attributes, there is an additional query that reads back the DirXML-Filter attribute from the driver. That is used to pass to dxevent.

Now the filter talks about Publisher and Subscriber channels for synchronize, ignore, notify, or reset values. But Publisher means coming from the application and Subscriber means coming from the Identity Vault eDirectory.

If you hand that list of attributes and XML to dxevent on the remote eDirectory, it is backwards to what you want. From the changelog's instance of dxevent's point of view, Publisher is the Identity Vault eDirectory, not what the engine meant.

The fix was to simply make sure to reverse the values that get sent to changelog. If the attribute was sync or notify on the Subscriber channel, then dxevent in Changelog would send it on. But if you had it set to Sub-Ignore and Pub-Sync then it would be ignored, because it was really the Subscriber channel settings that dxevent was reading. You could do this in the filter to work around it and just put a rule in to block them. But that was a hack.

Glad to see this resolved.

881814 Driver-eDirectory Need to include new deliverables for new ChangeLog utility for Bi-Directional Driver

clutil added

With the 4.5 release they added a command line tool, clutil that allows you some basic management of the changelog instances cache file. Remember it is basically dxevent, the core event subsystem of the IDM engine, so you will recall dxcmd and iManager can be used to look at TAO files (the cache of unprocessed events). The clutil is a basic tool that provides the first steps of being able to do this. It is not quite dxcmd yet but it is much better than nothing.

855272 Driver-LDAP LDAP Driver 402 driver does not pick up publisher events when connected to Sun LDAP

This bug was only happening on large sites, and was an issue with how the drivers handle caching state to generate changed events. Many drivers implement a model where on first start up, you query for the state of the current environment, and then on each polling interval you get everything with newer modification times, and compare to the previous state. The library used has been updated and renamed in IDM 4.02 Patch 6 or 7 and in IDM 4.5. This means that drivers become version specific, since the class in use has changed names (Under the covers I think) and you need the latest engine patch to have the right JDBM/MapDB version to work with the driver.

The real consequence of this one is related to the version dependency. Previously it was pretty rare for a specific shim version to require a specific engine or remote loader patch level, but that is no longer the case.

881954 Driver-LDAP Driver should honor the sensitive attribute list sent by the engine

This one is important as a bug fix. The XDS DTD (NetIQ's specific XML dialect that IDM uses) defines an XML attribute is-sensitive which if set to true means the contents should not be trace. You will see that when a password changes, both the <password> node of an <add> or <modify-password> event and the modify-attr of the attribute ndpsDistributionPassword will show in trace as:

<!--- Content Suppressed --->

This is great as it makes it harder to accidentally show passwords in trace. But it also makes it hard sometimes to debug password issues since you cannot see what is there. Should you have that problem, you can add a policy early on, that does a Strip By XPATH Expression of modify-attr[@attr-name="nspmDistributionPassword"]/@is-sensitive or whatever your specific use case is. Then you can debug and when you are done, remove the policy to get back to a more secure model.

894316 Driver-LDAP - Config LDAP driver packages show MANY policies with the same weight

There is a bug for this issue on many of the drivers. I am glad to see this addressed. I know I had discussed this with folk at NetIQ before. With the introduction of Packages in IDM 4.0 the ability to link in a policy on the fly was added. Add a package it links in GCV objects, Schema Map policies, Policies, etc. For this to work reliably, you need a way to indicate ordering.

For the first release, Designer had a notion of weights, but only stored it in Designer and not in eDirectory. This meant an import from a live system had all sorts of problems with ordering.

With IDM 4.02 and I think Designer 4.02 Auto Update 2 a new option called Migrate Linkages was added to the right click menu on the Package Catalog. This stores the linkage information in the DirXML-pkgLinkages attribute to the DirXML-pkgAux class. This is an XML blob that allows for describing the linkage weight. In fact I suspect they use the same format as they use in the Package definition itself.

With Designer 4.5, when you open a project from earlier Designers, it will ask if you want to Migrate Linkages. Alas, that causes the drivers to re-arrange themselves in Auto mode, which is horrible in a large project. If you just say No to Migrate Linkages, open the Project, then Migrate Linkages from the Package Catalog, this issue goes away.

Anyway, the problem is, the default weight for a package element being created is 500. I used to think it was 0-1000 but it turns out, weight is a 32 bit integer. I just work in the 0-1000 space personally.

But if you have 3 policies all at weight 500, which comes first? It is sort of indeterminate. This is seen all over the NetIQ packages. My personal preference is even if there is only one object in the set (Policy set, GCV's, etc), modify the linkage weight to anything other than 500. This way two packages (which you do not control) cannot collide on two objects with the same linkage.

Looks like for this release they went back and changed all the weights to NOT be 500. Which is a good thing.

771887 Driver-ManagedSystemGateway Jetty server shipped with IDM is not updated since Dorado
864818 Driver-ManagedSystemGateway Upgrade MSgateway Driver shim code to support jetty 9.0
864821 Driver-ManagedSystemGateway Upgrade MSgateway Driver shim code to support jdbm 3.0
878523 Driver-ManagedSystemGateway Upgrade Servlet container to Jersey 1.18 for REST enabled Drivers

This set of bugs are all good to see. Updating some of the back end engine jars from other sources like JDBM (for state files, see bug above), Jetty for web services to the latest versions.

I recall there was a bug a while back, where JDBM was leaking huge amounts of disk space. We found at that time, that it was a JDBM bug and getting a slightly newer version worked. But this is a bad practice since you never know what internal dependencies might change or break. Thus it is nice seeing these refreshed for the 4.5 release.

842596 Driver-Office365 Unable to disable Outlook authentication with Office 365 Driver

The Office 365 driver by default would always try to connect to O365's Exchange instance, even when you were not using it. This is fine if you are paying for Exchange. But there are so many license possibilities for O365 that you might run without Exchange. An updated driver and package provide support for turning off this automatic connection if you are not using it.

877354 Driver-Office365 Office 365 Connector: Unable to assign this license because it is invalid

I am not sure if this is the bug I am thinking of, but the way the that the shim was doing License assignment worked to add or remove, but had issues when you wanted to do both in one operation. This bug resolves some of that issue, but I am not sure if it completely resolves it.

882466 Driver-Packages Adding PUM driver to Identity Manager

Privileged User Manager (PUM) is a neat tool for managing permission access on Unix based computers. You define packages of permissions (like sudo, commands you can execute, etc) and assign them to Groups and Users. This driver allows you to automate some of those assignments. This is a nice fit, as it is a good permission to manage via Roles and Resources.

This had to be added to Designer's palette and the JAR registered as a new driver type. Then the packages get added to the Package Update site for everyone to download or else come with the newer Designer versions.

883827 Driver-Packages Add new gcv service-account-dn to Common Settings Advanced Edition GCV package

The Jade project, produced the PCRS (Permissions Collection and Reconciliation Service) for reading permissions out of text files and connected systems to map to Resources to increase the value of the Role catalog. But it needs to make a series of SOAP calls to the User App since there is no Create Role, Create Resource, or Assign Resource to Role tokens in the engine. (You could always call a Workflow with an Integration Activity to do the work, or write a custom Java class that used the stubs). In fact in the PCRS drivers they include a Java class that makes the SOAP calls. To do this you need to know the User Applications URL and admin user to use. There is no simple and reliable way to discover where the User Application is, so there is DriverSet level GCV object added, the Advanced Settings Common package that includes the URL of the User Application. This bug requested adding in the user name to login with as well. This looks like it might have been done earlier and out of band of the 4.5 release since I recall seeing this GCV in earlier than the 4.5 package versions.

As you can see there are lots of fun new things and bugs fixed in Identity Manager 4.5. Stayed tuned for more, if you look at the TID there are more than a hundred bugs listed. I am just trying to pick ones I consider interesting, and being very picky but I have much more to say on this topic.

Let me know in the comments if this sort of article is helpful. I am willing to do it just so I know what is new, but let me know if writing it up and sharing it is of value to you folks out there.


How To-Best Practice
Comment List
Related Discussions