Let's talk some more about Packages in Designer 4 - Part 1

Novell Identity Manager initially came with a Console One plugin to manage policies and stylesheets, back when it was still called DirXML. Then as the product matured and DirXML Script became available we got iManager with plugins to manage it.

When Designer for Identity Manager was released, it really was a game changing paradigm shift for Novell's product. Being able to take a project offline, work on it, come back, push out (and double check) your changes made how we work on IDM projects totally different. Documentation generation did not hurt either!

Each release of Designer has added new features and functionality. I recently worked through a series on what is new in Identity Manager 4 Advanced Edition (and compared it to what was introduced with IDM 3.5 and 3.6 releases), which you can read here:

Of course most of those features discussed included support for managing them in Designer. In fact, the series on what has changed is more focused on what has changed in Designer. But that was a high level overview and so much more has changed down low in weeds it is going to take a fair bit of work to get through them all.

With Identity Manager 4 Advanced Edition there are some major engine and Designer changes that are yet another example of a paradigm shift, and it can be summed up in one word, Packages. I started talking about Packages in this article:
A first look at some of Designer 4.0 New Features

But that barely scratched the surface. In fact, when I submitted that article to be published, I had written it before the release of IDM 4 and was under a non disclosure agreement at the time. So I needed to get it cleared that all I was talking about were publicly mentioned features and thus ok to write about before the official release. One of the architects of the Packages in reviewing it sent me some huge responses noting the MANY features I did not discuss in that overview article (Thanks VS! You gave me fodder for a whole stack of articles!)

It is clear that while the concept of Packages seems simple, there is a lot of stuff required for it to work, and a lot of changes under the covers that needed to be done.

Now that I have said "Packages" a few times. let me quickly define it. Prior to this release, drivers for Identity Manager came in two pieces. On the one hand you need some kind of code, the shim to do the work. This might be a Java class, a DLL (as in the case of the AD Driver), a .NET assembly (our first example is the Sharepoint driver in IDM 4), or some other kind of native code. In the case of the Lotus Domino (aka Notes) driver it is a mix of both. There is a Java driver shim, but that requires some native code in the form of the ndsrep program. There is a version compiled for each OS platform it is supported on, and it runs inside the Lotus Domino server instance's process space.

Upgrades to driver shims is quite straight forward. Get the new class, DLL, assembly, or native code widget, replace the files, restart either the engine or Remote Loader, depending on how it is running. Sometimes you need to add a driver configuration option, via the XML view to enable a new feature, other times you need to add a policy to support it.

The second part of drivers is the default configuration. With later versions of Designer (I think version 3 or 3.5) they started versioning configurations, which sort of helped. This way at least you could see what the latest configuration, for your IDM engine version is and decide to use it.

However upgrading is almost an impossible task. If you made even a single change inside one of those default policies, then if you tried to apply the latest driver configuration on top of it, your change will be over written. This means an upgrade of a running IDM system to use the newest policy approaches is quite hard. Now there are some best practices that can help make this easier. Things like never modifying shipping policy, rather linking in additional policies with your custom code. But that gets tricky as well, as often you want to unlink some of the shipping policy objects to replace them with your modified copies, which too will be over written if you were to just apply the latest configuration.

This situation has been tolerated for a while, and it seemed like there is not much you can do about it. But that has all changed in IDM 4 with Designer 4! There are now Packages! A Package contains all the bits and pieces needed to implement the function it is targeted at. That is, they have broken down a driver into its base components, and there is a bare minimum to be able to synchronize anything at all in the driver. That is a base package. Then perhaps you want to do password synchronization, which is an additional package you can add on. Then if you have seen the later configurations there are the Identity Tracking policies for connectivity and tracking with Sentinel's Identity Injection. You can see how this might extend, I think.

Thus you can select a driver, and its base package, and then all the features you would like to add on top of the base.

So far, this sounds like Default Configuration files, but in smaller chunks, and that by itself would be helpful, even if that were all it added. But there is one key thing added, which is versioning. Each Package has a version number. (Usually 3 part, 1.0.2 for example) Again, useful by itself, but once we have version numbers it allows for concepts like upgrades and downgrades. Each package can be upgraded to a newer version if available, or reverted back to an earlier version. Since the Package is a complete entity in terms of managing its part of a driver configuration that means you can easily upgrade or downgrade if needed too.

Heck there is even a factory defaults setting that will throw away any non-package delivered changes and run in just the shipped configuration for testing. What I want is the inverse of Factory mode, which is bundle up all the changes made from the default package setup, and build me a package out of it, so as a consultant I can easily build packages to deliver to clients. Then I could version it, and as we need to make changes, just deliver new package builds. This would mean that backout plans are even easier! Just downgrade to the previous version if you run into trouble.

Again this is sort of high level and there are all sorts of back end things that needed to be changed to make this all actually work. For example, previously Global Configuration Variables were a monolithic block of XML, sitting in one attribute on the driver object. (DirXML-ConfigValues I believe). But now you need the ability for the base package to deliver some GCV's into the configuration and then for each add on package to be able to add some. But more than that, if you choose to revert or remove a package, you need a way to easily pull out those GCV values from the big block of XML. The solution that came up was to break GCV's out of the single attribute of the XML world and into a standalone Resource object type. There is now a GCV Resource object. I thought it would be a DirXML-Resource object, whose DirXML-ContentType attribute is set to be a GCV, Like was done for Mapping table resource objects and ECMA Resource objects, however it looks like they went with a new object class. Be interesting to hear the rationale for that decision.

Now, on the driver object there is a linkage attribute that links the set of GCVs to the driver. Just like there is for ECMA Script objects. What is also new in IDM 4 is that in the Roles Based Provisioning Module, if you link a GCV or ECMA object to the User Application driver object, then they are available in the Provisioning view to use in a number of places, which is a wonderful enhancement!

GCV's are sort of the obvious examples, but also driver configuration settings, which actually use the same DTD as the GCVs. Filters had to be broken up in a similar fashion, but were handled differently, via extensions. Filter extensions are a critical point, the filter is still one gigantic XML blob. However filter extensions allow packages to contribute filter fragments to an installed driver. When uninstalling packages, Package Manager checks each filter piece and if the last contributing extension for any part of the filter has been removed, that part of the filter will be removed as well.

There are some even more subtle low level things that had to be handled as well. For example, the current driver configuration model allows you to build prompting into the driver import process. . I once went through and customized the prompting for a client since they were going to reuse the template driver literally hundreds of times, and while it was not terrible it was kind of painful. Now imagine you want each package able to prompt for information it needs all together in a manageable manner.

A sophisticated framework has been put in place to allow package developers to create very powerful prompting wizards for packages. Prompts can be pre-populated during upgrades and downgrades so users will not have to remember all their settings and prompts are totally dynamic. The developer can define when to show a prompt and how it should look. There are even packages which dynamically generate prompts based on different criteria. Out of the box they provide default prompts which should handle >90% of all prompting needs so developers do not have to spend much time learning about the prompting framework.

I still need to play with this part of the product more to get a better handle on it, but it looks like it is going to be a lot of fun!

A really subtle trick that was needed as well is the notion of managing linkages. That is, currently the various policies are linked to specific places in a configuration. But with packages you can specify where and when to link policies based on other criteria. The example given is the password policy rules. If you look every driver pretty much has the same set of rules in the Publisher Command Transformation (4 or 5 policy objects) and a similar set in the Subscriber Command Transform. I wrote some articles explaining the mechanics behind what is happening, and answer the question, why does nspmDistributionPassword need to be set to Subscriber notify, Publisher ignore for password synchronization to work, you can those at:

Anyway, it turns out there is some subtle and minor differences between these rules for drivers. An example I know of is that the Lotus Domino (aka Notes) driver needs a slightly different password rule on the Subscriber channel, since it MUST send the old password as well as the new password, in order to successfully change a Notes ID file. With the ability to specify linkage in the Package, one base Password Policy package can include all the iterations of these rules, and only link in the correct ones based on the driver type.

There is lots more to say about packages, so I think I will keep digging in and try to provide more detail as I figure it all out.


How To-Best Practice
Comment List
Related Discussions