Approaches to changing Micro Focus/NetIQ delivered packages

With the introduction of Identity Manager 4, NetIQ added a new feature I really like called Packages. Previously, driver configurations were made available as large XML files that defined everything about the driver. This worked fine for a while, but it meant that pushing out minor fixes was hard. Upgrading driver configuration versions was very hard. Integrating customizations around shipping functionality was possible, but once an upgrade was available to the configuration getting your changes integrated back into it were potentially the same amount of work as the first time through.

I have written a lot about packages before, and you can review some of it here:

Packages add a number of useful features to fix some of these issues. In the first place it tries to break down all the elements of a Driver configuration into modular pieces that can be added or removed at will. This means that the individual Policy objects can be handled as part of the package, but also that a new object class, DirXML-Config can hold Global Configuration Values (GCV) that are additive and be managed in a package. Now instead of just the driver set object's GCVs being considered, then the driver object level, there is now also inside both the driver set and driver level the possibility of additional objects.

Introduced as well is the notion of ordering via weight to keep the policies in order properly, but also at the GCV level so there can be precedence as to which one applies last and wins the override wars.

Filters can be managed via Filter Extensions, which is confusing as it is handled different than GCVs and while there is a specific object for it (DirXML-Resource object, with a specific DirXML-ContentType value for Filter extensions) there is still only one Filter object that is effective. (This is probably the part of packaging that confuses most people. Make a New Resource object and look at the available content types, usually the last one is for filter extensions).

There are many more subtle details they had to deal with, and interesting solutions to each problem were derived. The Package Prompts system is an example that is complicated but does what it needs to do.

Those were important changes to make packages work in a granular fashion, but more was needed. There needed to be a way to ensure that the shipping packages were not modified. This is managed by keeping two copies of the XML on each object. There is the shipping XML or data with a checksum calculated. Then there is the active XML also with a checksum. When they differ the item is flagged as dirty with an asterisk. (Asterix is a French Gaul from the cartoons).

It took a few revisions of Designer but a major feature got added, called "Compare Customizations". You get to this option by going to the Driver object properties in Designer, selecting the Packages side tab, and then looking at the list of packages applied. Any package that has a change will be flagged with an asterisk as dirty and then you have several options.

You can:
  • Upgrade (If there is a higher version number available)

  • Downgrade (if there is a lower version number available)

  • Revert Customizations (Throw away all changes and go back to shipping)

  • Compare Customizations

  • Sync to Package (If there is an open for development package)

The Compare Customizations options is very powerful as it will show you the objects changed in what looks like a standard driver compare window, and then show you the data in each object that is changed. This is great since we are well used to looking at this kind of compare to decide if we want changes to apply in eDirectory or be reverted in Designer.

Alas, there is no way to save the output easily that I am aware of, so I often screen shot or take paper notes.

This means you can see in one quick way all the changes you have made. If you decide to revert the package you throw them away. If you chose to Sync to Package you update the packages configuration to include these changes. I like this last option as it reminds me of all the changes I made to review before committing it to my latest package version.

Now having said all that as a quick review of packages lets address the problem that then comes up. In principle, NetIQ should be shipping a Base package for each driver that holds the driver configuration and necessary GCV objects, and pretty much nothing else. Then there should be a Default Configuration package that holds an example of syncing Users and possibly groups, in an attempt at an out of the box driver. There are a bunch of additional add on packages available like Entitlements support, Password Synchronization, Account Tracking, Data Collection and more. These may consist of several related packages or be single packages. They are designed to be independent and removable or addable at will.

The theory is, you enable Package Developer mode at the tree level in your Designer project (Or else you do not get any of the Package building options) and create your own Package in the Catalog. Then you add it to your driver, which has the Base package and whatever others you like. Then all your custom logic goes into policies, GCVs, resources, and whatever else you need in your custom package.

The trick is to do this in a way that allows other packages to come and go as they please. That is a harder trick that it might sound like at first. An example of a trick you can use is in the placement policy, you might have the Default Configuration package added, and it has a placement model you do not like. Well just add a policy after, and redo the placement decision and set it a second time which would work. This does not work so well for Matching policies, since once a match is found, all further Find Matching Object tokens do not actually do any queries to optimize performance.

Thus we find the real issue at hand. What do you do when the NetIQ shipping Default Configuration does 98% of what you need, and you just need some minor tweaks to make it work the way you need it to work?

There are three basic approaches that I can think of:
  1. Just edit the NetIQ packages and run dirty.

  • See if it is possible to add a policy or schema map right after the NetIQ one that reverses its change and does yours instead.

  • Copy the NetIQ package, make the changes and own them. When the next netIQ version comes out, do a Compare Package and port their changes into your package to stay up to date.

The first option is simply to modify the package, run with it dirty, and live that way. The problem is that when new changes from NetIQ come out that possibly fix issues you are not aware of, if you upgrade packages, dirty objects will stay as they were, just becoming 'dirtier' for not having changed to the new values. At a policy level, you could record what you changed, revert the customization on that one object to see is new in the updated package and then re-apply your change. You still are dirty at that point, but you can live like that if needed.

The second object is what I discussed above, try to live with it, and make your changes outside the packaged content, by overriding decisions you disagree with and fixing them in a follow up policy or other object type. As noted, there is a simple way to do this for Placement policies, but much harder for Matching policies. Schema maps are easy, just add another Schema Map object, weighted later, so it applies last and thus wins, but Filter is hard, since Filter Extensions are just additive, you cannot remove attributes.

There are tradeoffs and some things are easily accomplished and others are harder if not impossible.

The third option is the most work intensive, but probably not that bad. Here you take the NetIQ package, copy it to your own new package name, and make your changes in there. I would recommend NOT renaming the policy objects, which is confusing since they will look like NetIQ owned policies, but it will allow an important step in managing them.

If you do it this way, you can use the Compare Packages functionality added in a later Designer build, that allows you to select any two packages and compare their contents. If the objects are named the same, you will be able to see what is different to understand if any changes in the new NetIQ package need to be back ported to your custom package.

Previously, before the Compare Packages functionality, my approach was more brute force. I would have a junk driver set, where I would deploy drivers with 100% base NetIQ packages, and then compare my modified drivers to that, so Designer would show me how they differed. I.e. I was using a spare instance in eDirectory to do the package comparison for me. I needed a way to do compares, eDirectory vs Designer was all that existed, so I forced that to work. Now that Designer can do the compare internally it is easier.

Now with the new functionality you have two approaches. The one I just listed of comparing your package (object names kept the same) to the shipping package entirely in Designers memory. The other option is to use the fact you can compare specific revisions of a package to other revisions of the package. That means you could compare the NetIQ version 1.1.0 to the 1.1.1 version and see a list of changes. Take note of what changed and back port that into your private copy of the package. In this second model you can rename all the policies to match your package name. This is vaguely important, because there is no indicator on a Policy object, GCV object, Resource object, or any other packaged content that tells you where it came from, except the name of it. Thus it is very helpful to have a name that is useful.

There is an 11 character (why 11 characters? Darned if I know. If you happen to know, please post a comment, it is one of those low level things that bugs me not knowing.) short name you can set for a package. The general trend is to name in the format of 11CHARNAME-sub-etp-WhatThisPolicyDoes where 11CHARNAME is the package name, sub for subscriber channel or else pub for the publisher channel. Then the policy set itp, otp, startup, shutdown, etp, cp, mp, pp, ctp, or smp. This makes it easy to look at an object in isolation and know a lot about it. Lothar of PWNotify fame likes adding the weight he wants into the name as well. I agree with his point, since you can now look at the list of policies, see where they should be and identify when they are out of place. However in a later Designer update, they added a Weight column to the policy list display which helps obviate the need for this somewhat. (I have an outstanding request to make that column editable. I.e. If it says this policy has a weight of 800, I should be able to click in that cell, change it to 600 and have it change. One day I hope that makes it out. Seems like an obvious UI thing to me.)

This way you can stay in lock step with NetIQ as they fix driver configuration bugs and keep your customized driver configurations modern and as your customer needs them.

Once you have everything in packages, staging, upgrading, and support really are much easier.



How To-Best Practice
Comment List
  • Be careful with option 3 (copying the NetIQ package) if the package contains entitlement objects. Only do this before you deploy a driver. If you replace the NetIQ package with your cloned one on a driver that has previously been delpoyed, Desiger will delete a recreate all object belonging to the package - including the Entitlement objects. This causes eDirectory to remove all DirXML-EntitlementRef values that referenced those Entitlements from Users and Resources!


  • well, that "-." part in my previous post used <> characters and got lost. Any script kid around to exploit this flaw in the webserver?

    Let's see if brackets print correctly. This is my preferred naming scheme:

    (policyset)-(order)-(shortname) (description)

    for example: "S-ET-320-PWNOTIFYBASE (Source Scoping)"
  • It's actually 12 characters for package short names, probably because someone thought quite correctly more than a dozen would not be short anymore.
    And my general trend is to name policies ---. which has the benefit that you can see even outside fishbone which policy follows which. And you can verify in fishbone view that you got the ordering/weight on the package item correctly. :-)
Related Discussions