IDM Array Index out of Bounds error


If you use NetIQ Identity Manager you are probably used to seeing all sorts of interesting errors come across your trace screen. The best kinds are the ones where the error message itself gives you a hint and helps you figure out what the root cause is.

I happen to like collecting these errors and then when I resolve them, writing up articles to explain what is going on, hoping that it helps other folk who come across this same issue later.

The errors can be specific to the engine, as I wrote about in this series of articles:

Or they might be driver specific as in these articles:

Active Directory driver:

JDBC Driver:


Heck even about an error in iManager:

What I usually do is, when I see an error come by, that I can copy, I paste it into a text file, and when I figure out what happened, I leave myself a short note on how I caused it, and how I fixed it. Then later, while riding the bus into work I expand upon it into a full length article like this.

Recently I ran into this interesting edge case, that takes quite a bit of work to trigger, but is pretty funny once you get there and figure out what caused it.

Designer is a great tool for working with IDM and it has its strengths and weaknesses. As it grows and gets bigger and more complex the patches for it become larger and more complex. The last couple of Auto Updates have had some interesting little errors, that thankfully have not been show stoppers, mostly annoyances. For example in Auto Update 3 for Designer 4.02 if you tried to add a POlicy object to a Package it worked everywhere, except in the Subscriber Event Transform policy set. If you did it in the Sub Event, then you got the single policy, linked in three times. Very strange error and a patch was available if you needed it to fix it.

With Auto Update 4, there is an issue when you create a Role in Designer, that you can type the object name ok, which then defaults to be the Display Name as well. If you try to update the Display Name though, you get jumped to the beginning of the line, and it sort of types backwards. It is really weird to see, and hard to explain, but easy to trigger.

This issue that I ran into is very specific, and requires just the right set of versions of Designer, the Identity Manager engine, and a Package to trigger it. This will all just work if you had up to date everything or older everything. It is only because of a mixed version state that it is even possible. But it is worth discussing because it allows you to infer some of how the engine works by seeing the error message it generates when you go out of bounds.

If you are using Designer 4.02 AU4, then you have support for the new feature that supports the a new IDM engine enhancement that has added two new policy sets called Startup and Shutdown. Just like we have the Schema Map, Event Transform, Command Transform policy sets, there are now two more. Interestingly they did not update the fishbone policy set view, which both makes me happy (dozens of screen shots in my book on IDM tokens would have to be redone and that is a lot of work) and annoyed, since the way they are shown is a little out of the way. In order to see the new policy sets, you need to click on a driver, in a vault that supports the new feature, then in the Outline view, select the driver, and then look below in the Policy Set tab (Bottom left, usually a smallish window, with 4 tab panes, Provisioning, Properties, Dataflow, and Policy Set) and now you will see the Startup and Shutdown policies.


I knew they were there, I knew there had to be a way to get there, but I could not figure it out, which is usually a bad sign. Someone had to show me, so let me share that info back with everyone else here.

To see this in action, in Designer 4.02 with Auto Update 4, make a new Identity Vault and leave it with a 4.02 engine version setting. If you never do the Properties of the IDVault, Server tab, edit the server, click the get current status, it seems like it will assume it is a engine, and thus supports the new policy sets.

It will then allow you to deploy policies in those policy sets using those values.

If you do this with a non-IDM 4.0.2 patch 3 (aka engine level) you will find that the engine is NOT amused, and you get this error:

DirXML Log Event -------------------
Driver: \ACME-IDV\acme\idm\DriverSet\Active Directory Driver
Status: Error
Message: Code(-9010) An exception occurred: java.lang.ArrayIndexOutOfBoundsException: 15
at com.novell.nds.dirxml.engine.ConfigAbstraction.loadPolicyList(
at com.novell.nds.dirxml.engine.ConfigAbstraction.getPolicyList(
at com.novell.nds.dirxml.engine.Driver.initializeScripting(
at com.novell.nds.dirxml.engine.Driver.startShim(
at com.novell.nds.dirxml.engine.Driver.initialize(
at com.novell.nds.dirxml.engine.Driver.(
at Source)

Which ultimately comes down to this line:

java.lang.ArrayIndexOutOfBoundsException: 15

15? Why 15? Why is 15 out of bounds, that is an odd line to draw in the sand, usually you would expect 0-15 for a four bit counter, so 16 would be out of bounds. What set of values in the engine do I know of that tops out at 14?

Turns out I do know the answer to this, and have written extensively on the topic. With the change over from IDM 3.0 to 3.5 and higher, we got support for properly linking policies and the ability to use Library objects, and reuse policies. In IDM 3.0 and earlier, there were actual attributes, on the Subscriber object, named DirXML-EventTransformation, which was a DN reference to the first policy in the list. That policy had an attribute DirXML-NextTransformation which linked to the next policy in the chain. But as you can imagine if you tried to reuse a policy from a policy set with many policy objects, you did not just link in one object, you linked into the chain at that point.

With IDM 3.5 they revamped all that to use a cool attributes DirXML-Policies to store all this info. A side consequence is that in the good old days, Designer could simply modify the values of the DirXML-EventTransformation during a compare. Now those are shown on that attribute but are really translated into changes to the DirXML-Policies attribute and only a Deploy will update them.

For more information on this issue check out: Talking about the DirXML-Policies attributes

Now it turns out DirXML-Policies is a Typed Name attribute, which means a DN, then two integers. (Of course the DN is really an integer under the covers storing the object ID, we just always see it expanded to an object name by all the tools).

The attribute is stored on the driver object.

The DN component is a reference to the object to be linked (A policy object, XSLT Stylesheet, ECMA Script resource, DirXML-GlobalConfig object, whatever). One of the integers is the numerical value of the policy set. The second integer is its ordering within the policy set. I always forget which is which, to be honest.

Currently the defined values for the policy set are (from that article):
0 Schema Map
1 Input Transform
2 Output Transform
3 ECMA Script Object
4 Sub Event Transform
5 Pub Event Transform
6 Sub Match
7 Pub Match
8 Sub Create
9 Pub Create
10 Sub Command Transform
11 Pub Command Transform
12 Sub Placement
13 Pub Placement
14 GCV Objects

Well, it turns out that as of IDM 4.0.2 engine patch 3, the Startup and shutdown policy sets get the numbers 15 and 16. Which makes sense, as they are next in sequence. With a 32 bit integer to store values in, makes you wonder how many more such things they are going to add. I really enjoy when they add new engine features. Last few IDM updates have been somewhat sparse on engine changes, so this is a welcome new feature.

So that is why 15 is out of bounds. Darn, now I need to go update that article. My work here is never done...

Thankfully I realized this, because that number of 14 was familiar to me, otherwise there was very little hint as to what the error was otherwise all about.

Anyway this is a pretty funny error, that you have to work pretty hard to get into... The other major piece you would need to trigger this, is something actually using those policy sets, which I had in a test policy I had been playing with, that I should never have allowed into a production system.

The new Startup and Shutdown policy sets will be executed only when the driver starts or is told to stop. This is actually quite useful, especially when you realize how much work is being done in the Input Transform these days, as a driver starts up. You have the MsysInfo object being read, and recalculated for reasons I am not sure I agree with. Basically it is to make less work on an Admin in terms of filling out some data. Go take a look at the MSysInfo object in any of your packaged IDM 4 drivers that have the Reporting add on packages.

You have the EntitlementConfiguration object being recreated each restart, basically because there is no simple way to get the LDAP DN of the driver into the object. It is an XML object that has descriptions of the entitlements in this driver. You basically have to make a series of changes in GCV's or add new GCVs and new code to support new entitlements anyway, just as easy to turn the rule off, and edit the object yourself. I get what they are trying to do, I just wish there was a simpler way to do it, that did not need to be in the Input Transform, running every driver startup.

Anyway, these new policy sets will allow this code to move into the Startup policy set, which I am curious to see if they will do. No immediate need, other than it would be cleaner that way.

Now fixing this issue turns out to be interesting as well. As they said in the kids cartoons, knowing is half the battle, and once you know what the error means, you can start thinking of ways to fix it.

I removed the package with the linkages but in identifying the problem, I had first clicked the Version update button and now Designer knew this was a 4.0.2 patch 0 system. At this point, just removing the package and deploying did not update the DirXML-Policies to remove the incorrect values (actually the removed values).

There seems to be no way to fix it in Designer. Because you cannot tell Designer the IDM version is by hand (only options are 4.02 and then Standard or Advanced Edition) and it appears they nicely filtered the option to manage the 15/16 options if the vault engine version is anything less. To be fair, had I NOT hit the Version update and just removed the package, I strongly expect it would have removed the policies properly.

If you manage to be as foolish as I was, and get into this circumstance, first of all, please let me know the steps it took you to get there. But if you need to fix it, use the understanding of what is happening under the covers to clean up your mess. Just use an LDAP browser and edit the DirXML-Policies attribute, and kill all references that have 15 or 16 in them, they are pretty obvious. Do not try this in iManager unless it is completely up to date, as its handling of Typed Name syntax, when the attribute is multivalued was broken a few versions back, and if you edit one value, it throws away the rest. I am sure it is fixed, but be careful in iManager, that it is latest versions of the plugins. I think this is in the eDirectory plugins, but I am not sure.

If you run into a fun issue like this, please consider writing it up and sharing it so everyone can marvel at how foolish you were to get into such a circumstance!


How To-Best Practice
Comment List
  • So how do I get these options to appear in an existing project. I ma using the correct version of Designer
    "Version: 4.0.2 Auto Update 4a Build id: 20140107" the correct version of IDM If I create a scratch project they appear but do not appear in current projects ?