When they released Identity Manager 4, I had a lot of fun working through all the new stuff and I wrote a couple of articles looking at what is new in IDM 4, and what came with previous releases.

• Series: What has changed between Identity Manager versions

• More on what's new in IDM 4

Those articles addressed some of the new things added at a somewhat higher level of detail.

Some more details about new features in IDM 4 is available in this series of articles on Packages, since Packages are one of the major new features in IDM 4, and they are just plain awesome! I wish they were around years ago!

• Let's talk some more about Packages in Designer 4

One of the other major high level features added was the Reporting module, which needs a couple of drivers to support the Reporting module in quite a bit of detail, in the following series of articles:

Data Collection Service:
SeriesData Collection Service Driver Walkthrough

Managed System Gateway:
Trying to understand the Managed System Gateway driver in IDM 4

Now with IDM 4 SP 1, we did not get a lot of new features as such, beyond the inclusion of three new drivers, from some Novell partners:

RSA by TriVir ( http://www.trivir.com )
Google Apps by Concencus ( https://www.concensus.com )
Blackboard by Omnibond ( http://www.omnibond.com )

The rest is bug fixes, as is to be expected, and worthwhile to see. In Part 1 New features in IDM 4 and 4.01 of this series, I discussed a few of the interesting bugs I saw in the list that Novell published in a TID numbered 7008566. You can search at http://support.novell.com for the TID number, or you can try this ugly direct link. Sometimes a link like this will fail the first time, but work the second. I have no clue why that is.

Defects fixed in the IDM 4.0.1 release

• 640695 Driver-ManagedSystemGateway - Config Managed System Gateway Driver should support queries across driversets

• 652290 RMA - Resource Aware RMA fails to locate entitlement based drivers on multiple driversets

• 652925 Framework - Roles/Resources ENH: Have the UA be able to work with IDM Drivers on different DriverSets

These three bugs really all interrelate and are needed altogether in order to fix the core issue. The relationship between the RBPM process and the User Application is complicated. As I have reported before, the User Application driver, that is needed in the IDM project seems like an odd thing. What is it used for? Well it turns out there are at least two core things needed that relates to these bugs.

First off, it will generate and submit queries into the queue of other drivers. With the release of the Data Collection Server (DCS) and Managed System Gateway (MSG) drivers needed for the Reporting module, this approach is also used there. Well it turns out the first bug was that the User App driver (and implicitly the MSG and DCS) driver, could not properly inject events into drivers that were running on a different server, but within the same driver set. That turns out to be an engine side bug. I am not sure where the actual bug for that is, I think it might be buried as part of one of the others.

Now that helps within the same driver set, which was the stated documented functionality. But what if you have two driver sets? Well that it turned out was not supported in the past.

The third bug, that UA should be able to work with multiple driver sets is a new feature in the User Application, and thus since it is the User App, it should work to allow the MSG and DCS drivers should benefit from, and the Role Mapping Administrator (RMA) uses the same basic facility as well.

This is good news, as before this was an issue. Now the Roles and Resource Service driver (RRSD) still only works with a single User Application instance, since it watches the AppConfig container, which is where the User Application configuration is stored, as a child of the User App Driver object in your eDirectory tree. Get that?

Basically one of the things the User App driver is used for is to be a place to store the User App configuration as eDirectory objects.

One of the objects in the AppConfig container is a place to store the nrfRequest objects. These are used by the RBPM module, to request a Role or Resource change. That is, when in User Application you grant a Role or Resource, it does not directly implement it, rather it writes out a nrfRequest object in the AppConfig container. The RRSD driver sees the event of the nrfRequest object being created and processes it. Since these also have begin and end dates, like a Work Order they can be used to time delay start and end dates for Roles or Resource grants. Which is probably the major reason for implementing this functionality in these two pieces.

Thus a single RRSD is still really the only supported way to set it up as it needs to monitor a User Application driver, and handle objects in a particular scope. If you are interested in more of the inner workings of this process, I worked through them in an article about issues when you do not get User Application working after install:
User App Roles Failing to Apply for Administrators

What is kind of neat, is that the permission model with User Application, is itself granted via Roles and Resources. Thus there is a Role for the Resource Manager and they are set up, based on what you entered in configudpate.sh at install time, the first time the WAR file is unpacked. (I.e. First time a web browser requests the User Application page. It is not when you start JBoss, nor when you finish the install). One nice add on to that article is that in IDM 4, and RBPM 4, there is a button on configupdate.sh to force resetting of these Roles, as if you read my article you will see it is somewhat of a pain to reset it by hand.

Now as it turns out, the functionality to allow the User Application to work cross driver sets, was added in the patch for the User Application. It does require the engine patch, which will work with older User Applications, and I think was back ported to IDM 3.6.1 in engine patch 5, but the User Application patch is only available in RBPM 4 and has not been back ported to RBPM 3.7.

UA/RBPM:

• 574769 Installation/Build RBPM Install restarts eDirectory without notification/prompt/option/need

This was a really annoying bug. When you ran an install of RBPM, on a server with eDirectory on it, the installer would restart eDirectory without any warning. Nor was this actually noted in the documentation, that a restart would occur. Thus if you had a server that needed the update, but you did not want to restart eDirectory on, this was an unexpected unpleasant surprise. Alas the resolution of this bug is basically to document the fact that eDirectory will be started and stopped several times in the process. The core issue is that of updated files. For Java classes, the JVM is loaded by eDirectory at start time, and that is when it builds its list of classes, and starts loading them. Once that is done, it will not notice an updated JAR file. It will notice a new JAR file, if a call is made that would use it. Thus a restart of eDirectory is needed to get the updated classes into the system.

Designer 4.01:

• 658693 Application Framework Identity Manager Designer appears to leak memory during normal use.

This bug is one I was quite involved in. The good news is that it REALLY makes a difference. Try Designer 4.01 vs Designer 4.0 and you will see that the memory consumption and performance are very noticeably better. This was actually a fun bug, since it turns out, whatever it is I do every day, leaks memory like a sieve in older Designer builds. What amazes me is how hard it was for someone in engineering to reproduce it. I mean, I just do my day to day work in Designer, and keep finding leaks.

You too can participate if you would like, in tracking down further Designer memory leaks. There is a new bug, tracking additional leaks post Designer 4.01, at: https://bugzilla.novell.com/show_bug.cgi?id=690081 and there are already 5 patched JARs for more leaks found.

What you do is use Designer. Work away as usual, and keep an eye on the memory gas gauge and you can see when you have leaks. When you open Designer, you will see it eat about 80 or so Megs of memory. That is sort of the base Eclipse and IDM plugins consumption. Then go ahead and do your thing. Open your editors, work away, all day long. When you see your memory gauge starting to look saturated, consider taking a few moments, closing all tabs (You can right click in the tab bar to Close All or just Close Others or just the one tab at a time) and then hit the garbage collection icon next to the gauge. Wait about 5 minutes and do it again, just to be safe. It should go down to pretty close to where it was when you opened Designer.

The reality is, that so far the best I have gotten it down too was in the 118 Megs range. Usually I seem to get down to 180 Megs or so after a days usage.

Then there is a tool in most JDK's named jmap that can dump a map of memory that the Designer guys (and gals of course) can look at, to see what bits of memory are orphaned and garbage collection is not cleaning up. I am told that there were a number of objects in Designer that had multiple references to them. That is multiple places in the code would reference the object in memory. If one code path decided to get rid of the object, not all the other locations would clean up their references and thus garbage collection would consider them in use and not clean up the mess. This set of patches was a lot of fun to work through, as each one seemed like it helped a bit. and then after more usage, it would still be leaking (slower, but still leaking) so I would send in another memory dump, another patch and so on.

This is still ongoing, even post IDM 4 SP1, as there look to still be more leaks around. Or at least I seem to be able to find more leaks. I seem to be a real troublemaker when it comes to these!

You can follow the post SP1 shenanigans in bug 690081: https://bugzilla.novell.com/show_bug.cgi?id=690081

At the time of this writing there have been two more sets of patches to try and patch more leaks. I am still finding them so the snipe hunt continues! You can follow along too if you would like.

Regardless, this one set of fixes that made it into SP1 made a huge world of difference in Designer's performance. Ever since Designer 3.0 M2 I think, where the object model in memory to support SVN was changed, this problem has existed and it is a pleasure to see it go away for the most part. They had been making things faster with each release, but the problem was that with all the leaked memory, garbage collection was dragging Designer to a halt as well. In Designer 3.5.1 or 4 to see this kind of performance you had to close the Outline view, and then everything was fast again. But of course, without the Outline view, Designer for Identity Manager is not that useful. Now with this patch, it is much closer in performance, even with the Outline view open.

If you want to contribute, use Designer 4.01, go to bug 690081 and get the latest JAR file patches, and see if you can make it leak. There is no point in testing without the latest patches, only because it will just potentially duplicate existing known/fixed issues. To validate, close all tabs, garbage collect, wait a couple of minutes and do it again, and then you can run jmap to grab a memory dump to upload to Novell.

My jmap is in the JDK/JVM shipped with iManager Workstation. So my path is a bit silly to it:

d:\iman\iMan_27_workstation\imanager\bin\windows\java\bin\jmap -dump:format=b,file=u:\D401-234MB-Apr22.hprof pid#

Replace <pid> with the process ID of your Designer instance. I switched operating systems from Win XP to Windows 7 Pro 64 bit in the midst of this testing and discovered that the PID of interest changed from Designer.exe to the javaw.exe instance eating about a gig of memory. Additionally I found that in order for this to work on Windows 7, due to the very annoying User Access Control functionality, I needed to launch cmd.exe as an Administrator process. That is, Start Run, type in cmd and instead of just hitting enter, hit Control Shift and then Enter, which will launch it as an Administrator and then jmap will work.

When done, you ZIP it up, since memory dumps compress notoriously well, and send it to ftp.novell.com in the incoming directory with a distinctive name, and add a comment to the patch with your file name, and what you did to generate the leak. Clearly there are many many possible ways to generate the problem, and the more hands on deck the better to find them all. I am sure the developer at Novell may feel slightly overwhelmed, but really, we need to just get this issue fixed, once and for all properly.