In the article, https://community.microfocus.com/cyberres/idm/w/identity_mgr_tips/3736/idm-engine-event-caching I discussed some things about Novell Identity Manager engine cache files.
The Identity Manager engine, is the component that runs as an application within the eDirectory memory space (so it runs anywhere eDirectory can run, so very cross platform), and one of its most powerful features is the its event driven nature.
If an eDirectory instance has the Identity Manager engine installed, and loaded (which will require an Identity Manager Driver Set to be assigned to the server) then it will know what events the drivers in the driver set are looking for, and will cache those events that meet its criteria for processing for the driver. (The exception is if the driver is set to Disabled, (instead of Manual or Automatic on the drivers properties page, in which case the engine will ignore the events and not cache them for this driver).
The events are stored in a .TAO file, see the previous referenced article for lots more detail on the subject. The most common way to deal with this cache file has been to plain delete it, (which can be done via the filesystem, or just Disabling the driver, and when setting it back to either Manual or Automatic ticking the box that appears about not resyncing the application) or to look at its contents via the DXCMD toolkit.
With the release of the latest sets of Identity Manager plugins for iManager, they have been adding some really amazing functionality into the plugins.
Updating an older article, even in IDM 4.8 these snapins are pretty much the same.
I was able to grab a screen shot using the Novell Identity Manager 3.6 M1 Plugins for iManager 2.7 and it shows some neat stuff.
This it turns out is from a Linux/Unix Bidirectional driver connecting to an Solaris environment running NIS. We had submitted a couple of hundred delete jobs overnight, and you can see that 111 removes are left to go. But in the intervening time period while it was deleting the many hundreds of users, 237 more Modify events came in. (Some of this has to do with things that loop back when we delete them in NIS.)
One of our issues was that the Linux/Unix driver had to run a script after deleting a user to clean up the remote environment, and that was taking almost 20 minutes in many cases, due to one set of servers at a remote location.
The engine cache in Novell Identity Manager is a FIFO (First In, First Out) stack, so the events that piled up behind it, could not process until it got to be their turn.
As you can see, the new driver stats page, could show us, while the driver was running, what was going on inside the driver. In our case, we were getting complaints that password changes were not flowing, and a-yup, you can see why. They were probably stuck in the 237 Modify events waiting to process.
If you stop the driver, then you can use the iManager plugins to look at the actual events that make up the queue, much like you could via the DXCMD toolkit, just this time there is a User Interface on top of it, in a web browser.
Another interesting point that I alluded too in the article I mentioned above, is that the size of the TAO file is shown as two values.
The file itself, which if you looked in the filesystem, shows as 172,874 bytes. I.e. if you looked in SYS:_NEWTARE\*.TAO and you knew the object ID of the driver object, you would see a file of that size. (The locations are with your DIB set, so depending on your OS, and where you stored the DIB, would tell you where the TAO files are to be found).
Up till now, that is about all the info you had. In the previous article I discussed why the TAO file gets bigger with each event, but not smaller until all the events are gone. (Basically imagine how you would handle the file system event. Appending a new event to the end of the file is fast and easy. Deleting one in the middle and shifting everything back to use the cleared space is much slower and harder to do, so why bother?). Here is a nice example where we see first off that the plugin tells us the name of the TAO file, (no need to look it up in iMonitor, DSBROWSE, or other tool) tells us the file size in the file system, but adds a very useful line, Unprocessed size, of 69,762 bytes, so we now know that a good portion of that TAO file is empty. This is something we previously had little to no insight into.
To look at the contents of the cache requires that the driver be stopped, and this is apparently a limitation of the API. I really hope the engine team can modify the API some day to allow the plugins to look at the TAO file cache while the driver is running, (and hopefully delete or even better, modify, individual cached events) in some future release. If you try to look at a running drivers cache you get a 641 error via DXCMD, as well as via the iManager plugins, alas.
The use case for a modify of a cached event might be if you have a bunch of stuck events that you want to process but due to some bug in your rules, the exact event sent and sitting stuck in the cache is not going to work, but if you made a minor change, like say changing the case or spelling of an entitlement attribute, might now start to process. I.e. I have a driver that looks for an attribute acmeAddADAccount to go to a value, but for some reason my rule was setting the wrong value, and I stopped the driver after noticing. It would be really nice to dive into the TAO file, edit each event that has the wrong value, and let them process through.
Another nice feature the snapins reveal, that in principle you could get via the DXCMD tookit (by reading the first event and then the last event) is the age of the transactions in the cache file. It shows us the older and newest transaction. In our example, we can see that we are still processing events from 5AM, and there are events as new as 2:28PM. (I think I took the screen shot around 2:30PM).
This is very helpful in trying to understand how far behind your driver is, and whether you want to let it finish processing or stop it, clear the cache and try to get back going again.
Overall these new snapins are a great improvement, with lots of really useful new features. If you are not yet using them go out and get them, install iManager Mobile 2.7 on your workstation and try them in that isolated sandbox. Once you are happy, then you can roll them out to your production iManager instances. They technically are just milestone releases (the 3.6 build at least for now) but they seem to be pretty good so far, and now that I have gotten used to them, I find I cannot live without the new features.