With Identity Manager 4.5 the way the User Application is configured changed a great deal. Previously, the configupdate.sh tool would do a fair bit of work, unpacking the IDMProv.war file, which is really just a very large ZIP file, changing some of the internal configuration files, and then repack it. As you can imagine, this caused all sorts of unnecessary complexity and problems over the years.

With the release of IDM 4.5 the configuration method changed to use a file ism-configuration.properties instead. Now configupdate.sh is designed to manage this simple text file, (Plus some extra stuff) and not have to fiddle with the guts of the WAR file. This is a huge change since it means if you are having trouble running configupdate.sh, you can now just edit the file. The best example of this is working on OSP related issues. There are many URL's you potentially have to change. There is no search and replace in the configupdate.sh UI but there is in almost any text file editors.

Do keep in mind that configupdate.sh is still needed since it still does 'some stuff' outside of editing the file. The big example is configuring the eDirectory SAML NMAS method. User Application used to require username and password to login. It then held your credentials and used them to login to eDirectory. This way in a form, when you would query eDirectory for a list of departments, job codes, Groups, or even Roles, Resources, Workflows it would do it as your user ID, thus the form would only render the things you were allowed to see.

This is an incredibly powerful component of the User Application and is often under appreciated. This means the mature and powerful permission model built into eDirectory can be leveraged to handle how forms render, which workflows you can see, and much more. However, this requires an authentication to eDirectory.

With IDM Home in late IDM 4.02 timeframe, and all of IDM 4.5 you actually login to OSP, not User Application, then User App requests an OAuth ticket for your session, which means it trusts you, since OSP trusts you via OAuth. But eDirectory is not aware of this, so who to recover this really useful feature? Well they took advantage of the fact that eDirectory has an extensible external authentication model in NMAS (Netiq Modular Authentication Services) which traditionally was used for things like smart cards or biometrics. But it turns out they also wrote a SAML method.

Consider this for a moment, if you had a User Application system, that is using SAML federation to say NAM (NetIQ Access Manager) or Shibboleth then when a user logged in the following would happen.

User logs in to Shibboleth IDP (Identity Provider), which might be pointed back at your main IDV LDAP or some other system. OSP trusts the Shibboleth system (since you exchanged metadata which includes a certificate to sign everything with). Now User App asks OSP for an OAuth ticket, and since User App trusts OSP it trusts the OAuth ticket that identifies your user. Then User App does a SAML federated authentication to eDirectory as your user.

The only time your password traverses the wire is to the Shibboleth IDP, (which pretty much must happen over SSL) and then everything else is systems trusting each other passing around tokens. But you can see how many links in that chain there can be. It works shockingly well, since each step is pretty mature technology, but every so often I like to step back and marvel at the complexity behind it all.

You can read more about all these issues along with some troubleshooting tips in these articles:

• Configuring IDM 4.5s OSP to talk to a Shibboleth IDP

• Series: Troubleshooting OSP in IDM 4.5

• Troubleshooting OSP and SSPR – Part 3

• Troubleshooting SSO in User Application 4.02

In IDM 4.02 you had to manually do the setup of the eDirectory SAML method. The ndsschema tool would install the NMAS extension. This is also a bit amusing, since the actual executable code for the plug in, is stored as attributes in eDirectory!! The LSM itself is stored as binary data in an attribute, and is loaded when eDirectory needs it. (If you look you will see that each platform, Windows, Linux, Solaris, AIX have specific attributes 32 bit and 64 to hold each possible version of the LSM).

A certificate was needed in the Security container, and an authSamlAffiliate object needed to be created. Neither of these tasks were particularly onerous, they were just slightly off the beaten path of stuff we normally do. Now in configupdate.sh in IDM 4.5, if you click the Advanced Options button, and go to the middle tab, in the RBPM section there is a drop down selector that normally sits at No Change, but if you needed too, you could change to Auto, which would go out and create the proper objects and link them as needed. In fact, on a fresh install, if you never run configupdate.sh this will not be properly configured for you.

Do be aware that the automatically created authSamlAffiliate object in the Security container has some time out values, that you should take care to match to the other systems. If this one times out first, you get strange red errors on your workflows, since although your User Application session is still valid, your SAML federation to eDirectory has timed out and your queries will all fail. The best option would be for User Application to notice and re-initiate the SAML federated authentication.

Having said all that, with IDM 4.5 things got a LOT easier in terms of doing upgrades. Anyone who remembers following all the directions in the IDM 4.02 Patch E for User Application know what I am talking about. It got pretty ridiculous. Copy this file, change this value in a database, replace this file, edit this text file in the WAR and repack it, and so on. With 4.5 it seemed like we had almost moved away from that.

Alas with IDM 4.5 Service Patch 2, this sort of thing is back, at least in a minor fashion. I ran into this recently, where I had a problem and totally missed the solution, because I thought we past those bad old days.

When you execute the idmapp_tomcat_init command to start the User Application instance, and peruse the catalina.out file, (On Linux /opt/netiq/idm/apps/tomcat/logs ) you might see an error like this:

2016-08-12 15:20:02,396 [Timer-0] TRACE com.novell.soa.notification.impl.MailEngine- [RBPM] Trying to (re) establish mailserver connection. Will try this every 300000 ms
2016-08-12 15:20:02,401 [localhost-startStop-1] INFO  com.novell.soa.notification.impl.jms.JMSConnectionMediator- [RBPM] Starting JMS notification system
2016-08-12 15:20:02,403 [localhost-startStop-1] TRACE com.novell.soa.notification.impl.NotificationEngine- [RBPM] Could not properly initialize JMS persistence for the notification system. Will revert back to non-persistent asynchronous notification system.
com.novell.soa.notification.impl.NotificationException: An error occurred locating/connecting to the notification Topic as a subscriber.
        at com.novell.soa.notification.impl.jms.JMSConnectionMediator.getConnection(JMSConnectionMediator.java:223)
        at com.novell.soa.notification.impl.jms.JMSConnectionMediator.init(JMSConnectionMediator.java:180)
        at com.novell.soa.notification.impl.NotificationEngine.<init>(NotificationEngine.java:99)
        at com.novell.soa.notification.impl.NotificationEngine.<clinit>(NotificationEngine.java:61)
        at com.sssw.portal.servlet.EboPortalBootServlet.init(EboPortalBootServlet.java:122)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5229)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5516)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1083)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1880)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: javax.naming.NameNotFoundException: Name [ConnectionFactory] is not bound in this Context. Unable to find [ConnectionFactory].
        at org.apache.naming.NamingContext.lookup(NamingContext.java:819)
        at org.apache.naming.NamingContext.lookup(NamingContext.java:167)
        at com.novell.soa.notification.impl.jms.JMSConnectionMediator.getConnectionFactory(JMSConnectionMediator.java:83)
        at com.novell.soa.notification.impl.jms.JMSConnectionMediator.getConnection(JMSConnectionMediator.java:207)
        ... 21 more
2016-08-12 15:20:02,405 [localhost-startStop-1] WARN  com.novell.soa.notification.impl.NotificationEngine- [RBPM] Could not properly initialize JMS persistence for the notification system. Will revert back to non-persistent asynchronous notification system.
2016-08-12 15:20:02,408 [NOTIFICATION THREAD] INFO  com.novell.soa.notification.impl.NotificationThread- [RBPM] Starting asynchronous notification system

There are several parts to this error message. First up, you need to know that JMS (Java Message Queue System) is used to handle emails that the User Application sends. This way, the User App dumps the email on the message bus, and eventually the email delivery system handles them. It moves the emailing code out of User App, makes the handling of timeouts (Which SMTP potentially can have lots of) someone else's problem, and allows for more asynchronous operations.

What is kind of neat is that in fact it actually has a backup mode, that is not JMS based and when the JMS mode fails to start it 'Will revert back to non-persistent asynchronous notification system'

So even with this error your email will still get through.

Like any eager beaver IDM guy, the first thing I did when I saw this message was ask the Goog. Alas, nothing came up at all, except a message in one of the forums from Steve Williams (one of the architects of User Application, and who posts frequently on the NetIQ (formerly Novell, now probably Microfocus) forums that this was fixed in Service Pack 2 of IDM 4.5.

I had just applied Service Pack 4 and I was still getting the issue, which led me to think that the bug had perhaps resurfaced. Alas, I misunderstood his cryptic comment in the forums, since there is some file editing needed to do to fix this issue.

If you look at the readme for Service Pack 2:

https://www.netiq.com/documentation/idm45/idm452-releasenotes/data/idm452-releasenotes.html#b1hq56zz

You will see buried many dozens of pages down, in section 6.7

6.7 Identity Manager Fails to Use ActiveMQ for Guaranteed Mail Delivery#

Issue: Although you use ActiveMQ to guarantee mail delivery in your JBoss and Tomcat environments, Identity Manager might default to the asynchronous JMS mode to deliver notifications from the identity applications. This mode cannot guarantee the delivery of every notification. (Bug 944784)

When this issue occurs, the logs include the following message:

WARN com.novell.soa.notification.impl.NotificationEngine- [RBPM] Could not properly initialize JMS persistence for the notification system. Will revert back to non-persistent asynchronous notification system.

Restarting ActiveMQ does not resolve the problem.

Workaround:In a text editor, remove jms from the name of the Active MQ ConnectionFactory attribute from the context.xml and server.xml files for the application server:

In the context.xml file, change the following line:

ResourceLink global="ConnectionFactory" name="jms/ConnectionFactory" type="javax.jms.ConnectionFactory"/

To:

ResourceLink global="ConnectionFactory" name="ConnectionFactory" type="javax.jms.ConnectionFactory"/

In the server.xml file, change the following line:

Resource auth="Container" brokerName="LocalActiveMQBroker" brokerURL="tcp://localhost:61716" description="JMS Connection Factory" factory="org.apache.activemq.jndi.JNDIReferenceFactory" name="jms/ConnectionFactory" type="org.apache.activemq.ActiveMQConnectionFactory"/

To:

Resource auth="Container" brokerName="LocalActiveMQBroker" brokerURL="tcp:

Now in the code examples, they removed the angle brackets, since that requires escaping to show in HTML. So here it is with the angle brackets.

<Resource auth="Container" brokerName="LocalActiveMQBroker" brokerURL="tcp://localhost:61716" description="JMS Connection Factory" factory="org.apache.activemq.jndi.JNDIReferenceFactory" name="jms/ConnectionFactory" type="org.apache.activemq.ActiveMQConnectionFactory"/>

I did the change, in fact, I first only did one file, I was busy and got distracted and even though I looked at the second I forgot to fix it.

Basically both of them reference a value: name="jms/ConnectionFactory"

To fix it, both references need to become: name="ConnectionFactory"

Now this is kind of interesting as this is a fix at the Application server level. So you can see how it would be hard to deliver this as part of a Service Pack. So I am annoyed at being back to editing files again, but I am glad it is easily available to edit.

What I note the readme does NOT explain is where these files would reside. For Tomcat I know they are in:
/opt/netiq/idm/apps/tomcat/conf as server.xml and context.xml.

In WebSphere and JBoss I am not entirely sure. I find it interesting that the readme does not tell you where either. However, I assume if you use JBoss a lot you are used to it and know where these paths might be found.

When you have made the two changes in the files and restart the IDM applications, the log should look more like this:

2016-08-15 09:51:56,318 [Timer-0] TRACE com.novell.soa.notification.impl.MailEngine- [RBPM] Trying to (re) establish mailserver connection. Will try this every 300000 ms
2016-08-15 09:51:56,324 [localhost-startStop-1] INFO  com.novell.soa.notification.impl.jms.JMSConnectionMediator- [RBPM] Starting JMS notification system
2016-08-15 09:51:56,561 [NOTIFICATION THREAD] INFO  com.novell.soa.notification.impl.NotificationThread- [RBPM] Starting asynchronous notification system
2016-08-15 09:51:56,562 [NOTIFICATION THREAD] TRACE com.novell.soa.notification.impl.NotificationThread- [RBPM] QUEUE EMPTY WAITING...

This would have been nice for the docs/readme to have and show you what success looks like, as opposed to just implying the absence of an error. But we can all wish for better information in the official stuff, and then just come here to Cool Solutions for the actually useful stuff.

You might ask why I am wasting my time writing about a bug that is in the docs, well as noted just above, the success message is missing there, but also, the real complaint I have in this entire issue is that the error is not easily find able via Google searching. I do not know why Google is not indexing the Readme for Service Pack 2, but with this article, I do know that all of Cool Solutions is find able in the Google cache, and this way everyone who runs into this issue in the future will have an easier time finding the solution.

If you happen to run into an interesting error like this, be sure to save the text of the message and be sure to write it up in this format. Making the errors easier to find, and explanations of fixing it more clear is a great thing to have available to you. So if you found this interesting, please consider contributing an error you found on your own as well, you never know it might help me out, so let me be selfish in this request.

As a side note as well, the readme actually has some critical information if you are installing on Red Hat, and some other issues that may come up, so be sure to read deep into it, apparently you really must this time around.