Troubleshooting OSP in IDM 4.5 - Part 2

Recently I spent a lot of time working through the new OSP (One SSO Provider) used in Identity Manager 4.5 for logging into the various Identity Applications (User Application, Catalog Access, Home and Provisioning Dashboard, Reporting, Access Review). When I work on something like this, I always collect interesting error messages, and try to write articles sharing what I learned.

This is part two of a series, in the first one I ran through some fun errors, like missing the tree CA cert and LDAP failing to work, or bad permissions on WAR files not deploying.

This next error was interesting. I was trying to configure SAML as the front end authentication for the Identity Applications. That is for Single Sign On (SSO) purposes. But I saw this in the trace, and spun my wheels for a while trying to figure it out. In the end it was a bit of a red herring and not entirely related to my issue. It was a real issue, but not related to mine. In hindsight this was not the SSO I was looking for.

When I looked in catalina.out on startup of Tomcat I saw the following:

2015-03-10 11:35:15,449 [localhost-startStop-1] INFO  org.jboss.cache.factories.ComponentRegistry- JBoss Cache version: JBossCache 'Malagueta' 3.2.1.GA
[RBPM] Logging successfully initialized by class com.sssw.fw.servlet.InitListener.
2015-03-10 11:35:16,804 [localhost-startStop-1] WARN com.novell.common.auth.saml.AuthTokenGenerator- [RBPM] Failed to initialize SSO: Missing SAML signing cert/key.
2015-03-10 11:35:16,804 [localhost-startStop-1] INFO com.novell.common.auth.saml.AuthTokenGenerator- [RBPM] SSO Framework is disabled.
2015-03-10 11:35:16,819 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Missing private key for SSO header signature.
2015-03-10 11:35:16,819 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Failed to initialize SSO Filter oauth due to configuration problem.
2015-03-10 11:35:16,827 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthRestFilter- [RBPM] Missing private key for SSO header signature.
2015-03-10 11:35:16,827 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthRestFilter- [RBPM] Failed to initialize SSO Filter oauth due to configuration problem.

With Tomcat, the logs are usually in /opt/netiq/idm/apps/tomcat/logs/catalina.out.

So here I was, working on an OSP SSO problem and getting a message the SSO Framework was not initializing. Additionally the error "Failed to initialize SSO: Missing SAML signing cert/key" made me think I had a metadata/key issue in the SAML configuration for the SSO federation I was trying to set up.

I spun my wheels for a bit about this, but the clue that finally gave it to me was something I knew from the IDM 4.02 days, but is not as obvious in the IDM 4.5 world. This error came during the IDMProv.war deploy, but before the OSP.war deploy, which made me think maybe this is an RBPM/User Application federation issue, not an OSP federation issue. Once I got that far I realized that in fact I actually knew what was going on. I had just recently helped someone with this process. What is neat is that User App started doing something pretty clever when it started supporting SSO methodologies to the front end.

Initially User Application allowed a Username/password combination for login. But later versions added support for Kerberos auth (Using Active Directory login of local machines Kerberos ticket), SAP Tickets, and SAML for federation.

But when you log in to User Application, start a PRD, look at the Org Chart, or do anything that uses the DAL (Directory Abstraction Layer) it needs to talk to eDirectory as the logged in user context. In the good old days on Netware, the server could impersonate any user. In these more modern times that is a harder problem. Generally you login as the user again to eDirectory. When using Username/Password for logging in that is pretty easy, keep the info in memory and then bind to LDAP with those credentials and away you go. Unbind, and bind as the next user for the next transaction. (Or per thread, or whatever).

When you start talking about other methodologies for login, SAML, SAP Tickets, Kerberos, the LDAP username and password are not readily available. So how do you impersonate the user? One way might be how NAM (NetIQ Access Manager) can do it, which is to identify the user in eDirectory somehow (Unique naming attribute somewhere in the request) and then read back the nspmDistributionPassword so it can be injected into form fills or header based authentications. This is very powerful and has a really useful set of use cases.

User App however uses a different approach. Federation. In fact, User Application ships with an NMAS (Novell Modular Authentication Services) module for SAML to eDirectory. Which is pretty neat. Would be kind of fun if something else used this NMAS SAML method. But alas I am not aware of any other user of this.

NMAS methods are actually pretty strange. The binary code to run the NMAS Method is actually stored as attributes of the object in the Security container. It bootstraps itself, from code, out of the directory. Thus these objects get pretty large, but since there are maybe 10 or so in any given tree it is not really a big deal. There is actually a different copy of the binary for each supported platform. If you look at the schema definition for the objects it is kind of interesting. There is a nspmPolicyAgent that has a nspmPolicyAgentLinux, nspmPolicyAgentAIX and so on. But this is not the kind of NMAS method I am talking about here.

In fact, I think that the LDAP browser I like to use, LBE, which is only a 600K Java application, does not like reading back these objects. I think because they are too big. (I doubly think that is so, as I was trying to read a user with a bunch of Roles, Resources, and Entitlement Results, nrfHistory attributes, so it was a very 'thick' user and it would not open it either, as annoying as that was.)

The eDirectory Object Class is SAS:NMAS Base Login Method, but if you are looking via LDAP it is sASNMASBaseLoginMethod.

The attributes involved are listed below:

description , sASAdvisoryMethodGrade , sASEncryptionType , sASLoginClientMethodNetWare , sASLoginClientMethodWINNT , sASLoginConfiguration , sASLoginConfigurationKey , sASLoginPolicyUpdate , sASLoginSecret , sASLoginSecretKey , sASLoginServerMethodNetWare , sASLoginServerMethodWINNT , sASMethodIdentifier , sASMethodVendor , sASVendorSupport , sasCertificateSearchContainers , sasClientModuleEntryPointName , sasClientModuleName , sasLoginClientMethodAIX , sasLoginClientMethodAIX64 , sasLoginClientMethodHPUX , sasLoginClientMethodLinux , sasLoginClientMethodLinuxX64 , sasLoginClientMethods390 , sasLoginClientMethodSolaris , sasLoginClientMethodSolaris64 , sasLoginClientMethodSolarisi386 , sasLoginClientMethodTru64 , sasLoginClientMethodWinX64 , sasLoginServerMethodAIX , sasLoginServerMethodAIX64 , sasLoginServerMethodHPUX , sasLoginServerMethodLinux , sasLoginServerMethodLinuxX64 , sasLoginServerMethods390 , sasLoginServerMethodSolaris , sasLoginServerMethodSolaris64 , sasLoginServerMethodSolarisi386 , sasLoginServerMethodTru64 , sasLoginServerMethodWinX64 , sasMethodVersion , sasNMASMethodConfigData , sasSASLMechanismEntryPointName , sasSASLMechanismName , sasServerModuleEntryPointName , sasServerModuleName , sasUnsignedMethodModules 

Notice how there are sasLoginServerMethodXXXX and sasLoginClientMethodXXXX where XXXX is one of:


With User App they released the SAML NMAS method, which has methods for a bunch of the modern supported platforms. (I remember there being an issue where the 64 bit Windows method was late in shipping).

What is interesting is that it shows all the platforms that eDirectory was once supported on, though the s390 is a bit of a surprise. The lack of support for a platform was an issue for a while, on some of the platforms. This used to require manual setup in IDM 4.02 in the past.

If you do not set it up, you will see messages like this in the catalina.out (or JBoss server.log, or the WebSphere SystemOut.log):

2015-03-10 11:35:16,804 [localhost-startStop-1] WARN  com.novell.common.auth.saml.AuthTokenGenerator- [RBPM] Failed to initialize SSO: Missing SAML signing cert/key.
2015-03-10 11:35:16,804 [localhost-startStop-1] INFO com.novell.common.auth.saml.AuthTokenGenerator- [RBPM] SSO Framework is disabled.
2015-03-10 11:35:16,819 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Missing private key for SSO header signature.
2015-03-10 11:35:16,819 [localhost-startStop-1] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Failed to initialize SSO Filter oauth due to configuration problem.

Then when you actually try to use a login function, like IDMProv you will see:

2015-03-06 15:09:49,115 [http-bio-8080-exec-10] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Missing private key for SSO header signature.

The user interface will show an error that reads something like this. The actual screen is pretty boring and just has this text on it:

    Identity Manager authentication is not correctly configured or Identity Manager to eDirectory SAML communication is not functioning correctly.
    Please contact an administrator to correct the problem.

The underlying issue here is really around whether the NMAS SAML method is configured or not. In IDM 4.02 and earlier, you needed to go make a couple of objects, but with IDM 4.5, the tool can do it for you now. The way you do it, is launch, and switch to the tab named "SSO Clients", look at the RBPM section.

Take a look at the image attached, you can see what it looks like normally in this screen shot.


Then when you click Show Advanced Options at the bottom the display changes, and you get a new line as shown in this screen shot.


The new line "RBPM to eDirectory SAML Configuration" gets added, and the default is No Change. If you did the install via the console command line version of the installer, it probably did not run the step that leaves this setting as Auto, which would do all the creates that are needed. This is a nice touch since it was a bit fiddly making objects in iManager that do not have any editors. You had to select the object class, and pass in the specific values that are needed. Like the authSamlProviderID, it must be a very specific string. There is a reference to a Certificate/Trusted Root object that needs to be set as well.

If you look in your tree for the objects that it creates that I will show below, then if they are missing, set this to Auto and save. When you do that it creates the following objects.

In the Security container it will create the following object:
	RBPMTrustedRootContainer, under which will be an object: 
RBPMTrustedRoot_1426083363636 (Assume that is CTIME string to unique it) - a self signed cert only 2 years though.

Then it will make this object, nested under SAML Assertion object, under the Authorized Login Methods container in the Security container.

RBPMSAML_1426083363643.SAML Assertion.Authorized Login Methods.Security
Authorized Login Methods
SAML Assertion (Created by NMASInst and schema extensions)
This object contains the attributes:
authsamlCertContainerDN: DN of RBPMTrustedRoot object above
authsamlTrustedCertDN: certDN above.
authsamlValidAfter: 1000
authsamlValidBefore: 1000
objectClass: authsamlAffiliate

If you then restart the Tomcat instance, you will now see in catalina.out something much more appropriate:

2015-03-11 10:29:28,527 [localhost-startStop-1] INFO  com.novell.common.auth.saml.AuthTokenGenerator- [RBPM] SSO Framework is enabled.
2015-03-11 10:29:28,567 [localhost-startStop-1] INFO com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] SSO Filter oauth is enabled.
2015-03-11 10:29:28,575 [localhost-startStop-1] INFO com.netiq.idm.auth.oauth.OAuthRestFilter- [RBPM] SSO Filter oauth is enabled.

Now one interesting thing you need to keep in mind is that this object, the authSamlAffiliate defines the timeout for the SSO session to eDirectory. This is basically independent of the timeout value for the overall SSO session either through SAML to some IDP (NAM? Shibboleth?). Thus if these two are out of sync, you can get to a situation where your token from your IDP that allows you access to the User App or Identity Apps is still perfectly valid, but your SAML session with eDirectory is no longer valid, and you start getting strange errors and short of logging out, and logging back in, you cannot seem to renew it.

Additionally if the time in your system is not closely aligned (identical is best! NTP is your friend here!) to the federation sites time it is possible for the token to fail to work.

In this case, the values out of the box are 1000 seconds before and 1000 seconds after the Federation auth token, which is interesting and you almost certainly will want to tune these.

Valid before is not very much of a problem, but as you can imagine you will likely want to increase the value of the Valid After, since that timeout looks pretty bad to the end user when you hit it.

Now the consequence of a long timeout here, even if abnormally long, like 8 hours, should not be a big deal, since the trusting party, that uses the certificates signed appropriately, is just the User Application. By that I mean, it should not be possible for anyone else to take advantage of this, since both sides have to agree to the federation configuration. did that for us, between RBPM and eDirectory. Anyone else who wants to do federation to eDirectory would need to set up their own federation, which would require sufficient rights in eDirectory that they already have access to all of your directory.

But people seem to like to minimize the window here, but make sure it is as long as the timeout for your SSO session and then a little bit more. One problem is that not everything you do in User App may refresh this connection.

Keep this in mind in case you start seeing these odd timeout errors, as now you know where to go tune the values.

That about wraps this one up. I hope you find it helpful. It was one of those annoying things that in hindsight was really obvious and simple, but the error message mislead me initially and sent looking in the wrong direction.

Once I got reoriented, it was really quick and interesting to fix.


How To-Best Practice
Comment List
  • So it gets strange. In 4.7.3, ID Apps seem like they are moving AWAY from the NMAS SAML method, and using an LDAP control in eDir 9.x where there is a LDAP Proxy as User control. I.e. I geoffc, can bind to LDAP as 'admin'.  (Actually I think, Admin (ID APps process) can bind to eDir over LDAP as geoffc, without knowing geoffc's password)

    However the Auto tweek is not really a tweek it is about knowing how it works.

  • Hi Sir,

    Would like to know is there any permanent fix for the "Auto" Tweak?

  • Just did an IDM 4.6 to IDM 4.7 upgrade, same solution, thanks!
  • Sir, you get a 5 star for that "Auto" tweak.
    Happens even when not using silent install.
    On 4.5.5. in catalina.out I got
    [http-bio-80-exec-2] ERROR com.netiq.idm.auth.oauth.OAuthFilter- [RBPM] Unable to initialize private key for SSO header signature.