In this series of articles I am trying to look at the One SSO Provider (OSP) which is used internally by NetIQ IDM to allow single sign on to the web interfaces IDM offers. This is a very interesting product in its own light, being used in a very narrow use case for IDM. Looking at the logs it generates you can gain considerable insight into how things work, which is immensely helpful when things break.

I hope that by excerpting some log entries that the next person seeing this in their log and not understanding it, will stand a chance if they paste a snippet into Google of finding this article.

If you happen to find a cool or interesting snippet in your OSP trace file, please write about it, and if you are feeling lazy, feel free to pass it on to me, or leave it in a comment.

Previously I looked at how during startup OSP reads the various keystores in use, and actually will trace out every single key loaded, which is great when debugging a crazy keystore problem. It will also show the JVM version which can help in some crazy cases as well.

One of the next interesting I saw as I noodled through a sample log file was the LDAP connection to eDirectory for user information being instantiated.

     <DataSource(uri.oidp.xml.config.01.2011)>:
      Id: idm_idv
      DisplayName: IDM eDir
      m_Class: com.novell.oidp.source.ldap.LDAPSource
      Subordinate Element: LDAPSource


     <AssertionAttributeMaps(uri.oidp.xml.config.01.2011)>:

      <AssertionAttributeMap(uri.oidp.xml.config.01.2011)>:
       Id: saml2-attr-map
       DisplayName: SAML2 Attribute Map

       <AssertionAttributeMapEntry(uri.oidp.xml.config.01.2011)>:
        Friendly Name: SAML IDP to Identity Vault mapping attribute
        Local Name: saml2-mapping-attr
        Saml Name: cn
        Remote Name Format: urn:oasis:names:tc:SAML:2.0:attrname-format:basic

     <AttributeGroup(uri.oidp.xml.config.01.2011)>:
      nId: 1
      Display Name: IDM Attributes
      Description: Attributes needed by IDM

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: Surname
       Display Name: Last Name

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: givenName
       Display Name: First Name

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: userDN
       Display Name: Distinguished name

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: userCN
       Display Name: userCN

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: initials
       Display Name: initials

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: mail
       Display Name: mail

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: nrfMemberOf
       Display Name: nrfMemberOf

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: srvprvPreferredLocale
       Display Name: srvprvPreferredLocale

      <AttributeGroupEntry(uri.oidp.xml.config.01.2011)>:
       Local Name: cacheable
       Display Name: cacheable

We see above that we are getting 'cn' as configured in configupdate.sh sent to use by the IDP (Which in this case is actually NAM, as we will see later when we look at how it parses the metadata). OSP looks for cn in eDirectory to match the value it got from NAM to link the object NAM's IDP says has successfully logged in, to an object it knows how to identify. Then it traces out the attributes it will request via LDAP from eDirectory.

Now we see part of the SAML Method being configured. There seems to be a lot of repetition in the log, which I assume is an artefact of using the ALL log level. That is, since everything is logging to ALL I would bet the same action might pass through two or more slightly different functions that both log, so we get slightly different copies of much the same thing.

     <AuthMethod(uri.oidp.xml.config.01.2011)>:
      Id: saml2-method
      DisplayName: SAML2 Method

      Class Ref Id: saml2-class
      Type: USER_AUTHENTICATE
      Enabled: true

     <AuthContract(uri.oidp.xml.config.01.2011)>:
      Id: saml2-contract
      DisplayName: SAML2 User Login
      Enabled: true
      Base URL: idm:login:user:saml2
      Show Password Expired UI: false
      Check Trust Levels: true
      Remote Contracts: false
      Trust Level: 2
      Timeout: 10
      Endless Loop Detection Window (millis): 2000
      Endless Loop Detection Threshold: 5

We can see that there are a number of settings we have no control over here, which is ok, since we probably do not care, but it does mean if one of these appears to be something you would need, it is likely that NetIQ engineering can easily add support if it turns out to be needed, since OSP has it, it is just not exposed to the user interface.

This can be really useful to help speed along an issue. You could imagine a situation where your network guys inform you that you need to expand one of these time windows, due to some vagary of your network. Without knowing there were even settings, you could not really proceed with support to try to ask how to change them. As always remember that OSP is meant almost as a standalone NAM competitor, so it has a lot of functionality, but IDM is only using a very limited subset of that functionality.

Now we see a Contract being processed, with an AuthCard, more of the NAM model here. (Unless those are actually SAML/Liberty/Federation concepts, in which case, Federation model stuff here).

      <ContractExecutableList(uri.oidp.xml.config.01.2011)>:

       <ContractExecutableReference(uri.oidp.xml.config.01.2011)>:
        Ref Id: saml2-method

      <AuthCards(uri.oidp.xml.config.01.2011)>:

       <AuthCard(uri.oidp.xml.config.01.2011)>:
        Id: eIDPLogin
        Endpoint Url: null
        Endpoint Type(s): 
        UserAgent Type(s): 
        Group Ref Id: null
        Show: true
        Authentication Required: true

        <ImageSetList(uri.oidp.xml.config.01.2011)>:

         <ImageSetReference(uri.oidp.xml.config.01.2011)>:
          Ref Id: ospImageRef

     <Protocols(uri.oidp.xml.config.01.2011)>:
      Require Signed Authn Requests: false
      Require Signed Assertions: false
      Sign Authn Requests: false
      Use IDP Publishing: false
      Use IDP Discovery: false
      Single Logout Method: false
      IDP Publish Domain: null
      IDP Service Domain: null
      SP Service Domain: null
      Assertion TTL: 5

Again, I am yet to be certain of what all this means, but it does tell us that these are configurable settings, should they be needed, even if there is no interface to set them in the current version.

Require Signed Authn Requests and Require Signed Assertions is probably the biggest of the two, where some IDP's do require these. In those case, since only NAM is 'supported' your best bet to get it all working is to get the IDP to not require signing of assertions and requests. These seem like useful settings to expose to the ism-configuration.properties file as an enhancement request.

I ran into these when I was trying to get Shibboleth configured as the IDP. You can read more about the issue and how I fixed it here:
https://www.netiq.com/communities/cool-solutions/configuring-idm-4-5s-osp-talk-shibboleth-idp/

    <UserLookups(uri.oidp.xml.config.01.2011)>:


     <UserLookup(uri.oidp.xml.config.01.2011)>:
      Id: saml2-user-lookup
      DisplayName: Map SAML IDP user

      Matching Expression: =(saml2-mapping-attr,saml2-mapping-attr)

    <LocalizedStringList(uri.oidp.xml.config.01.2011)>:

     Decorator: SecondaryNamingAttrDisplayName

     <LocalizedString(uri.oidp.xml.config.01.2011)>:

      Locale: *
      Text: Email

Here we see how users are looked up by the OSP system in eDirectory. They use the saml2-mapping-attr (which earlier we saw is set to 'cn' in our example) which shows us that the name needs to be the same, as returned by the SAML assertion, as the name we define in the configupdate.sh. This was an issue I ran into with getting Shibboleth configured with a customer. The Shibboleth IDP was returning the name of the attribute as a proper SAML name, but we needed it as a FriendlyName that matched what we entered in configpdate.sh. You can read more about that issue in the same article as above:
https://www.netiq.com/communities/cool-solutions/configuring-idm-4-5s-osp-talk-shibboleth-idp/

We also see that Email is a Secondary attribute, even when not configured. That does make me wonder if you tell your IDP to release both the requested attribute (CN in our case) and email address as Email if that the secondary could help in cases where the first fails to match?

Again this seems like a useful configuration to expose, and possibly worth requesting an enhancement over.

    <Logout(uri.oidp.xml.config.01.2011)>:

     <RedirectUrl(uri.oidp.xml.config.01.2011)>:
      Url: https://www.acme.com/IDMProv/logout.do
      returnParamName: target
      final: false

     <RedirectUrl(uri.oidp.xml.config.01.2011)>:
      Url: http://localhost:8180/sspr/public/Logout
      returnParamName: logoutURL
      final: false

     <RedirectUrl(uri.oidp.xml.config.01.2011)>:
      Url: https://www.acme.com/nidp/jsp/logoutSuccess.jsp
      final: true
     hasFinal: 

Here we see some information abut the Logout settings. When you have federation in place, logging out becomes more complex than you would probably like.

It gets as silly as asking, what do you mean when you say Logout? Did you mean logout from this one application? But if you have SSO enabled, if you get redirected back to a page related to that application, you are as likely to get reauthenticated via SSO as anything else.

Did you mean logout of everything? (Clear SSO cookies/sessions and so on?) If so, what if you have say email that would like to persist, but logout of everything else?

These are known as first world problems, when you have SSO working so well, that you have a hard time logging out successfully.

You can see that this is chaining the logouts to first logout of IDMProv (User application) and then SSPR, and then finally the OSP IDP itself.

If you ever watch the load of OSP in the catalina.out log from Tomcat, you can see the result from this generated into a URL.

Look in the same directory as the osp log file, for catalina.out and look for this line, after a fresh restart:

2016-07-24 15:21:11,728 [localhost-startStop-1] INFO org.apache.catalina.startup.HostConfig- Deploying web application archive /opt/netiq/idm/apps/tomcat/webapps/osp.war

You will see it define all the OAauth clients that are installed, and then it will summarize the OSP configuration settings (It does look like this is no longer shown in catalina.out with later OSP patches, usually post 6.0.0.3 I think. Which is a shame, since it was useful, at low OSP Logging levels to have this info in catalina.out):

*[osp-conf]
2015-12-15T16:15:11.549-0500
Event:
Message: 
OAuth configuration properties:
	com.netiq.idm.osp.as.admins-container-dn: ou=ServiceAccounts,o=acme
	com.netiq.idm.osp.as.duplicate-resolution-naming-attr: mail
	com.netiq.idm.osp.as.naming-attr: cn
	com.netiq.idm.osp.as.scope: 2
	com.netiq.idm.osp.as.users-container-dn: ou=Active,ou=People,o=acme
	com.netiq.idm.osp.auth.pwd.expire.show: ********
	com.netiq.idm.osp.auth.pwd.expire.url: ********
	com.netiq.idm.osp.clients: [XML Node]
	com.netiq.idm.osp.forgotten-pwd-url: https://www.acme.com/IDMProv/jsps/pwdmgt/ForgotPassword.jsp
	com.netiq.idm.osp.ldap.admin-dn: cn=admin,o=acme
	com.netiq.idm.osp.ldap.admin-pwd: ********
	com.netiq.idm.osp.ldap.host: 10.1.1.4
	com.netiq.idm.osp.ldap.port: 636
	com.netiq.idm.osp.ldap.use-ssl: true
	com.netiq.idm.osp.localhost-auto-add: false
	com.netiq.idm.osp.login.method: saml2
	com.netiq.idm.osp.login.saml2.enabled: true
	com.netiq.idm.osp.login.saml2.mapping-attr: cn
	com.netiq.idm.osp.login.saml2.metadata-url: https://www.acme.com/nidp/saml2/metadata
	com.netiq.idm.osp.logout-urls: 
<RedirectUrl xmlns="uri.oidp.xml.config.01.2011" returnParamName="target">https://www.acme.com/IDMProv/logout.do</RedirectUrl><RedirectUrl xmlns="uri.oidp.xml.config.01.2011" returnParamName="logoutURL">http://localhost:8180/sspr/public/Logout</RedirectUrl><RedirectUrl xmlns="uri.oidp.xml.config.01.2011" final="true">https://www.acme.com/nidp/jsp/logoutSuccess.jsp</RedirectUrl>

	com.netiq.idm.osp.oauth-key-alias: osp
	com.netiq.idm.osp.oauth-key.pwd: ********
	com.netiq.idm.osp.oauth-keystore.file: /opt/netiq/idm/apps/osp_sspr/osp/osp.jks
	com.netiq.idm.osp.oauth-keystore.pwd: ********
	com.netiq.idm.osp.oauth-truststore.file: /opt/netiq/idm/apps/jre8/lib/security/cacerts
	com.netiq.idm.osp.oauth-truststore.pwd: ********
	com.netiq.idm.osp.oauth.accessTokenTTL: 120
	com.netiq.idm.osp.oauth.refreshTokenTTL: 2592000
	com.netiq.idm.osp.oauth.sessionTokenRevocationTTL: 172800
	com.netiq.idm.osp.sessionTTL: 1200
	com.netiq.idm.osp.ssl-keystore.file: /opt/netiq/idm/apps/osp_sspr/osp/osp.jks
	com.netiq.idm.osp.ssl-keystore.pwd: ********
	com.netiq.idm.osp.tenant.additional-hosts: 
<AdditionalHosts xmlns="uri.osp.xml.config.01.2011">
<Host>169.254.186.36</Host>
</AdditionalHosts>

	com.netiq.idm.osp.tenant.host: www.acme.com
	com.netiq.idm.osp.tenant.port: 8443
	com.netiq.idm.osp.tenant.protocol: https
	com.netiq.idm.osp.url.host: https://www.acme.com:8443
	com.netiq.idm.pwdmgt.provider: ********

I wanted to quote that whole section so you can get a feel for where the variables used in OSP come from.

You can see that the names, com.netiq.idm.osp.login.saml2.mapping-attr are truncated somewhere to the internal OSP variables we have seen so far.

But it is nice that it reports what is set in configupdate, which is useful to understand if you have perhaps edited the wrong ism-configuration.properties file. (There are possibly, as many as three on a server if you install it 'right').

The specific snippet I wanted to relate back the OSP log is this one:

	com.netiq.idm.osp.logout-urls: 
<RedirectUrl xmlns="uri.oidp.xml.config.01.2011" returnParamName="target">https://www.acme.com/IDMProv/logout.do</RedirectUrl><RedirectUrl xmlns="uri.oidp.xml.config.01.2011" returnParamName="logoutURL">http://localhost:8180/sspr/public/Logout</RedirectUrl><RedirectUrl xmlns="uri.oidp.xml.config.01.2011" final="true">https://www.acme.com/nidp/jsp/logoutSuccess.jsp</RedirectUrl>

You can see that this logout is the final concatenation of the earlier set of logout URL's we had in our list. It tries each of the logout URL's until it gets to the final one, where it stops.

This is important, since using federation can mean many actual sessions are instatiated and need to be properly closed. For example, you might be federating OSP with NAM (NetIQ Access Manager) in which case, if you just log out of OSP the NAM session might still stay active, and then the next visit (before it times out) to an OSP protected page will just redo the SSO process and let you back in. For each of the Idemtity Apps, you can log out of their individual sessions, but you also need to log out of OSP for the same reason.

I have had issues with this lohout URL not being generated properly before, and seeing the generated results in the catalina.out was just frustrating since I did not know where it was coming from. I could not find anywhere that had the actual string it was using, in a configuration file. Now I know, and can try and find the individual settings it is getting to use as a base for the URL so it can properly build the final value I require as an end result.

Once you try to login, assuming you are set to Username/password mode (even though the example above is for a SAML configuration) there is an attempted LDAP bind. As usual, since you are logging in with either username (CN) or email address an LDAP query is needed to find your proper DN, so that it can then take the password you entered and log you in with that pair of values.

Even in SAML federated mode, there will be an Administrative LDAP connection since it needs to look users up in LDAP based on the attribute the successful SAML Assertion returns to identify the user.

[OSP]
Time: 2015-12-15T17:02:38.188-0500
Level: TRACE
Java Execution:
 Class: com.novell.oidp.source.ldap.LDAPSource
 Method: <init>
 Line Number: -1
 Thread: localhost-startStop-1
Message: LDAPSource Configuration:
   <LDAPSource(uri.oidp.xml.config.01.2011)>:
    Admin User Name: cn=admin,o=acme
    Directory Type: 1
    Operation Timeout: 15000
    Idle Connection Timeout: 10000
    Install SAML Method: false
    Rebind Allowed: true
    Max Waiting Per Replica: -1

    <LDAPReplica(uri.oidp.xml.config.01.2011)>:
     Id: idvReplica0
     DisplayName: IDM Replica One

     Ip Address: 10.1.1.42
     Do SSL: true
     Maximum Connections: 31
     Port: 636

Here we see an Adminstrative connection to LDAP being made. This will allow OSP to search for your LDAP DN based on your CN or email address. This is nice tracing as we get ots of nice trace. The Admin User Name, so we know this is an admin login instance, some timeouts, the address, SSL choice and the port.

Interestingly we see a line about Install SAML Method. This is an interesting trace line, since it would not be a normal thing to do, in fact it is a once in an installation thing to do.

Originally there was just User Application, so it could do Username/password login and then store the password in memory and log you in to LDAP for use in Forms and Workflows. This was useful, since it meant the queries for data (Which PRD's are available to you, what data to populate in a form) are based on your eDirectory permissions. This is a clever way of avoiding building an entire rights model, since eDirectory has an excellent one built in. If you can read the data, as you, then you can use the data in a form. If you can see the PRD, you can run it.

This is why you need to block BC (Browse and Compare) rights to the eDirectory objects that represent PRD (Provisioning Request Definitions, aka workflows) and then grant access as needed to users, groups, or containers. Otherwise, everyone can see all the PRDs to run them.

However, now with IDM Home, Landing, Catalog Access, Access Review, SSPR, and User Application, there are many apps that need to share that login, so in fact, you never login to User Application with your password directly, even in Username/password mode. You login to OSP which then passes an OAuth ticket to User Application to let you in.

This is not an issue, except how can User Application authenticate an LDAP session to eDirectory on your behalf without credentails? Well it uses the NMAS (Novell Modular Authentication Services) SAML method. NMAS allows you to authenticate to eDirectory with things other than passwords, like biometric, smart cards, and even SAML.

Thus User Application is SAML federated to eDirectory. If you then SAML federate OSP to say NAM (NetIQ Access Manager) you end up with a login looking something like this:

Enter a URL to one of the IDM services web pages. As you are not authenticated, it redirects you to the NAM IDP web page to login via a SAML assertion. Once authenticated, it returns a SAML assertion that you are who you claim to be, with an identifying attribute value, but no password. Now OSP trusts your session.

You then redirect back to the web application you tried to connect to, which sees it does not have a session for you, asks OSP for an OAuth ticket, which is returned. Now you are connected to that application in an active session.

If that application was User Application it then further federates you to eDirectory over SAML. So there is a nice web of trust we weave, when first we try to login.

Anyway, configuring that SAML method to eDirectory is an option in configupdate.sh, and needs to be done once per install in a tree. (At least until they patch it in a new release). If you hit the Advanced Options button, then the middle tab, and in the RBPM section, there is a new drop down available that has an option to Auto create the SAML federation. I would guess it calls this function, with the noted value set to true when this happens.

In IDM 4.02 you had to configure this manually yourself and make three objects. This approach does it all for you. However, if you are looking for documentation on which objects are involved, go back to the 4.02 docs, since they mostly removed it in 4.5, as it is now done automatically for you. From a simpler doc perspective this makes sense. However from a troubleshooting perspective this is less than optimal.