NetIQ has a number of drivers meant to support the SAP Platform of applications. If you are getting started in this, I will warn you that SAP is probably the most complicated driver you will ever work on. Fun, but complicated.

I have written a fair bit about the SAP drivers, (there are many SAP drivers, User Module (UM), HR, Portal, GRC, and supporting drivers like Business Logic, ID, RR) and it may help if you are getting started to look at some of them if you need some help.

Start with this one to explain what the various drivers are, and when you would use each kind.

The SAP Driver Family for IDM

Then since the SAP HR is the most complex of the bunch, and usually the most interesting to use, there is lots of great material here:

• Troubleshooting iDOC Issues in the SAP HR Driver for Identity Manager

• IDM SAP HR Driver: Where to Get the JConnect Libraries

• SAP HR Driver and Organizational Management – Part 1

• Decoding iDOCs with the IDM SAP Driver

• Using the SAP HR Driver for IDM on a 64-bit OS

• Error Codes of the SAP HR driver for Identity Manager – Part 1

• SAP HR CMP Integration Driver

• Series: SAP HR CMP Integration Driver Walkthrough

Some of that stuff relies on the Business Logic driver to keep some stuff straight in the background and so there is this series.

• SAP Business Logic IDM Driver Walkthrough – Part 1

• SAP Business Logic IDM Driver Walkthrough – Part 2

• SAP Business Logic IDM Driver Walkthrough – Part 3

I know I left those hanging, because the versions I was working with, were then updated to newer ones by packaged versions, and most core functionality changed and I did not want to start again, and got busy so I never finished them.

A quick summary of the various drivers will help put some scope on this article I would think.

The HR driver is what you would expect. People get hired, fired, change jobs, name changes, etc, and you want that to flow to your Identity Vault. This aspect turns out to be simple and straightforward, with only vagaries of SAP to contend with. The Publisher channel is implemented by having the SAP system write out iDOC files (Intermediary documents) meant for data exchange and the shim picks up on them (like a Delimited Text driver) and converts them to XDS events.

However, that is where the simple part ends. There are actions in SAP that do not really relate to a single attribute change, and there are many actions in SAP that should all represent the same event to an attribute in eDirectory. Additionally the way future dated hires is handled is very tricky, and it is possible for a future event to be entered, changed, and all that needs to be handled properly, which turns out to be very tricky. The original SAP HR driver configuration did none of this, beyond simple attribute sync. The newer versions get more and more subtle and clever in how they handle it.

The CMP (Compliance Management Platform, which was IDM 3.6 plus some neat extra configuration stuff) version of the SAP drivers became the Packaged versions of the driver in IDM 4.0x.

The SAP UM (User Module) driver is a funny one. The thing about SAP is that it uses 'modules'.

• CRM - Customer Relationship Management

• HR - Human Resources

• CLM - Contractor Lifecycle Management

• BW/BODS/BOBJ - Business Warehouse, Business Object Data Services, Business Objects

• eRecruit - eRecruitment module for hiring

• SLM -

• Portal - Netweaver web portal

• CUA - Central User Administration. A UM to manage all the users in other UMs.

And many more I have never even heard of. On an amusing side note, the BOBJ/BODS stuff is Business Objects that you may be familiar with from the Crystal Reports successor or as the reporting part of Zenworks a few versions back. SAP bought Business Objects and 'integrated' it with SAP.

Anyway, each module needs users. The configuration options are fairly complex (as is EVERYTHING in SAP) and you can have one user store, called an ABAP User store, for each UM, or you can have all your UM modules point at the same ABAP user store. Or any number in between.

In my experience, customers seem to like many ABAP user store in the real world, and are resistant to using a single ABAP user store. Alas.

The granularity of the SAP UM driver from NetIQ allows two basic modes. One driver per UM, or if you are using CUA (Central User Administration) then one driver in fanout mode.

Even SAP realized that it was getting a bit silly, that a company that needed 10 different models would be maintaining ten different user stores, even though they could all point at one ABAP store and use but a single one. So they released the CUA which allows to manage users in all UM's from one UM (The CUA) but the CUA can only do users and permissions, it cannot do passwords. Which always makes me say, so what the heck is the point?

Well the UM driver is pretty clever. If you had 10 UM's you needed to support, each with its own ABAP stack, then you would need 10 drivers. Or you could get the SAP guys to implement CUA (UM #11 in this example) and connect one driver to the CUA system, and then use the driver in fanout mode, where a password change flows to CUA as a normal password sync event, then the driver config sends a password change to the configured child systems (10 of them in this example).

As you can see, tons of possibilities to configure this, and lots of needless complexity if you want it.

The next module is the SAP Portal module which was acquired from Netweaver, and has its own user store, called a UME. This is yet another totally different system, and there is a driver just for this one.

To make this fun, the Portal instance can be configured to use the ABAP user stack, its own UME user stack, I think even an LDAP user store, or via SAML anything you can authenticate via SAML.

Regardless, if you are using Portal, and it has its own user stack, then the driver will happily sync what you need. In this case, the API is SOAP, using the SPML variant mostly standardized. (This is SAP, I have no evidence that it is not 100% standard, but I would bet a donut there is some variant. This is SAP after all! It can't be that simple!)

Thus the SAP Portal driver is actually a SOAP driver with a built in SPML configuration. It is kind of a hybrid driver, in that some of the data transformation from XDS to SPML happens in stylesheets and policies and some of it happens inside the shim. Which makes for strange troubleshooting, and inconsistent data in the trace.

For example, you send an XDS <add> event to the shim, and internally it is converted to SPML carried in SOAP. But if you send some kinds of queries a stylesheet manipulates the query into a SPML query and you send in SOAP XML not XDS to the shim. Thankfully, this is the case, as I found a great example where I could fix it. For the stuff inside the shim, you need NetIQ to fix it.

The SAP GRC driver I do not know too much about, so I will defer till I learn more about it.

I was working on a Portal driver for a customer, and all testing looked good, but when we started to go live we found a number of existing Portal users did not properly match and we got the error below:

[12/26/13 09:59:39.518]:port ST:
DirXML Log Event -------------------
     Driver:   \ACME_IDM\IDM\Services\driverSet\SAP Portal
     Channel:  Subscriber
     Object:   \ACME_IDM\IDM\Users\Empl\kingm
     Status:   Error
     Message:  Code(-9063) Object matching policy found an object that is already associated: IDM\Users\Empl\kingm2.

But our user being migrated was kingm, not kingm2? What is happening? Why would we find the user kingm2 who is already associated, when we wanted to match kingm. This feels like we somehow mismatched? What went wrong? As always, we turn to the trace file.

Looking through the trace files, I found in the subscriber channels Matching policy that we issue the following query as the result of a Find Matching Object token (Note this snippet is after the Schema Map has fired, as you can see from the class of sapuser and the attribute of logonname. I guess I could have taken the snippet earlier, but you get the idea)

[12/26/13 09:59:39.310]:port ST:
<nds dtdversion="4.0" ndsversion="8.x">
  <source>
    <product edition="Advanced" version="4.0.2.2">DirXML</product>
    <contact>Novell, Inc.</contact>
  </source>
  <input>
    <query class-name="sapuser" dest-dn="kingm" event-id="0" scope="entry">
      <search-class class-name="sapuser"/>
      <search-attr attr-name="logonname">
        <value naming="true" timestamp="1355372979#116" type="string">kingm</value>
      </search-attr>
      <read-attr/>
      <operation-data timestamp="1388069979303"/>
    </query>
  </input>
</nds>

Parsing that XDS query, it looks good. Find me a sapuser (the User object class equivalent in SAP Portal) whose logonname is kingm. So far that is perfectly normal and looks exactly as we would expect. Of course knowing what to expect is half the battle (ignoring the advice I learned from GI Joe as a child, watching cartoons).

So where does this go wrong? Well, this query looks good, until the XSLT stylesheet NOVLPORTB-ots-SPMLOutputTransform converts that event into an SPML query:

[12/26/13 09:59:39.315]:port ST:        Applying XSLT policy: % CCNOVLPORTB-ots-SPMLOutputTransform%-C.
[12/26/13 09:59:39.315]:port ST:          Cxsl:message -> Output (searchRequest): Convert XDS to Portal SPML for className
            sapuser
[12/26/13 09:59:39.315]:port ST:          Cxsl:message -> special case for all users
[12/26/13 09:59:39.315]:port ST:        Policy returned:
[12/26/13 09:59:39.315]:port ST:
<nds dtdversion="4.0" ndsversion="8.x">
  <source>
    <product edition="Advanced" version="4.0.2.2">DirXML</product>
    <contact>Novell, Inc.</contact>
  </source>
  <input>
    <spml:searchRequest derefAliases="neverDerefAliases" requestID="search-1388069979303" xmlns:spml="urn:oasis:names:tc:SPML:1:0">
      <searchBase type="urn:oasis:names:tc:SPML:1:0#GenericString">
        <spml:identifier type="urn:oasis:names:tc:SPML:1:0#GenericString">
          <spml:id>kingm</spml:id>
        </spml:identifier>
      </searchBase>
      <filter>
        <and>
          <equalityMatch name="objectclass">
            <dsml:value xmlns:dsml="urn:oasis:names:tc:DSML:2:0:core">sapuser</dsml:value>
          </equalityMatch>
          <substrings name="logonname">
            <initial>*</initial>
          </substrings>
          <substrings name="logonname">
            <initial>kingm</initial>
          </substrings>
        </and>
      </filter>
      <spml:attributes>
        <attribute name="logonname"/>
      </spml:attributes>
      <operation-data parent-node-1="searchResponse" timestamp="1388069979303">
        <return-to-me class-name="sapuser" command="query" dest-dn="kingm" event-id="0" scope="entry"/>
      </operation-data>
    </spml:searchRequest>
  </input>
</nds>

That looks ok I guess. Sort of odd to do an equalityMatch on the objectclass of sapuser, but a substrings on first asterisk (*) then for one that starts with kingm (The initial XML node means the substring starts with). But hey, if I do not understand, lets see what happens before passing judgement.

This gets submitted to the shim, which should remind you of a SOAP driver if you did not notice it yourself.

[12/26/13 09:59:39.319]:port ST:        Stripping operation data from input document
[12/26/13 09:59:39.319]:port ST:        SAP Portal: Value of boolean flag 'remove-existing' is : false
[12/26/13 09:59:39.320]:port ST:        SAP Portal: HTTPSubscriberTransport.send()
[12/26/13 09:59:39.320]:port ST:        SAP Portal: Preparing HTTP POST connection to http://server01.acme.com:50200/spml/spmlservice
[12/26/13 09:59:39.320]:port ST:        SAP Portal: Setting the following HTTP request properties:
[12/26/13 09:59:39.320]:port ST:        SAP Portal:   Authorization: <credentials suppressed>
[12/26/13 09:59:39.320]:port ST:        SAP Portal:   X-Requested-With: XMLHttpRequest
[12/26/13 09:59:39.320]:port ST:        SAP Portal:   SOAPAction: #batchRequest
[12/26/13 09:59:39.320]:port ST:        SAP Portal:   Content-Type: text/xml; charset=utf-8
[12/26/13 09:59:39.321]:port ST:        SAP Portal: Cookie: saplb_*=(J2EE26240220)26240250; JSESSIONMARKID=1FunjgQ-cDRcl29TjfLE6k5qVeRRf8elzBQvpkkAE; JSESSIONID=1KSOHFtSIgGmK8XhLjooB71THk0vQwH6ZJAB_SAPemWRgzANjBiDC0gdc2N1zdHt
[12/26/13 09:59:39.342]:port ST:        SAP Portal: Did HTTP POST with 807 bytes of data to http://server01.acme.com:50200/spml/spmlservice
[12/26/13 09:59:39.410]:port ST:        SAP Portal: Setting cookie: JSESSIONMARKID=UU6ftgApu1WMACMEBjkr05q9MRSVSofGCJfIQvpkkAE; Version=1; Path=/
[12/26/13 09:59:39.410]:port ST:        SAP Portal: Response code and message: 200 OK
[12/26/13 09:59:39.411]:port ST:        Restoring operation data to output document
[12/26/13 09:59:39.411]:port ST:        SubscriptionShim.execute() returned:
[12/26/13 09:59:39.412]:port ST:
<nds dtdversion="2.0">
  <source>
    <product build="20100202_131201" instance="SAP Portal" version="3.6.1">Novell Identity Manager Driver 3.6.1 for SAP Portal</product>
    <contact>Novell, Inc.</contact>
  </source>
  <output>
    <searchResponse requestID="search-1388069979303" result="urn:oasis:names:tc:SPML:1:0#success" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
      <searchResultEntry xmlns="urn:oasis:names:tc:SPML:1:0">
        <identifier type="urn:oasis:names:tc:SPML:1:0#GenericString" xmlns="">
          <id>USER.PRIVATE_DATASOURCE.un:kingm2</id>
        </identifier>
        <attributes xmlns="">
          <attr name="logonname">
            <value>kingm2</value>
          </attr>
        </attributes>
      </searchResultEntry>
      <operation-data parent-node-1="searchResponse" timestamp="1388069979303">
        <return-to-me class-name="sapuser" command="query" dest-dn="kingm" event-id="0" scope="entry"/>
      </operation-data>
    </searchResponse>
  </output>
</nds>

That looks like it returned an id node with kingm2 in it. That looks wrong, I wanted kingm, not kingm2. This is like the query had been done for kingm* (with a wildcard character at the end, so kingm2 was found).

The Input transform has an XSLT Stylesheet to convert these responses back to XDS as follows:

[12/26/13 09:59:39.414]:port ST:        Applying input transformation policies.
[12/26/13 09:59:39.414]:port ST:        Applying XSLT policy: % CCNOVLPORTB-its-SPMLInputTransform%-C.
[12/26/13 09:59:39.414]:port ST:          Cxsl:message -> Input: (searchResponse) Convert SPML to XDS
[12/26/13 09:59:39.415]:port ST:          Cxsl:message -> searchResults for
            sapuser
[12/26/13 09:59:39.415]:port ST:          Cxsl:message -> processing
                sapuser id
                USER.PRIVATE_DATASOURCE.un:kingm2
[12/26/13 09:59:39.416]:port ST:        Policy returned:
[12/26/13 09:59:39.416]:port ST:
<nds dtdversion="2.0">
  <source>
    <product build="20100202_131201" instance="SAP Portal" version="3.6.1">Novell Identity Manager Driver 3.6.1 for SAP Portal</product>
    <contact>Novell, Inc.</contact>
  </source>
  <output>
    <instance class-name="sapuser" event-id="0" src-dn="SPML.SAPUSER.kingm2" xmlns:ncs="http://www.novell.com/nxsl/java/com.novell.ncs.dirxml.utilities.Utils">
      <association>SPML.SAPUSER.kingm2</association>
      <attr attr-name="logonname">
        <value>kingm2</value>
      </attr>
    </instance>
    <status class-name="sapuser" event-id="0" level="success" xmlns:ncs="http://www.novell.com/nxsl/java/com.novell.ncs.dirxml.utilities.Utils"/>
  </output>
</nds>

Well that is clearly wrong. We did not want a substring match, we wanted an equality test. Looking at the SPML XML that the stylesheet generated we can see that equalityMatch is the right node name, since we do that for the object class part of the query anyway. (Which is nice, having an example to steal code from, makes it easier than trying to find a reference document for SPML somewhere, and more specifically the exact SPML implementation that SAP uses)

Thus the problem becomes how to change from substrings to equalityMatch?

Well lets look at XSLT. I was using the Packaged version, package:

SAP Portal Base Package 2.0.0.20120510185127

When you are not sure if something is working, first run it through Designer's Simulator to see if it works the same as the engine, then you have a way to try and test and work on it. So I tried it, and got the following:

SAP Portal Driver :Applying XSLT policy: % CCNOVLPORTB-ots-SPMLOutputTransform%-C.
SAP Portal Driver :  Cxsl:message -> Output (searchRequest): Convert XDS to Portal SPML for className
            sapuser
SAP Portal Driver :  Cxsl:message -> special case for all users
SAP Portal Driver :Policy returned:
SAP Portal Driver :
<nds dtdversion="4.0" ndsversion="8.x">
  <source>
    <product version="?.?.?.?">DirXML</product>
    <contact>Novell, Inc.</contact>
  </source>
  <input>
    <spml:searchRequest derefAliases="
                        neverDerefAliases
                    " requestID="search-" xmlns:spml="urn:oasis:names:tc:SPML:1:0">
      <searchBase type="urn:oasis:names:tc:SPML:1:0#GenericString">
        <spml:identifier type="urn:oasis:names:tc:SPML:1:0#GenericString">
          <spml:id>kingm</spml:id>
        </spml:identifier>
      </searchBase>
      <filter>
        <and>
          <equalityMatch name="objectclass">
            <dsml:value xmlns:dsml="urn:oasis:names:tc:DSML:2:0:core">sapuser</dsml:value>
          </equalityMatch>
          <substrings name="logonname">
            <initial>
                                                    *</initial>
          </substrings>
          <substrings name="logonname">
            <initial>kingm</initial>
          </substrings>
        </and>
      </filter>
      <spml:attributes>
        <attribute name="logonname"/>
      </spml:attributes>
      <operation-data parent-node-1="searchResponse">
        <return-to-me class-name="sapuser" command="query" dest-dn="kingm" event-id="0" scope="entry"/>
      </operation-data>
    </spml:searchRequest>
  </input>
</nds>

Look in the <spml:searchRequest> node and note the escaped character. This is a bug in the package, some left over carriage returns that I have reported and I expect we will see new package versions to resolve this. This should not have mattered since white space is supposed to be ignored, but the escaped white space, will not be ignored.

But it helps us isolate where in the XSLT this is firing,

Looking at the XSLT you can see that the first search for <initial>*</initial> but with the extra spacing is earlier around lines: 156-164 as:

	<xsl:choose>
		<xsl:when test="$classname = 'sapuser'">
			<xsl:message>special case for all users</xsl:message>
			<substrings name="logonname">
				<initial>
*<!--  <xsl:value-of select="association"/>* -->
				</initial>
			</substrings>
		</xsl:when>

But it is the iteration through search-attr that does the trick, around line 187-211 as:

<xsl:for-each select="search-attr">
	<xsl:choose>
		<xsl:when test="$classname = 'sapuser'">
			<substrings>
				<xsl:attribute name="name">
					<xsl:value-of select="string(@attr-name)"/>
				</xsl:attribute>
				<initial>
					<xsl:value-of select="string(.)"/>
				</initial>
			</substrings>
		</xsl:when>
		<xsl:otherwise>
			<!--  <equalityMatch> -->
			<substrings>
				<xsl:attribute name="name">
					<xsl:value-of select="string(@attr-name)"/>
				</xsl:attribute>
				<initial>*</initial>
				<!--  <xsl:value-of select="string(.)"/>-->
				<!--  </equalityMatch> -->
			</substrings>
		</xsl:otherwise>
	</xsl:choose>
</xsl:for-each>

You can see from the commented out section, that it was once equalityMatch but was changed to a substring for some reason. Wish a comment had been left to explain why. Comments are so very helpful. But in fact it was the non-sapuser case that used to have equalityMatch. When it is really the sapuser case that matters. So using the example we have in the code, lets change it to:

<xsl:for-each select="search-attr">
	<xsl:choose>
		<xsl:when test="$classname = 'sapuser'">
			<equalityMatch>
				<xsl:attribute name="name">
					<xsl:value-of select="string(@attr-name)"/>
				</xsl:attribute>
				<dsml:value xmlns:dsml="urn:oasis:names:tc:DSML:2:0:core">
					<xsl:value-of select="string(.)"/>
				</dsml:value>
			</equalityMatch>
		</xsl:when>
		<xsl:otherwise>
			<!--  <equalityMatch> -->
			<substrings>
				<xsl:attribute name="name">
					<xsl:value-of select="string(@attr-name)"/>
				</xsl:attribute>
				<initial>*</initial>
				<!--  <xsl:value-of select="string(.)"/>-->
				<!--  </equalityMatch> -->
			</substrings>
		</xsl:otherwise>
	</xsl:choose>
</xsl:for-each>

Run that through simulator, and that works to convert an XDS query for a specific name to an SPML query for an equalityMatch for the logoname, which is really what we wanted. I am unclear why this issue is still there in the code, but it is an easy fix, and reported to be fixed in a later package release I hope.

One of the things that bugs me about this shim is that it is actually a SOAP driver with some stylesheets embedded in it, that you cannot see that do the magic inside the shim. I find this approach very bothersome, since had this issue been inside the shims set of stylesheets, there would be nothing I could do to fix it myself. Additionally this shim is very confusing as some XDS is converted internal to the shim to SPML and some XDS needs to be converted in Policy. This leaves you with a mix of stuff that is somewhat confusing. There is another example in this shim, where this causes confusion. The password handling internally converts oldpassword and newpassword attributes, if present to a password change, which is nice, but is unclear how it is working, since you cannot see it. But that is the topic for a whole other article.

Anyway, hopefully this will help someone who runs into this or a similar issue. By seeing trace samples, and examples of how to identify the problem, test, and resolve it I hope it helps in other cases as well. Let me know if you have similar examples, I would love to see them.