Novell Identity Manager has a number of interesting drivers. Some are very specific, like the Lotus Notes (or Domino) driver, that just synchronizes to Lotus Domino servers. There are others that are more generic, like the LDAP driver, which can to talk to most any LDAP server. The JDBC is probably even more generic since in the LDAP case, LDAP is a standard for objects and usually meant for the special case of users, and a little bit more. Whereas the JDBC driver talks to databases, whose design and layout can literally be anything.

I used to think there were really only these two types of drivers from Novell. It turns out that there is yet another category, that the SOAP driver fits into.

The SOAP driver for Novell Identity Manager is something of a unique driver. It is more of a transport, and less of a traditional driver.

By transport I mean that the connection made to the application (connected system) is truly generic and the driver basically provides some basic connectivity over HTTP and almost everything else needs to be done in the driver configuration. If you are lucky and your SOAP connected system speaks DSML or SPML, then there is a somewhat defined schema for discussing users and events about users. In which case the default configs are pretty useful.

However it turns out that more likely the SOAP endpoint you are working on connecting too, uses its own schema (pretty much defined in the WSDL, as always not a perfect analogy, but close enough) and may or may not resemble anything else in the world. Once you let some developer define their own API or schema, you know they are going to do goofy things with it. Way more fun to create your own than use an existing standard!

The traditional approach is to manage the transformation of Identity Manager events in XML documents (XDS format) into the appropriate SOAP documents. I have been working on doing the whole thing in Policy (DirXML Script) and avoiding XSLT when possible. So far so good, and zero XSLT. Turns out to be pretty easy, since the incoming documents have an <nds> node then an <input> or <output> node which means the DirXML Script policy options can work on it.

The other driver that cannot be done without XSLT, is the Delimited Text driver. The primary reason for that is that the driver shim does not return the data it reads from the file, wrapped in an <nds> and <input> node. If they add just those nodes, which you could do in a simple XSLT, then you could most everything else in DirXML Script. In the case of the SOAP driver, they offer both options. Have the event documents be wrapped in XDS headers, or just the bare SOAP XML. If you get it wrapped, then DirXML Script is the way to go!

You can read more about the DirXML Script versus XSLT controversy in this article I wrote, looking for things you can only do in XSLT. I think I need to update it, to remove the SOAP driver from the list!

Open Call: What Can You Do in XSLT that You Cannot Do in DirXML Script?

One other interesting side affect of the SOAP driver acting more like a transport, is that it does not support query-ex, the extended query token, which my SOAP endpoint happens to support. Query-ex is used to do paged queries. Anytime you use the Query token in Policy, and specify a max result returned, you are using query-ex (if the driver supports it) and the second query will use a cookie returned in the results in the <query-token> node, to do the next query for the next set of values.

I can enable query-ex functionality in DirXML Script, based on queries and the results, but I cannot get the engine to acknowledge it, since when the driver starts up, it tells the engine if it supports query-ex (at the same time it tells the engine what driver it is, its version, and which license version it needs), and that is hard coded into the driver. I have a bug open asking for a configuration option that tells the driver to say it supports query-ex. Each SOAP endpoint is different, and how you map XDS query and query-ex events to the SOAP API will decide whether you can support query-ex or not.

In the driver I am currently working on, I had a Query event into the connected system, where I needed to know that I wanted something different to happen in the Input transform when the response was received. That is, there could be two different queries sent to the SOAP driver shim. They both return the same type of data, but I need to treat them differently in the Input transform. The traditional Identity Manager approach to that is to use the <operation-data> node to carry the information, and decide in the Input Transform, based on the payload of information carried by the operation data node.

You probably have seen nodes like this in most drivers that support Password synchronization. The rules in the Subscriber Command Transform in most of the drivers add some information into the operation data nodes to carry it through right up to the shim, where the engine strips it off (after the Output Transform), and it gets added back onto the event just before the Input Transform.

You can read more about the password transform rule sets, both channels in these articles:

• Password Transformation Rule Sets

• Password Transformation Rules in the Publisher Channel

Now the SOAP driver makes interesting use of the operation data node. It actually lets you use it to override driver settings, which is pretty cool.

This actually makes a lot of sense and is pretty useful, as scalable web sites that have SOAP endpoints, often have a URL you login to via a SOAP conversation, after which you get a session ID that you use in all further transactions. They might even return a different URL than you started with and expect all further conversations to occur on that second URL. This is probably pretty common (since I really have only worked with two different SOAP endpoints, I am not sure I can draw anything other than a straight line from that data), so it is quite useful. If you add a node that looks like:

<operation-data url="https://app.acme.com:48180/18/c/soap/">

Then the shim will over ride the configured URL and use the one provided in the operation data node. The docs mention that you might have a really distributed SOAP system you are connecting too, where the endpoint for creating a user might be on one host, and the endpoint for deleting the user on another. I think I would find that maddening to work with, personally.

The same approach to overrides applies to "method" and "soap-action" options. The example for method is you might set it to GET, as in an HTTP GET operation, and your SOAP endpoint might expect something else, so this is a configurable item on a per event basis.

The docs at least give examples of each of these cases. It would be nice if they described WHERE the <operation-data> nodes should actually go in the document.

However what I needed to do was use the operation data node to carry some information through on a <query> doc I was sending to the SOAP service.

Well that is easy, I know how to add operation data either through Policy (set operation property token in the list of Actions) or through manipulating the XML (also in policy, via the Append XML Element, and Set XML Attribute tokens).

I built my rules and all looked good. This meant copying the operation-data node onto the SOAP document and I left it as a sibling of the <soapenv:Envelope>, and a child of the <input> node.

Basically my query event looked like:

<nds dtdversion="3.5" ndsversion="8.x">
 <source>
   <product version="3.6.1.4427">DirXML</product>
   <contact>Novell, Inc.</contact>
 </source>
 <input>
   <query-ex class-name="Contact" max-result-count="2000" scope="entry">
     <read-attr attr-name="Id"/>
   <operation-data migrate="true"/>
   </query-ex>
 </input>
</nds>

My output transform made that look more like:

<nds dtdversion="3.5" ndsversion="8.x">
 <source>
   <product version="3.6.1.4427">DirXML</product>
   <contact>Novell, Inc.</contact>
 </source>
 <input>
   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
     <soapenv:Header>
       <urn:SessionHeader xmlns:urn="urn:enterprise.soap.sforce.com">
        <urn:sessionId>WV.AjOK</urn:sessionId>
       </urn:SessionHeader>
       <urn:QueryOptions xmlns:urn="urn:enterprise.soap.sforce.com">
         <urn:batchSize>2000</urn:batchSize>
       </urn:QueryOptions>
     </soapenv:Header>
     <soapenv:Body>
       <urn:query xmlns:urn="urn:enterprise.soap.sforce.com">
         <urn:queryString>Select  a.Id FROM Contact a WHERE (a.RecordTypeId='012200000004PcJAAU')</urn:queryString>
       </urn:query>
     </soapenv:Body>
   </soapenv:Envelope>
   <operation-data migrate="true"/>
 </input>
</nds>

You can see the operation data node at the bottom, inside the <input> node, as a sibling to the <soapenv:Envelope> node.

The response doc that comes back from the shim looks something like:

<nds dtdversion="2.0">
 <source>
   <product build="20090519_235451" instance="SOAP-SPML" version="3.5.4">Identity Manager Driver for SOAP</product>
   <contact>Novell, Inc.</contact>
 </source>
 <output>
   <soapenv:Envelope xmlns="urn:enterprise.soap.sforce.com" xmlns:sf="urn:sobject.enterprise.soap.sforce.com" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
     <soapenv:Body>
       <queryResponse>
         <result>
           <done>false</done>
           <queryLocator>01gT0000001Lex7IAC-2000</queryLocator>
           <records xsi:type="sf:Contact">
             <sf:Id>00320000000gXCuAAM</sf:Id>
           </records>
         </result>
       </queryResponse>
      </soapenv:Body>
     </soapenv:Envelope>
 </output>
 <operation-data migrate="true"/>
</nds>  

Here the operation data node came back as a sibling of <output>, which leaves it as a child of <nds>. Well that is not even valid according to the XDS DTD. What on earth is going on here. The first time the document is handled outside the Input Transformation policy set, the <operation-data> node is thrown away as it is not valid XDS XML, according to the DTD.

I spent a fair bit of effort trying to figure this out, worked on some XSLT (much to my chagrin, I was doing so well on zero XSLT in this driver) to try and copy the operation data node around.

Finally I came across the solution. It was actually in the documentation. But in quite an opaque wording.

5.5.1 Using Operation Data to Specify XML to Be Returned on the Result

The sample configurations for the SOAP driver use the <operation-data> element to keep track of identifying information for a command, so the result can be recognized and associations can be properly assigned. Check these samples for details of how the <operation-data> element is used.

When the <operation-data> element is restored on the response, it appended as a child element of the root node. You can override this by providing one or more parent-node-n attributes to the <operation-data> element, where n is a number beginning with 1 that is incremented for each parent specifier provided. The driver shim looks for parent-node-n attributes. When they are found, the attribute is checked to see if the named node exists. If the node is found, it uses as the parent for the <operation-data> element on the response.

This seems like a mix of a documentation bug, and a real bug. I find the documentation really opaque here, and think that even a single example would immensely clarify the issue. I hope this document will serve that purpose. I submitted comments on the documentation, and will submit some simple examples for them as well.

The SOAP driver has two modes in terms of how the shim returns the XML documents. There is a configuration value, ndsElementHandling that the docs describe as:

<nds>, <input>, <output> Element Handling

Specify Remove/add elements if you want the driver shim to remove and add the required XML elements <nds>, <input>, and <output>.

The required elements are removed from XML documents sent to the application and are added to XML documents received from the application before presenting the document to the Metadirectory engine. Otherwise, specify Pass elements through to turn off this element handling.

In the case of Pass elements through, then the raw SOAP XML is passed back, so adding the operation data node to the root of the document makes sense as a default. After all, where else could you add it? Well the root node is pretty much the only place you can guarantee will exist.

In the case where you use the setting to have an <nds>, <input>, and or an <output> node then the default should be changed, since the root of the document, <nds> is probably the wrong place to add the operation data node, since operation-data cannot be contained in the nds node, according to the DTD.

There is a bug submitted on this and we shall see what comes of it.

But after reading that page of the docs several times, I realized that if what you need to do is add an XML attribute to the operation data node.

Thus I was sending:

<operation-data migrate="true"/>

and should have been sending:

<operation-data migrate="true" parent-node-1="output"/>

Thus the resulting doc would look more like this:

<nds dtdversion="2.0">
 <source>
   <product build="20090519_235451" instance="SOAP-SPML" version="3.5.4">Identity Manager Driver for SOAP</product>
   <contact>Novell, Inc.</contact>
 </source>
 <output>
   <soapenv:Envelope xmlns="urn:enterprise.soap.sforce.com" xmlns:sf="urn:sobject.enterprise.soap.sforce.com" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
     <soapenv:Body>
       <queryResponse>
         <result>
           <done>false</done>
           <queryLocator>01gT0000001Lex7IAC-2000</queryLocator>
           <records xsi:type="sf:Contact">
             <sf:Id>00320000000gXCuAAM</sf:Id>
           </records>
         </result>
       </queryResponse>
      </soapenv:Body>
     </soapenv:Envelope>
     <operation-data migrate="true" parent-node-1="output"/>
 </output>
</nds>  

which changes the placement away from under the <nds> node and to be under the <nds><output> node. Now as it turns out, this makes it a child of <output> but a sibling of the <soapenv:Envelope> node, which is problematic for my usage, as I wanted to treat each soapenv:Envelope instance as a separate operation, and if this particular operation is a queryResponse document, AND the operation data says this is a migrate = "true", case then handle it one way. If the operation data is not there, treat it differently.

I was closer, but not one hundred percent there yet. So the next question is, if there is a namespace declared, how would you address the SOAP documents defined namespace.

A little testing shows that the parent node option in operation data is not quite working the way you would usually expect in an XPATH/XML way, and that just specifying the following would be sufficient to stick the operation data node in a better place to work with it.

<operation-data migrate="true" parent-node-1="output/soapenv:Envelope"/>

This returns a document that looks more like:

<nds dtdversion="2.0">
 <source>
   <product build="20090519_235451" instance="SOAP-SPML" version="3.5.4">Identity Manager Driver for SOAP</product>
   <contact>Novell, Inc.</contact>
 </source>
 <output>
   <soapenv:Envelope xmlns="urn:enterprise.soap.sforce.com" xmlns:sf="urn:sobject.enterprise.soap.sforce.com" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
     <soapenv:Body>
       <queryResponse>
         <result>
           <done>false</done>
           <queryLocator>01gT0000001Lex7IAC-2000</queryLocator>
           <records xsi:type="sf:Contact">
             <sf:Id>00320000000gXCuAAM</sf:Id>
           </records>
         </result>
       </queryResponse>
      </soapenv:Body>
     <operation-data migrate="true" parent-node-1="output/soapenv:Envelope"/>
   </soapenv:Envelope>
 </output>
</nds>  

Now this is a lot easier to detect with an XPATH expression of:

self::soapenv:Envelope/soapenv:Body/queryResponse 

which will detect that it is a queryReponse document type (since there is also a queryMoreResponse which needs to be handled slightly differently) and then a second XPATH condition in Policy of:

self::soapenv:Envelope/operation-data[@migrate="true"] 

which detects if this is a migrate case, based on the operation data. Which is what I was looking for all this time.

This way I can add some operational data payload to events to use on the other side of the shim. I do hope they change the driver shim to support changing the default node, when ndsElementHandling is enabled. Lets see if the docs get updated, since they are pretty unclear right now.