Someone posted on the forum a question about how to parse some REST driver responses into XDS. In some ways this is the point of the REST driver. In some ways it is not.

The thing is that REST is more of an API than a complete solution. The distinction I am trying to make is like the one between the JDBC driver and the Active Directory driver. In the case of the Active Directory driver, the basic object classes are well known and supported out of the box in the packages. Users, Groups, and OU objects. Passwords are handled standard. Most of the configuration being done is to implement business logic or silly requirements that everyone seems to have.

In the case of a JDBC driver, the table you are reading from is in no way standard. There is no defined set of columns, so you have to build that all up from scratch most of the time. The transport to look at an arbitrary table is JDBC and provided by the driver. It does the authentication and manages reading and writing, but you have to go figure out from what table, and how the columns map to attributes. Then you can use that data to implement your business logic.

The REST driver is shipped as a sort of hybrid. It is mostly the API transport approach like the JDBC driver, but the default packages include an attempt to parse the JSON that comes in, to XDS. This only works if the JSON model used is exactly the same one they wrote the converter to use. This is a Java class they include in the driver and is not modifiable. Thus it is rarely of any use at all.

Even more confusing, a REST endpoint can return a JSON or XML response. In the forums, someone posted an example of how they get back a response from a REST endpoint in XML and was looking for help in converting that to XDS. Once the data is converted, then it is just normal driver stuff to process it. I posted an answer in the forum, but it was quick and I was in a hurry. I thought it would be useful to use as an example and explain why what I did works, in a bit more detail, since there is useful ideas to learn.

Amusingly, this response was a REST endpoint from Workday.com a SaaS HR provider. The company I work for has a SOAP based driver designed for Workday and here is someone trying it via REST. I would be interested in seeing more of the REST interface to understand if it is a better fit than the SOAP interface or not. Here is the example response he posted that needed to be converted to XDS to work with the engine. I happen to NOT be a fan of XSLT so I like doing things in Policy (DirXML-Script) instead of XSLT where possible.

<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product build="20150722_0750" instance="Workday Rest Driver"
version="1.0.0.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<input>
<driver-operation-data class-name="DIRXML-USER" command="add"
event-id="Workday Rest Driver##11716235148##0" remote-host="7.204.58.32"
url="https://extrldev-identity.ABC.com/DIRXML-USER">
<response>
<header content-type="application/xml"/>
<value>&lt;ws:Additional_Information>
&lt;ws:UserId>Rf03677&lt;/ws:UserId>
&lt;ws:MailDrop>W0522&lt;/ws:MailDrop>
&lt;ws:DistList>2701_Floor2@ABC.com&lt;/ws:DistList>
&lt;ws:CFI_LRV_SO_Executive_ID>015004&lt;/ws:CFI_LRV_SO_Executive_ID>
&lt;ws:MailDrop_Desc>Floor 2_testforIDM&lt;/ws:MailDrop_Desc>
&lt;ws:Org_Unit>ABC Technology Services&lt;/ws:Org_Unit>
&lt;ws:Organization>Applications&lt;/ws:Organization>
&lt;ws:Business_Unit>Office&lt;/ws:Business_Unit>
&lt;ws:JobProfile_Desc>Engineer
6-E_testforIDM&lt;/ws:JobProfile_Desc>
&lt;/ws:Additional_Information></value>
</response>
</driver-operation-data>
</input>
</nds>

There are several things to notice in this response document. The first is that everything is held in the <driver-operation-data>. If you are familiar with the SOAP driver you will know they started using the <operation-data> node (where Operation Properties would be found as XML attributes of the <operation-node myOpProp="true"> as an example). The SOAP driver let you specify a new URL, a parent-node, and other configuration parameters to control what the SOAP shim itself would do. However operation data is not really meant for that, the engine is supposed to strip it off and NOT send it to the shim. Thus they later switched to a <driver-operation-data> node which is left behind or sent by the shim itself and persists through the transaction.

As you work with the REST driver you will see that you can control a number of parameters this way, similar to how the SOAP shim works.

I notice there is a <header> node and that is nice to see. I know I was trying to use REST against an API that returned data in the header and it was not passing that data back which made it a real problem. I wonder if this is fixed since I last looked. But the date stamp in the XDS is 2015 so I doubt it.

Next up we see the <value> node holds everything else. The data we want is XML, escaped as simple text, inside that node.

To parse this, the basic approach would be, get the contents of the <value> node into a variable. Then XML Parse it to a node set. Now we can use XPATH on it. Loop over the ws:Additional_Information node, and for each node underneath it, make a <modify-attr> node, where the attr-name XML attribute is the name of the node in the source data. Then XPATH out the value of each node into the <add-value> node. Or maybe we want an <instance> doc as a response to a <query> doc. Our approach in each case would be similar but slightly different due to the slight differences in how XDS looks in each of those event types. For simplicity, since I could use an Add Destination Attribute token, and not have to build the XDS by hand I did it that way. Do recall I wrote this answer as a quick throw away post in the forum, since I saw the question and thought it would be a fun thought experiment. The forums are fun that way. You ca throw out an idea, ill formed, but with a useful hint and others will jump on it and refine it.

Here is the rule I generated, then I will try and explain how it works.

<rule>
	<description>Parse REST Response in Policy Example</description>
	<comment name="author" xml:space="preserve">Geoffrey Carman</comment>
	<comment name="version" xml:space="preserve">1</comment>
	<comment name="lastchanged" xml:space="preserve">Feb 7, 2017</comment>
	<conditions>
		<and>
			<if-operation mode="nocase" op="equal">driver-operation-data</if-operation>
		</and>
	</conditions>
	<actions>
		<do-set-local-variable name="XML" scope="policy">
			<arg-node-set>
				<token-xml-parse>
					<token-xpath expression="response/value"/>
				</token-xml-parse>
			</arg-node-set>
		</do-set-local-variable>
		<do-for-each>
			<arg-node-set>
				<token-xpath expression="$XML/*/*"/>
			</arg-node-set>
			<arg-actions>
				<do-set-local-variable name="ATTR-NAME" scope="policy">
					<arg-string>
						<token-xpath expression="local-name($current-node)"/>
					</arg-string>
				</do-set-local-variable>
				<do-set-local-variable name="VALUE" scope="policy">
					<arg-string>
						<token-xpath expression="$current-node/text()"/>
					</arg-string>
				</do-set-local-variable>
				<do-add-dest-attr-value name="$ATTR-NAME$">
					<arg-value type="string">
						<token-text xml:space="preserve">$VALUE$</token-text>
					</arg-value>
				</do-add-dest-attr-value>
			</arg-actions>
		</do-for-each>
	</actions>
</rule>

First up, I get the data into a local variable named XML. Notice that is a node set type variable, since I want to use XPATH on it and loop over the nodes in it. The text inside the <value> node that we started with is escaped XML, so we need to parse from text to XML but also specifically to a node set. Thus I used the XML Parse token to do that work for me.

Finally the target, that provides the data to XML parse is selected via XML as "response/value". If you look back up at the initial event the trick with XPATH in a case like this is to understand where the current context resides. For years the docs had a single line that said the current operation node is the current context. This was true and correct but there is a lot of detail that really matters for it to make any kind of useful sense.

What that line says is, in the case of an <add> operation, you XPATH current context is the <add> node. Thus to get the dest-dn XML attribute, the XPATH is @dest-dn, since that is relative to the <add dest-dn="\TREE\o\ou\cn"> node. The key is, you do not start at the root of the XML document, so no need to XPATH as /nds/input/add/@dest-dn the /nds/input/add is assumed.

Thus looking back at our original input document you will see the child of the <input> node is the <driver-operational-data> node, and thus that is where we begin. We want the text in the resource/value node, so our XPATH is simply "resource/value".

Now that we have the information in a variable XML as a node set, we can try to loop over its contents.

The XPATH for that is: $XML/*/*

That says, loop over the variable XML (the dollar sign means it is a variable reference in XPATH. Confusingly, dollar sign, variable name, then another dollar sign (So would be $XML$ instead of $XML) is how you would replace a string with a variable in regular policy tokens like If Source attribute Surname equal $SURNAME$ or the like). Then the asterisk means every node and the second one means also every node, but the nodes under the first level of nodes.

The contents of the XML node set variable look something like the following:

	<ws:Additional_Information>
	<ws:UserId>Rf03677</ws:UserId>
	<ws:MailDrop>W0522</ws:MailDrop>
	<ws:DistList>2701_Floor2@ABC.com</ws:DistList>
	<ws:CFI_LRV_SO_Executive_ID>015004</ws:CFI_LRV_SO_Executive_ID>
	<ws:MailDrop_Desc>Floor 2_testforIDM</ws:MailDrop_Desc>
	<ws:Org_Unit>ABC Technology Services</ws:Org_Unit>
	<ws:Organization>Applications</ws:Organization>
	<ws:Business_Unit>Office</ws:Business_Unit>
	<ws:JobProfile_Desc>Engineer 6-E_testforIDM</ws:JobProfile_Desc>
</ws:Additional_Information>

Thus we loop over the contents of the XML variable, which is this content, so one loop, then over any node on the next child level, which is <ws:Additional_Information> and there is only one such instance. Finally the grand child level has all the nodes we want, but why not just loop over $XML/* instead of $XML/*/* and then we would have the Additional_Information node?

Well by looping over the asterisk, we loop over every node at this level. This is important, since we want to grab the name of the node, to use as the attr-name value, and the contents of the node as value.

Thus the next step is to do just that task. I used two variables, really only needed one, the ATTR-NAME variable, since when you do a Do Add Destination Attribute you can specify the name of the attribute as a string, or replace it with a variable (The double dollar sign format of $ATTR-NAME$ ) but you cannot really do XPATH at that point, so I need to grab the data into a variable first.

The XPATH here is really simple.

	"local-name($current-node)"

There are a bunch of fun XPATH functions that rarely get used, but every so often one of them is really useful. The example of local-name() is a good one. If you took the XPATH of $current-node/* you would get the name of the node, but that would have the namespace included in it. We could leave that there, and have the schema map properly convert the namespace as part of the attribute name. (Though I have never tried, I THINK that the Schema Map will treat the attribute name as a string so ws: is just part of the string not as a proper XML node name, where a name space has meaning)

But by using the local-name() function tells you the name of the node, minus the name space. This is wise to do, since the name space per se, does not matter, just that it be declared and used. In theory your target web service could change this on you and it should not matter. Using this function takes the name space out of the equation which is also nice.

The value of each node is stored in a variable, which was probably not necessary but made it seem balanced, and is also a simple XPATH of $current-node/text() which leverages the built in variable current-node which is used in the For Each token, as it loops over a node set, and the current node that is current in the loop is available in current-node. There is also current-value which is really just used in the Reformat Operation Attribute token. There you apply a reformatting pattern to the values. THere might be an add-value or perhaps 3 remove-value nodes, and 2 add-value nodes. The current-value variable lets you specify a way to reformat it that will be applied to all the values for the operational attributes in the document. THere is one more, current-op which contains the entire event document which Alex McHugh has written a great series on how you could leverage that for very clever uses.

Finally, since I was trying to handle a modify case, and I did not want to build the XDS XML by hand, I simply used the Add Destination Attribute token, passing in the ATTR-NAME variable as the attribute name, and the local variable VALUE as the value. I used the double dollar sign notation to make it consistent and balanced, it would have been the same to use the Local Variable noun token in Argument Builder.

This outputs

<nds dtdversion="3.5" ndsversion="8.x">
  <source>
    <product build="20150722_0750" instance="Workday Rest Driver" version="1.0.0.0">Identity Manager REST Driver</product>
    <contact>NetIQ Corporation.</contact>
  </source>
  <input>
    <driver-operation-data class-name="DIRXML-USER" command="add" event-id="Workday Rest Driver##11716235148##0" remote-host="7.204.58.32" url="https://extrldev-identity.ABC.com/DIRXML-USER">
      <response>
        <header content-type="application/xml"/>
        <value>&lt;ws:Additional_Information>
&lt;ws:UserId>Rf03677&lt;/ws:UserId>
&lt;ws:MailDrop>W0522&lt;/ws:MailDrop>
&lt;ws:DistList>2701_Floor2@ABC.com&lt;/ws:DistList>
&lt;ws:CFI_LRV_SO_Executive_ID>015004&lt;/ws:CFI_LRV_SO_Executive_ID>
&lt;ws:MailDrop_Desc>Floor 2_testforIDM&lt;/ws:MailDrop_Desc>
&lt;ws:Org_Unit>ABC Technology Services&lt;/ws:Org_Unit>
&lt;ws:Organization>Applications&lt;/ws:Organization>
&lt;ws:Business_Unit>Office&lt;/ws:Business_Unit>
&lt;ws:JobProfile_Desc>Engineer6-E_testforIDM&lt;/ws:JobProfile_Desc>
&lt;/ws:Additional_Information></value>
      </response>
    </driver-operation-data>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="UserId">
        <add-value>
          <value type="string">Rf03677</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="MailDrop">
        <add-value>
          <value type="string">W0522</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="DistList">
        <add-value>
          <value type="string">2701_Floor2@ABC.com</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="CFI_LRV_SO_Executive_ID">
        <add-value>
          <value type="string">015004</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="MailDrop_Desc">
        <add-value>
          <value type="string">Floor 2_testforIDM</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="Org_Unit">
        <add-value>
          <value type="string">ABC Technology Services</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="Organization">
        <add-value>
          <value type="string">Applications</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="Business_Unit">
        <add-value>
          <value type="string">Office</value>
        </add-value>
      </modify-attr>
    </modify>
    <modify class-name="DIRXML-USER" event-id="Workday Rest Driver##11716235148##0">
      <modify-attr attr-name="JobProfile_Desc">
        <add-value>
          <value type="string">Engineer 6-E_testforIDM</value>
        </add-value>
      </modify-attr>
    </modify>
  </input>
</nds>

To clean it up, I might strip off the driver-operation-data node. I could probably do it with XPATH, but simpler in a following policy I would test if the operation is drover-operation-data and just Veto it. Then you have a standard <modify> event that the engine can process.

You might want to add an <association> node or make a dest-dn XML attribute, it all depends on what your goal is for the driver.

If you have any nice examples like this, consider writing it up since it can be useful to reuse but also useful for learning how to this task.