Have you ever wanted a workflow form to be able to query for something in one of your drivers? Maybe you want to pre-fill form data, or maybe you want to validate data entered without storing it within the identity vault.

The IDVault object in IDM is used to query the IDVault only, but recently I had a client who had a need from a workflow to query a connected system to verify that the user had a valid record before they were created in the ID Vault. This provided assurance that data in the vault would fully match up with data in the system of record, eliminating a lot of nasty processes surrounding resolving issues where the data was entered incorrectly.

The client proposed a totally separate JSP page, integrated with User Application, which would be made to look consistent with UA, and would do a separate query via JDBC, then make SOAP calls into the User App. In my mind, this was too much custom code, and it limited their agility. I set out to define a solution which could do this query from within a workflow form. This cool solution describes how I did it.

Main Components of Workflow

 
There are three main components of the IDM workflow engine:

• Forms

• Workflows

• The Directory Abstraction Layer

There are many possibilities within the workflow engine, as you can run almost any java class you could install on the server, and there are well defined mechanisms for integration with SOAP and REST web services. But on within the form, your options are more limited, because browsers are trying to protect you from “the bad guys”.

This creates a challenge for application developers who want to leverage external data sources, such as you need to within an IDM solution. The solution to this is a SOAP service called VDX. It is VDX's behavior that is informed and configured through the DAL. (The workflow engine can also use this component.)

The DAL allows you to abstract the technicalities of directory data based on its use case. You might have users in one container representing employees, and in another representing customers; the DAL can allow you to define this in a formal manner and determines which attributes are available to your forms and workflow.

But the VDX (which you access through the IDVault object in a form, and configure using the Directory Abstraction Layer or DAL) is only capable of providing that abstraction to the Identity Vault data. What if you want to query one of your connected systems, to provide validation, or to allow you to pre-load a provisioning form before the user is created in the vault?

What I have defined is an infrastructure such that from within a form, you can make a query to your connected systems.

Data Flow

While normal IDVault object methods will make a SOAP call to VDX, which then makes an LDAP call to the ID Vault, this approach uses the XML capabilities in the browser to form an XDS (XML for NDS) query which is then transmitted to the IDM engine. The policies in this driver will selectively query the connected system, and return results from that query (rather than from the Publisher query).

The policy functions to query the IDVault first, and responds with this data if it is found. If it is not found, the connected system is queried and its results are returned instead. Which system returned the query is identified by an XML attribute so that the form script can determine what action to take.

What's the JSP for? Understanding SOP.

 
The first attempts at this solution were successfully sending a query to the driver, and we could see the driver returning results in trace as well as in wireshark, but the results were never returning to the form script. I attempted to place the remote loader co-resident with the User Application but it had no effect.

A little research reminded of the cause: by RFC, all browsers implement a policy known as “SOP” . The origin of “Same Origin Policy” dates back to Netscape Navigator 2.0 (a distant ancestor of all browsers), and it means that for a web site to return data to your browser, it must be part of the same URL including the port number. So the data was returning but was being blocked by the browser.

This is something that has become more and more of a requirement and many sites do provide content in this manner using a few different strategies. The most current one is “CORS” or “Cross Origin Resource Sharing”. The challenge with implementing that within this solution is that it requires customization of the destination web server. Since the server in this case is the SOAP driver shim, based on JETTY, there is no simple way to make such a modification.

The solution was to implement a single JSP page to act as a rudimentary proxy from the User Application server to the SOAP driver and back again. This server is installed into the same web application server as User Application is, and is therefore at the same URL but at a different port. The SOAP driver has been configured to listen only on the well known loopback address 127.0.0.1 and at an arbitrary high port (I use 60609, the old Spiegel Catalog port). The JSP page takes HTTP requests and simply makes the same request to the SOAP driver, gets the results from the shim and returns them back to the browser.

The source code for this element of the solution is included below:

<%-- 
    SOP Circumnavigator 
    Copyright (C) 2015 Robert Rawson (for CIS, LLC)

    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation; either version 2 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License along
    with this program; if not, write to the Free Software Foundation, Inc.,
    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

    CIS may be contacted by eMail at service@cisus.com, by phone at (212) 577-6033 or by
    US Mail at 561 Seventh Ave, 13th Floor, New York, NY 10018

                                     - - -
    
    With due respect to the late Douglas Adams, SOP is not "Somebody Else's Problem" 

    This file is intended to circumvent the browser SOP (Same Origin Policy) in order to allow a 
    query to IDM from a browser form, specifically built for NetIQ's form engine. It depends on 
    a remote loader on the same server as UA set to listen on port 60609 connecting to a specially 
    configured SOAP driver. 
    
    To deploy this code, place this jsp file in the ROOT.war, and copy the file commons-httpclient-3.1.jar to 
    the ROOT.war/WEB-INF/lib directory. A web server restart may be required. 
    
--%> 

<%@page import="java.io.StringWriter"%> 
<%@page language="java" 
                        contentType="application/json;" 
                        pageEncoding="ISO-8859-1" 
                        import="java.net.*"%> 
<%@page import="java.io.*"%> 
<%@page import="java.util.*"%> 
<%@page import="org.apache.commons.httpclient.*"%> 
<%@page import="org.apache.commons.httpclient.methods.*"%> 


<% 
BufferedReader in = request.getReader(); 
String XML = ""; 
String line = null; 

while((line = in.readLine()) != null) { 
        XML = XML   line   "\n"; 
} 
try 
{ 
        // read data sent from the client 
        String strURL = "http://127.0.0.1:60609/"; 

        PostMethod post = new PostMethod(strURL); 
        try 
                { 
                System.out.println("[CIS]: Making request to SOAP driver"); 
                System.out.println(XML); 

                StringRequestEntity requestEntity = new StringRequestEntity(XML); 
                post.setRequestEntity(requestEntity); 
    
                post.setRequestHeader("Content-type","text/xml; charset=UTF-8"); 

                HttpClient httpclient = new HttpClient(); 

                int result = httpclient.executeMethod(post); 
                System.out.println("[CIS] Received response code from SOAP driver: "   Integer.toString(result)); 
                System.out.println(post.getResponseBodyAsString()); 
                
                PrintWriter writer = response.getWriter(); 
                writer.println(post.getResponseBodyAsString()); 
                response.flushBuffer(); 

                } 
        catch ( Exception e ) 
                {       
                System.out.print( "[CIS] e="   e.toString() ); 
                out.write( "{[ "   e.toString()   " ]}" ); 
                } 
        finally 
                { 
                post.releaseConnection(); 
                } 

        }        
catch (IOException e) 
        { 
        e.printStackTrace(); 
        } 

%> 

The SOAP driver

 
The SOAP driver is a very simplistic implementation. It does not provide any subscriber channel support at all, in fact all logic is contained entirely within the driver's input and output transformations.

The Input Transformation

 

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE policy PUBLIC "policy-builder-dtd" "/home/rrawson/designer/plugins/com.novell.idm.policybuilder_4.0.0.201410091552/DTD/dirxmlscript4.0.2.dtd"><policy>
	<rule>
		<description>[CIS] tag query dox</description>
		<conditions>
			<and>
				<if-operation mode="nocase" op="equal">query</if-operation>
				<if-xml-attr name="dest-driver-dn" op="available"/>
			</and>
		</conditions>
		<actions>
			<do-set-op-property name="orig-query">
				<arg-string>
					<token-time format="!JTIME" tz="UTC"/>
				</arg-string>
			</do-set-op-property>
			<do-clone-xpath dest-expression="operation-data" src-expression="."/>
		</actions>
	</rule>
</policy>

The first thing that the input transformation does is to make certain that only the query operation is permitted, and that only specific attributes we allow may be queried for. Following that, the main purpose of the input transformation is to clone the original query and place it in an operation-data element (an op-property) so that it will be available after it passes through the IDM engine.

The query is allowed to go to the engine and if the object is in fact found there, an instance document will be returned. For my use case, we wanted to use the ID Vault data if it is there, so in that case we indicate the source was the engine by placing the source DN of the SOAP driver in an XML attribute called from-driver-dn and then pass it back to the calling process.

The Output Transformation

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE policy PUBLIC "policy-builder-dtd" "/home/rrawson/designer/plugins/com.novell.idm.policybuilder_4.0.0.201410091552/DTD/dirxmlscript4.0.2.dtd"><policy xmlns:dircmd="http://www.novell.com/nxsl/java/com.novell.nds.dirxml.driver.cmd.DriverCmd">
	<rule>
		<description>[CIS] Tag instance doc from engine</description>
		<conditions>
			<and>
				<if-operation mode="nocase" op="equal">instance</if-operation>
				<if-op-property name="orig-query" op="available"/>
			</and>
		</conditions>
		<actions>
			<do-set-xml-attr expression="." name="from-driver-dn">
				<arg-string>
					<token-global-variable name="dirxml.auto.driverdn"/>
				</arg-string>
			</do-set-xml-attr>
			<do-strip-xpath expression="/nds/output/status"/>
		</actions>
	</rule>
	<rule>
		<description>[CIS] Separate dest-driver-dn from incoming XDS</description>
		<conditions>
			<and>
				<if-operation mode="nocase" op="equal">status</if-operation>
				<if-op-property name="orig-query" op="available"/>
			</and>
		</conditions>
		<actions>
			<do-set-local-variable name="xds-query" scope="policy">
				<arg-node-set>
					<token-xml-parse>
						<token-text xml:space="preserve"><nds><input/></nds></token-text>
					</token-xml-parse>
				</arg-node-set>
			</do-set-local-variable>
			<do-clone-xpath dest-expression="$xds-query/nds/input" src-expression="operation-data/query"/>
			<do-set-local-variable name="dest-driver-dn" scope="policy">
				<arg-string>
					<token-xpath expression="$xds-query/nds/input/query/@dest-driver-dn"/>
				</arg-string>
			</do-set-local-variable>
			<do-strip-xpath expression="$xds-query/nds/input/query/@dest-driver-dn"/>
			<do-set-local-variable name="Q" scope="policy">
				<arg-node-set>
					<token-xpath expression="dircmd:sendDriverCommand($dest-driver-dn, $xds-query/nds)"/>
				</arg-node-set>
			</do-set-local-variable>
			<do-set-xml-attr expression="$Q//instance" name="from-driver-dn">
				<arg-string>
					<token-local-variable name="dest-driver-dn"/>
				</arg-string>
			</do-set-xml-attr>
			<do-strip-xpath expression="/nds/*"/>
			<do-append-xml-element expression="/nds" name="output"/>
			<do-clone-xpath dest-expression="/nds/output" src-expression="$Q//output"/>
		</actions>
	</rule>
</policy>

If the query fails, the data from the op-property is fed back to the engine but sent to a driver as indicated by an extra XML attribute called source-driver-dn. The engine passes that to the target driver's subscriber and returns the result. If we have an instance document returned, we tag it with the DN of the driver it came from in from-driver-dn, and return it to the calling process.

Processing XML Documents Inside Forms

 
Modern browsers have support for both an HTML and XML Document Object Model (DOM). XML support is enabled by placing the following code within the global form onload event:

/** enable XML support **/ 
 function createXMLHttpRequest() { 
   try { return new XMLHttpRequest(); } catch(e) {} 
   try { return new ActiveXObject("Msxml2.XMLHTTP"); } catch (e) {} 
   form.alert("XMLHttpRequest not supported by this browser"); 
   return null; 
 } 

xmlhttp = createXMLHttpRequest(); 

I created some utility functions to assist with XML processing, these were added to an inline script:

function xmlParse(XML)
{
var oParser = new DOMParser();
var oDOM = oParser.parseFromString(XML, "text/xml");
return oDOM;
}

function xmlSetText(DOM, elementName, value)
{
var textNode = DOM.getElementsByTagName("value")[0].childNodes[0]
textNode.nodeValue = value;
}

function xmlSetAttribute(DOM, elementName, attributeName, value)
{
var attrNode = DOM.getElementsByTagName(elementName)[0].getAttributeNode(attributeName);
attrNode.nodeValue = value;
}
function getXMLResponse(form,field,xmlhttp,url,domRequest,callbackFunction)
{
txtRequest = new XMLSerializer().serializeToString(domRequest.documentElement);

xmlhttp.open("POST",url,true);
xmlhttp.setRequestHeader("Content-type", "text/xml");
xmlhttp.send(txtRequest);
xmlhttp.onreadystatechange= callbackFunction;
}
function xmlGetText(DOM, elementName)
{
var textNode = DOM.getElementsByTagName("value")[0].childNodes[0]
return textNode.nodeValue;
}

function xmlGetAttribute(DOM, elementName, attributeName)
{
var attrNode = DOM.getElementsByTagName(elementName)[0].getAttributeNode(attributeName);
return attrNode.nodeValue;
}

function ndap2ldap(ndapName)
{
var parts = ndapName.split("\\");
var ldapName = "";

for (var i = 0; i < parts.length; i  )
	{
	ldapName = parts[i] "," ldapName;	
	}
return ldapName.substring(0,ldapName.length-1);	
}

function xmlGetElementByAttribute(DOM, elementName, attributeName, value)
{
var textNodes = DOM.getElementsByTagName(elementName)
//Form.showDebugMsg("Found " textNodes.length.toString()  " GCVs")

for (var i = 0; i < textNodes.length; i  )
	{
	
	var attrNode = textNodes[i].getAttributeNode(attributeName);
	if (attrNode.nodeValue == value)
		{
		return textNodes[i]
		}
	}
return "";	
}

Although I could have created the base query document out of whole cloth using XMLHTTP methods, in order to make this easier to administer, I create the base document as a string and then convert it to a DOM. This code is in another inline script:

// Functions specific to this workflow
function queryDoc()
{
var XML = "<query dest-driver-dn='{driverDN}' class-name='User' event-id='0' scope='subtree'>"
XML  ="                <search-class class-name='User'/>"
XML  ="                <search-attr attr-name='{searchAttr}'>"
XML  ="                     <value>{userName}</value>"
XML  ="                </search-attr>"
XML  ="                <read-attr attr-name='CN'/>"
XML  ="                <read-attr attr-name='Full Name'/>"
XML  ="                <read-attr attr-name='Given Name'/>"
XML  ="                <read-attr attr-name='Surname'/>"
XML  ="                <read-attr attr-name='acmeASDaccountId'/>"
XML  ="                <read-attr attr-name='acmeAUVloginId'/>"
XML  ="                <read-attr attr-name='acmeAHCloginId'/>"
XML  ="                <read-attr attr-name='acmeASDsuppressInfo'/>"
XML  ="            </query>";
return XML;
}

The code to actually execute the query is broken into two event handlers. The event is initiated by the press of a button instantiated on to a text field called “queryVal”. This first event handle receives that button press and generates a query document. It then sets up a function to be called-back when the respnse is ready which publishes the results by firing another event who's name is onresponse-<field name>; this is picked up by the second event handler.

driverName="\\ASD JDBC" 
driverSet="\\ACMEDEV\\AcmeDev\\Servers\\DRIVERSET01" 

//this function fires when the response is returned. It fires an event which starts with onresponse-
function OnStateChange() { 
            if (xmlhttp.readyState==4) { 
		    field.fireEvent("onresponse-" field.getName(),xmlhttp.responseText) 
			} 
            } 
        
 
function userExists(attributeName, value) 
{ 
var domRequest = xmlParse(queryDoc()); 

xmlSetAttribute(domRequest, "query", "dest-driver-dn", driverSet driverName); 
xmlSetAttribute(domRequest, "search-attr", "attr-name", attributeName) 
xmlSetText(domRequest, "value", value) 
getXMLResponse(form,field,xmlhttp,"/queryNDAP.jsp",domRequest, OnStateChange); 

} 

hideSubmit(); 
form.clearMessages(); 

field.setValues("0","Searching, please wait...", false); 

field.show(); 
field.fireEvent("onSearchStart"); 

var num = '-1'; 
var attr = form.getValue("queryAttr"); 
var result = ''; 
var fullName = ''; 
var ppid = ''; 
var num = 0; 
var searchValue = form.getValue("queryVal"); 

if (searchValue != "") 
	{ 
	form.clearMessages(); 
	if (attr=="CN") searchValue=searchValue.toUpperCase() 
	userExists(attr,searchValue) 
	} 
else 
	{ 
	form.showError("Search value cannot be blank.") 
	} 

This second handler processes the response

function ud(va) 
{ 
if (undef(va)) 
	{return "(unvalued)"} 
else 
	{return va} 
} 
ldapUserContainer = "OU=Users,OU=Data,O=AcmeDev"; 
driverName="\\ASD JDBC" 
driverSet="\\ACMEDEV\\AcmeDev\\Servers\\DRIVERSET01" 

var queryResponse = new Array() 
var domResponse = xmlParse(event.getCustomData()); 

var instances = domResponse.getElementsByTagName("instance") 
var attrs = domResponse.getElementsByTagName("attr") 

var num = instances.length; 

if (instances.length.toString() > 0 ) 
	{ 
	queryResponse["from-driver-dn"] = xmlGetAttribute(domResponse,"instance","from-driver-dn"); 
	queryResponse["src-dn"] = xmlGetAttribute(domResponse,"instance","src-dn"); 
	if (queryResponse["from-driver-dn"].toLowerCase() !=  driverSet.toLowerCase() driverName.toLowerCase()) 	 
		{ 
		queryResponse["qualified-src-dn"] = xmlGetAttribute(domResponse,"instance","qualified-src-dn"); 
		queryResponse["ldap-dn"] = ndap2ldap(queryResponse["qualified-src-dn"] );	 
		} 

	for (var i=0; i < attrs.length; i  ) 
		{ 
		var attr = attrs.item(i); 
		var attrName = attr.getAttributeNode("attr-name").value; 
		var value = attr.getElementsByTagName("value")[0].textContent.toString(); z
		queryResponse[attrName] = value; 
		form.setValues(vacuum(attrName), value) 
		} 

	} 
else 
	{ 
	queryResponse["from-driver-dn"] = "NOT FOUND" 
	} 

if (queryResponse["from-driver-dn"] == "NOT FOUND") 
	{ 
	form.showError("No user found") 
	} 
else 
	{ 

	var subHead = "Existing"; 
	if (queryResponse["from-driver-dn"].toLowerCase() !=  driverSet.toLowerCase() driverName.toLowerCase())  //found in VAULT 
		{ 
		field.setValues(queryResponse["ldap-dn"], queryResponse["Full Name"], false);	 
		} 
	else // found in ASD 
		{ 
		var newObjectDN="CN=" queryResponse["CN"] "," ldapUserContainer; 
		subHead = "New" 
		field.setValues(newObjectDN, queryResponse["Full Name"], false); 
		field.fireEvent("NeedsPassword"); 
		} 
	field.fireEvent("onValidQueryResultsValue()",queryResponse); 
	var Head = "Create User" 
	var nbsp=" "; 
	var sep = nbsp "|" nbsp; 
	var br = "<br/>\n"; 
	var b1 = "<b>"; 
	var b0 = "</b>"; 
	var rowstart= "<tr><td>"; 
	var rowend= "</td></tr>"; 

	html = "<table>" 
	html= html   rowstart "<h3>" Head " - "   ud(queryResponse["Full Name"])   " ("   ud(queryResponse["CN"])  ")</h3></td><tr>" rowend; 
	 
	html = html   rowstart "<h4>" subHead " user:</h4>" rowend; 
	html = html   rowstart b1 "Last Name:" b0 nbsp queryResponse["Surname"] sep b1 "First Name:" b0 nbsp ud(queryResponse["Given Name"]) sep b1 "Info (FERPA) Suppress?" b0 nbsp ud(queryResponse["AcmeASDsuppressInfo"]) rowend; 
	html = html   rowstart b1 "Public Person Id:" b0 nbsp queryResponse["CN"] sep b1 "AUV Login Id:" b0 nbsp ud(queryResponse["AcmeEUVloginId"]) sep b1 "ACH Login Id:" b0 nbsp ud(queryResponse["ACH Login Id"]) sep b1 "Person Number:" b0 nbsp ud(queryResponse["AcmeASDaccountId"]) rowend; 
	html = html   "</table>" 
	field.fireEvent("htmlready-" field.getName(),html) 
	} 

field.select(); 
field.hide(); 

Securing the Solution

 
The one big caveat here is that this solution is an open web service that anyone could potentially use without authentication. To limit this exposure, the IDM driver policies do not permit anything but query commands and only to query for specific attributes, and to return specific data. Additionally, as this was crafted for s a regulated educational institution, the driver enforces the client's FERPA flag, which indicates whether users should be retrievable via search; if the user has elected to be protected then the search returns nothing.

Going forward, the intention is that the JSP page will be integrated with the User Application's SSO solution, either directly via Shibboleth or via OSP.

Conclusion

 
This approach is one way of solving the challenge of querying a connected system from within a workflow. Had this been a directory rather than a database, I might have considered using my JSP to communicate with a DSML server. Similarly, had this been a database which supports a web service, the JSP could have been used against that. But this solution has the unique characteristic that it extends the functionality of the NetIQ IDM Metadirectory Engine into the NetIQ IDM User Application workflow space, allowing a workflow to take advantage of what the IDM Engine has to offer.