In part 1 of this series, Getting Started Building a SOAP Driver for IDM - Part 1 I discussed some of the things you need to get started building a SOAP driver. I was using the example of Salesforce.com (henceforth known as SFDC, since typing the full name is too much of a pain each time). In Part 1 I focused on how you might start connecting via SOAP to get a session ID,

In Part 2 of this series, Getting Started Building a SOAP Driver for IDM - Part 2 I discussed how you might process the results from SFDC after you submit a login request, and converting it into an <instance> document.

In Part 3 of this series, Getting Started Building a SOAP Driver for IDM - Part 3 I discussed how you might handle query events and their responses.

Once you have the basic components of login, queries, and query responses you sort of have most everything you need for the most simple of drivers. At least for migrating users in from the remote system.

But there is so much more to a driver than just these components, and lots of little details. Lets list off some of those details that you would need to cover, and see how many we can work through.

On the Subscriber channel, we need the ability to send <add> and <modify> events to SFDC. So far we have nothing implemented that will handle that. Not being able to write back to SFDC would make this kind of valueless, so we definitely need to handle this.

For the Publisher channel, we need the ability to get changes out of SFDC. Otherwise there is not a whole heck of a lot of value in the driver. Though, a Subscriber channel only driver (aka a push driver) can have some value, but in general bidirectional synchronization is so much more useful.

A subtlety that crosses all parts of the driver is managing attribute syntaxes. SFDC has only a few attribute syntaxes, in comparison to eDirectory. So lets talk about this issue first.

The SFDC syntax types of interest are basically:

strings

date

date/time

reference

multi picklist

Pseudo Booleans

The attributes I needed to manage fell into those basic categories. There are some interesting details when working with SFDC that may not be obvious. For example, SFDC really cannot handle multi valued attributes. The closest it can come is a multi picklist attribute, where multiple values can be stored, but in fact is stored as a semi colon (;) separated string in the data base. The good news is this makes it a lot easier to manage, as it appears the ordering does not really matter.

Date and date time are slightly different, as date holds just a date string, but date time includes the time of day as well as the day. The syntax was not totally obvious, and it turns out that the way Apex Explorer showed the date string was not how the database actually stores it, which is quite misleading as you are working on this sort of thing.

Reference is the equivalent to DN (Distinguished Name) syntax in eDirectory. The version of Apex Explorer I was using was broken, since it was supposed to be able to resolve reference attributes, when you right click on them, but alas it did not work. This made tracking down information in Apex Explorer hard, but otherwise did not matter.

The SFDC implementation had some goofy attributes that I called pseudo booleans. Really they were picklists, with two values, Yes or No, but were used as Booleans and I wanted to store them in eDirectory as booleans. But of course, need to map yes to true and no to false for these odd attributes.

For these types of attributes you probably want to convert them to DN syntax in eDirectory.

Now this is where the DirXML-ApplicationSchema would really come in handy, if it supported this information. It turns out that every driver is expected to implement getSchema() as part of the shim (or at least return an error saying "This driver does not support returning the application schema" or the like. In fact the SOAP driver does just this later thing. It returns the error message during the first driver start.) to populate the DirXML-ApplicationSchema attribute on the object.

As it happens, for fun, I implemented getSchema() for SFDC via Policy as well, and it takes a while, since SFDC has a LOT of schema to convert and deal with, but is not really useful.

This is because the typical data about schema, beyond naming is whether the attribute is multi valued, which is useful, as you can see in this article, about managing multi valued attributes with Active Directory, Generic Single-valued Schema Enforcement and it turns out that SFDC really does not have anything multi valued, the closest being multi-picklist, which is really just a semi colon (;) delimited string.

What would be great if getSchema() would populate more information about the remote schema, like syntax type, perhaps any size requirements (say an attribute that is sized from 1-64 characters as a string, or 32 bit integer). Then the engine could automate a lot of code we have had to write to make sure data is of acceptable syntax between systems.

This is an important step, to convert from SFDC syntax to eDirectory syntax in the Input Transform and the converse in the Output Transform.

If you do it there, with the Reformat Operational Attribute, then the token will handle Query events, Modify, Add, Instance, and whatever else events. This makes it very powerful and more over, easy to write.

Consider for a moment how you would handle the differences between an <instance> document, where the data looks something like:

<instance class-name='User'>
    <attr attr-name='SomeDNattr'>
        <value>12345678</value>
    </attr>   
</instance>

so the XPATH to get the current operational attribute would be
attr[@attr-name='SomeDNattr']/value

But on an <add> event it would be more like:

<add class-name='User' src-dn='something'>
    <add-attr attr-name='SomeDNattr'>
        <value>12345678</value>
    </add-attr>   
</add>

thus the XPATH to select the operational attribute would be more like:

add-attr[@attr-name='SomeDNattr']/value

For a modify event it would look something like:

<modify class-name='User' src-dn='something'>
  <association>something</association>
    <modify-attr attr-name='SomeDNattr'>
        <add-value>
           <value>12345678</value>
         </add-value>
    </modify-attr>   
</modify>

thus the XPATH to select the operational attribute would be more like:

modify-attr[@attr-name='SomeDNattr']/add-value/value

Thus building this logic on your own could be hard, whereas Reformat operational Attribute just magically does it for you! Cool token!

Now as discussed above, the DirXML-ApplicationSchema attribute does not get built by the SOAP driver shim, since it does not implement getSchema() though you could do it yourself in policy using the describeGlobal() and describeSObject() SOAP API functions.

If you did, you could probably use that to identify the various syntax types coming out of SFDC.

It is a fair bit simpler, to my mind, to just maintain a Global Configuration Variable (GCV) that is a List type, and just add to the list, all the names of the attributes that need to be converted.

Now I initially started (foolishly, in hindsight) doing these conversions in the Command Transform rules. So I started off my policy using the eDirectory namespace for the attribute names. For example, acmeSFDCLastModifiedById (What I called it in eDirectory) instead of LastModifiedById (what SFDC calls it DN reference attribute to the last modifier of a particular object).

When I later wanted to move it to the Input and Output transforms, I was lazy, and did not want to retype all the various attribute names (Since I had a good 20 or so of them) and more so, did not want to have to debug the inevitable typos that would ensue. SFDC attribute names are case sensitive, often have funny capitalizations, and if they are not part of the basic SFDC schema, end in __c. So Status might Status or Status__c depending on if the attribute was part of the base SFDC schema or not.

What I ended up doing was realizing that the Schema Map has all this information and it is actually really easy in XPATH to do the conversion when needed. In fact I thought it was interesting enough to dedicate an entire article to the topic: XPATH to do schema mapping rule

To summarize, if you have the schema map XML loaded up into a local variable called SCHEMA-MAP, then this is mostly all you need to do the conversions:

Convert eDir name to App name:

$SCHEMA-MAP/attr-name-map/attr-name[app-name/text()=string($current-node)]/nds-name/text()

Convert App Name to eDir name:

$SCHEMA-MAP/attr-name-map/attr-name[nds-name/text()=string($current-node)]/app-name/text()

Where $current-node is the attribute name you are trying to convert.

I find working in the eDirectory namespace in the driver easier (Since I usually am the one who made up the attribute names, and thus they make sense to me) than working the SFDC namespace (since those kooky folk make up the darndest names!) and these two little tricks make it really easy to manage that.

Heck it is even using the Schema Map data to map schema from one system to another! How cool is that? I did NOT reinvent the wheel!

Converting a reference (SFDC) syntax attribute to a DN (eDirectory) attribute is pretty easy in both directions, as long as you take care to have an Id value on every object. We called our acmeSFDCid which is also the association value.

In fact in hindsight, it probably would have been easier to just use the Resolve token, when given a reference value (a 18 character alpha numeric string that is the database Id in the Salesforce.com internal database. I got this great answer as to why there is a 15 and 18 character version of this value in SFDC at times: Salesforce.com Id attribute seems to have a 15 and 18 character value, whats the difference? ) to just ask the Resolve token to resolve from Association to DN, and vica versa as needed.

However, I instead chose to use the query token to find the object whose acmeSFDCid matches the value I have in the document, with the results in a nodeset local variable called QUERY, and then test if XPATH is true $QUERY/@src-dn which if true means we found a DN and can use it. If not, the object does not yet exist in our tree and we will have to drop it, since a DN syntax attribute cannot hold a blank value, nor anything other than a valid DN value.

Going the other way, sending data back to SFDC on the Subscriber channel through the Output transform, you could Resolve from DN to association to get the acmeSFDCid value, or else do the same query described before, but this time get the Source Attribute acmeSFDCid from the DN provided. If there is a value, send it on, if not, drop the attribute, since SFDC also requires a valid database ID to be sent or else it is an error condition.

Date was pretty easy to convert using the Convert Time() token. I did not worry too much about wrapping the day around the day/night boundary and time zones, and just simply took the simple date value from the CTIME value.

This is an easy Reformat Operation Attribute call, that looks something like this in the Input Transform:

<do-reformat-op-attr name="$current-node$">
	<arg-value type="string">
		<token-convert-time dest-format="!CTIME" dest-tz="UTC" src-format="yyyy-MM-dd">
			<token-local-variable name="current-value"/>
		</token-convert-time>
	</arg-value>
</do-reformat-op-attr>

Current-node is the name of the attribute being managed. I was looping through .//@attr-name which loops through all the nodes in the nodeset that have a attr-name=Something" XML attribute, and then reformatting that operational attribute.

You can see from this example that SFDC is using 2010-09-21 formatting for its date internally.

For Date Time format, it is a little more annoyingly formatted. But again the power of Convert Time comes to the rescue.

<do-reformat-op-attr name="$current-node$">
	<arg-value type="string">
		<token-convert-time dest-format="!CTIME" dest-tz="UTC" src-format="yyyy-MM-dd HH:mm:ss.000;" src-tz="UTC">
			<token-replace-all regex="Z" replace-with=";">
				<token-replace-all regex="T" replace-with=" ">
					<token-local-variable name="current-value"/>
				</token-replace-all>
			</token-replace-all>
		</token-convert-time>
	</arg-value>
</do-reformat-op-attr>

This one was a little trickier since the formatting in SFDC is something like:

yyy-MM-ddTHH:mm:ss.000Z

I am not sure what that T in the between the day and hour segment is for, but it is often there, so I did a replace all on the T with a space to clean it up, and then replaced the Z with a semi colon since the Convert Time token interprets the Z as a time zone reference, which does not seem to work the way I wanted it too.

Later I learned that in Convert Time, you can use `T` to escape a literal character to ignore, so I probably could have used a convert string of:

yyy-MM-dd`T` HH:mm:ss.000`Z`

which would have worked just as well.

Multipicklists I handled by splitting along the semi colon into a node set and adding each value to the eDirectory attribute, and when sending back to SFDC joining the nodeset of values with a semi colon separator did the trick.

Stay tuned for the next episode in this series where we start working through getUpdated() and writing back to SFDC with the upsert() function.