New features of the Query token in Identity Manager 3.5.1

The release of Novell Identity Manager 3.5.1 added a number of great new features. Some are major new features, like Jobs, and the Work Order driver, other are simpler like the Unique Name, Time, Convert Time tokens. I have written about some of these before:

Work Order:
https://community.microfocus.com/cyberres/idm/w/identity_mgr_tips/3683/using-the-work-order-driver-in-idm
Time/Convert Time:
https://community.microfocus.com/cyberres/idm/w/identity_mgr_tips/3666/using-the-time-tokens-in-idm-3-5
Jobs:
https://community.microfocus.com/cyberres/idm/w/identity_mgr_tips/4272/identity-manager-new-feature-jobs
Engine Cache statistics:
https://community.microfocus.com/cyberres/idm/w/identity_mgr_tips/3421/driver-cache-stats-in-the-idm-3-6-imanager-plugins

Another token I am a huge fan of, which is new around the 3.5 time frame, is the Query token. In the past, you could do a Source Attribute and specify a DN, or a Destination Attribute and also specify a DN. But it was a bit hard to use those to find say, all users with an Employee ID of 12345.

For that you would need to call out in an XPATH expression to the Java query: class and use the srcCommandProcessor or the destCommandProcessor, depending on the direction of your query and the channel you are in.

Like many of my other favorite new tokens (Time, Convert Time, Unique Name and so on) the Query token is really just an interface wrapper around the existing Java class. But again, what a great wrapper it is!

Click to view.

You get a bunch of options. Source or Destination, so one token suffices for both.

Scope, which is powerful in terms of limiting your search to a single container (subordinates), subtree which would look at all objects recursivly, and entry, which would limit you to looking at a single value. This means you can tune your search to make it less costly and more efficient, if you know enough about what you are looking for and can limit it.

Next up is one that was new to me, and turns out be quite powerful! Max results. Initially I thought this meant only return the set number of results. For example if you are searching for all users with the Employee ID 12345, and you expect Employee ID to be unique, setting a max result count of 1 or 2 could make sense. (Of course, you would want to test and see if you got more than one back, since that should be an error condition you should be watching out for).

But actually it means something entirly different, and in fact, is much more powerful and better than I ever imagined!

What the max result count does is basically tell the Identity Manager engine to use a query-ex event instead of a query event. A query event will return an instance document with all the results in it.

A typical Query event would look something like this:

<nds dtdversion="3.5" ndsversion="8.x">

  <source>

    <product version="3.5.10.20070918 ">DirXML</product>

    <contact>Novell, Inc.</contact>

  </source>

  <input>

    <query scope="subtree">

      <search-attr attr-name="Employee ID"/>

      <read-attr attr-name="Employee ID"/>

    </query>

  </input>

</nds>

In this example we are querying the source (which seems to be the default, so the lack of specifying a direction implies the source) and we are looking at a subtree searching starting at the Root of the tree (since we did not specify where to start the search from). We are looking at all object classes, since again we did not specify anything else to limit it. Finally we will return the value of Employee ID since that is the attribute we are reading (read-attr node).

A query-ex event is a paged result set, so if you set the Max result count to 50, and your query for all users with a value for Employee ID, a query request would normally return all 10,000 users in your vault in one instance document, that could eat a lot of memory in the driver. The query-ex will specify the maximum result count as part of the query, and only return one 'page' of items at a time.

A typical query-ex event would look something like this:

<nds dtdversion="3.5" ndsversion="8.x">

  <source>

    <product version="3.5.10.20070918 ">DirXML</product>

    <contact>Novell, Inc.</contact>

  </source>

  <input>

    <query-ex class-name="Organizational Unit" max-result-count="50" scope="subtree">

      <search-class class-name="Organizational Unit"/>

      <read-attr attr-name="Employee ID"/>

    </query-ex>

  </input>

</nds>

Here above we have a query-ex event, with a max-result-count of 50, for a subtree search from Root. This will request 50 values in a page of results.

I won't show the entire Instance document that gets returned by this query, since showing 50 results that I have to hide the real employee numbers from would be kind of boring, but here is sort of what it will look like, showing a fragment from the end of the results set. (Note that the nds and output tag opening were of course at the beginnning of the document but I am not showing them for brevity).

    <instance class-name="Organizational Unit" qualified-src-dn="O=acme\OU=Users\OU=Active" src-dn="\ACMETREE\acme\Users\Active" src-entry-id="33585">

      <attr attr-name="Employee ID">

        <value timestamp="1187250893#11" type="string">E12345</value>

      </attr>

    </instance>

    <query-token>rO0ABXNyACxjb20ubm92ZWxsLm5kcy5kaXJ4bWwuZW5naW5lLk5EU1JlYWRlciRUb2tlbuWeJE0ga5xBAgACSgAFc3RhbXBJAAlzdGF0ZUhhc2h4cAAAARo1A2qGAdFxxw==</query-token>

    <status level="success"></status>

  </output>

</nds>

Note the query-token node at the very end of the returned document. Once this returns, there will be another query, for the next 50 items in a page.

The second query looks something like:

<nds dtdversion="3.5" ndsversion="8.x">

  <source>

    <product version="3.5.10.20070918 ">DirXML</product>

    <contact>Novell, Inc.</contact>

  </source>

  <input>

    <query-ex>

      <query-token>rO0ABXNyACxjb20ubm92ZWxsLm5kcy5kaXJ4bWwuZW5naW5lLk5EU1JlYWRlciRUb2tlbuWeJE0ga5xBAgACSgAFc3RhbXBJAAlzdGF0ZUhhc2h4cAAAARo1A2qGAdFxxw==</query-token>

    </query-ex>

  </input>

</nds>

The engine keeps track of these events, query-ex's, and query-tokens, so that it can properly return the correct data.

This can be really powerful if the target system is a bit slower, and large queries cause issues. It can help break up data sets into smaller chunks as well.

One of the additions to the Identity Manager 3.5 release was to get most of the connected systems drivers using query-ex instead of query events where possible, for better performance.

The next field is Class name, which alas, is limited to either a single object class to look for, or if you specify nothing for that field it is an implicit wildcard and returns all object classes in the results. I had hoped we could use something like User|Organization|NCP Server to try and search for three object classes, but alas the token does not support that.

However, it is worth remembering that you use the query token in Argument Builder, often inside a set local variable action, in which case, you can use three query tokens, to add all the results into one nodeset of a local variable. That is, set local variablem, and in Argument Builder, you would define your first query token to look for Employee ID on User objects, then add a second Query token to search for the same attribute on Organization objects (ok, stupid example, why would an Organization or NCP Server have an Employee ID value, but you get my point), and a third Query token to search for the same attribute on NCP Server objects.

This would return one large nodeset to the local variable, and the goal would be accomplished. (I have to thank Father Ramon for this one, I completely missed the possibility until he suggested it).

Our next field we can provide a value for is Select Object: That is, where to start the search from. The default is root of the datastore, which in eDirectory means Root and down. In other datastores it may have slightly different meanings. You can choose to set this to an Association value, or a DN value. I usually use a Global Configuration Value here for my User's container, since I am often looking for a value of something on a User object, and why waste CPU cycles and effort for the engine on searching areas I do not care about? But you have the flexibility to do pretty much whatever you need here. This is a nice pattern to see within the new tokens that have been coming out. The user is free to do whatever they need to do with it. Lovely!

The next two attributes are probably the most important. Match Attributes and Read attributes. Match is similar to how in the Matching rule, on either channel you define the criteria that will link the object in eDirectory with the object in the application, and the other way around (depending on the channel, Subscriber or Publisher, that you are talking about.).

Matching will help you identify the object you are querying for. That is, if I get an event from HR for a user with Employee ID of E12345 that is nice to know, but you really need a name in eDirectory's namespace, a Distinguished Name (DN), in order to start working with the event about the user. (Now in principle, as the event comes in directly for a change to that User in the HR application, the driver should have the association value to base that on, and provide the destination DN for you. But imagine you had some other reference to the user that is changing and you need to identify the User in eDirectory for the event. The actual example I have in mind is a Helpdesk application that is synchronizing Work Orders into eDirectory, to process and add entitlements. In that case, the Work Order has an object in eDirectory that the driver automatically synchronizes and provides or generates a destination DN. But the user it is targeting is only provided as an Employee ID. Thus you would need to find the DN of the User, based only upon the Employee ID value).

Conversely, you might want to find all Users who have a flagging attribute of some kind, that you need the list of.

Whatever the case may be, there are many times you will need to use the Matching element here.

Finally we get to the last field in the query definition, Read Attributes. This specifies what attributes to return. What is a smidgin frustrating, is that there is no such attribute of destination DN, or source DN, or the DN of the user. Alas, there you need to resort to an XPATH'ism.

What you would do is you set a local variable that generates a query such that it returns just your user, and you set the local variable as a nodeset. Then in the next action, you set the local variable (same one or a different one, doesn't matter) as a string, with an XPATH expression of $LOCAL-VARIABLE/@src-dn

I think a nice enhancement would be some kind of option or method to retrieve the DN of the target object as part of the query. But even without it, it is pretty easy.

New in IDM 4.8.3 is a @get-token="false" option, default is true, which means in a query-ex if true, do all the paging needed (Get the next page token).  If false, it only returns the first page, which is helpful.