When working with the default policies for the Active Directory driver, one quickly notices that this driver calls out to several Java functions. This is true with both the package based version or the pre-configured XML driver export used up until the release of NetIQ Identity Manager 4

Overall, the Active Directory driver is one of the best examples of how to write good driver policy and it always puzzled me somewhat as to why this driver still needed to rely on an external java class. Of all the drivers that ship by default with IDM 4.0.x, few still do this.

As the Active Directory driver and its associated vendor supplied default driver policies date back to the introduction of DirXML-Script with Identity Manager 2.0, I can only guess that this is just a hangover from the product's early days. The Active Directory driver is also one of the most commonly deployed drivers and as such, it is frustrating that part of the logic is hidden in an opaque "black box" JAR file.

This article walks through my best guesses at the logic embedded inside.


From JAR to ECMAScript

First, a quick history lesson. NetIQ Identity Manager runs on Java. For a long time, if your needs grew beyond what XSLT stylesheets or DirXML Script could do, then your only real option was to write function(s) and classes in Java. Then the classes are bundled into a JAR file, and then the JAR must be copied to the correct class path. Only then would they loaded by the engine or remote loader (the next time they were restarted. Such a lot of effort, just to make additional functions available via XPath from policies.

The intention of the early editions of Identity Manager was that they were to be sold and implemented only by specialised IDM consultants, so custom functions in JAR files abounded. This posed problems when the time finally came to upgrade these installs. If the person performing the upgrade forgot to copy and non-default JAR files. This would prevent some of the drivers from starting correctly on the newly installed replacement server (as they could not resolve the references to the functions contained in this file). Same thing if the server crashed and was unrecoverable. The IDM drivers and users were replicated on another eDirectory server, but not any files like this JAR. Neither are uncommon scenarios.

One of the reasons for the introduction of ECMAScript in Identity Manager 3.5 was to eliminate these kinds of problems. ECMAScript objects are stored within eDirectory, just like policies and drivers. Adding or updating ECMAScript objects requires only a restart of the any drivers that make use of the functions within. ECMAScript is a standardized version of JavaScript (which is ubiquitous in web browsers). One can even call Java methods and classes from within ECMAScript. In addition, ECMAScript objects can be included in packages, offering a far more structured and manageable way to ensure that customers get updated versions and bug fixes to these functions.

As calling ECMAScript from IDM policy offers so many advantages, it makes excellent sense (where feasible) to convert any legacy JARs to ECMAScript.
NetIQ has already done this with another commonly referenced library (NOVLLIBAJC). This was ported to ECMAScript by Shon Vella from a JAR that Novell Consulting used in many of their deployments.

Thus far, it does not seem that NetIQ has any interest in replacing adutil.jar with ECMAScript; I think it would be a great move. One less file to remember to copy during patching and more importantly - less opaque policy logic.

So what is ADUtil.jar and does it do?

The file is named "ADUtil.jar" and is located in the lib subdirectory relative to the actual Identity Manager Engine or remote loader binary directory.
I am not sure when it was first introduced. An educated guess would be the introduction of the Active Directory driver as the logic it implements has always been necessary even back in the pre DirXML Script days of IDM 1.x. I can however confirm, that the functions within this JAR are still referenced and used in the current shipping AD driver packages in 4.0.2

From what I can tell, this Java class several contains functions which mostly perform time conversions. These functions come in pairs, where one function implements the reverse of the other. With each pair, a rule in an input transform calls one function and the corresponding reverse function runs from a rule in an output transform. These ease conversion between the different time and string formats used by Active Directory and eDirectory attributes. It is difficult to know exactly how these functions work, as there is no documentation.

The ADUtil.jar file is installed when you install the Active Directory files (either as part of the engine install, or as a connected system/remote loader install), it's also included with Designer so that one can properly simulate operations within the Active Directory driver. Updated versions of this JAR have also shipped as part of an AD Driver patch.

Known ADUTUIL.jar functions

I found the following whilst trawling through driver pre-configs. There may be more.

• translateFileTime2Epoch

• translateEpoch2FileTime

• translateTimeMap2eDir

• translateTimeMap2ADLenient

• fixLines

Can we substitute built-in DirXML Script tokens?

This is an excellent question. This suggestion has popped up several times in the forums (and in other cool solutions articles).

The DirXML-Script convert time token can convert between Microsoft FILETIME and eDirectory CTIME (which is what one pair of the functions do).

The existing function - using translateFileTime2Epoch()

<rule>
    <description>accountExpires: Convert to Identity Vault time format</description>
    <comment xml:space="preserve">The Identity Vault uses a 32 bit value to store certain time values while Active Directory uses a 64 bit time value. Reformat the 64 bit value to fit within the vault's 32 bit syntax.</comment>
    <conditions/>
    <actions>
        <do-reformat-op-attr name="accountExpires">
            <arg-value type="time">
                <token-xpath expression="jadutil:translateFileTime2Epoch($current-value)"/>
            </arg-value>
        </do-reformat-op-attr>
    </actions>
</rule>

Could be written using the convert time token.

<rule>
    <description>accountExpires: Convert to Identity Vault time format (Convert Time Token)</description>
    <comment xml:space="preserve">The Identity Vault uses a 32 bit value to store certain time values while Active Directory uses a 64 bit time value. Reformat the 64 bit value to fit within the vault's 32 bit syntax.</comment>
    <conditions/>
    <actions>
        <do-reformat-op-attr name="accountExpires">
            <arg-value type="time">
                <token-convert-time dest-format="!CTIME" dest-tz="UTC" src-format="!FILETIME" src-tz="UTC">
                    <token-local-variable name="current-value"/>
                </token-convert-time>
            </arg-value>
        </do-reformat-op-attr>
    </actions>
</rule>

However, quick tests Designer's simulator feature (or testing with a real AD driver) show that some of the conversions do not match. For example, if one feeds in some common edge case test values: like "9223372036854775807" (max FILETIME) and "0".

The only reasonable explanation must be that the function translateFileTime2Epoch() seems to know about some "magic numbers" that Microsoft uses which have special significance. Therefore, there is clearly a bit more to these functions than meets the eye.


In addition, the convert time token is aware of things like your local time zone, daylight savings, leap seconds and other wrinkles that often trip up time and date calculations. FILETIME itself sticks with pure UTC without any of the factors mentioned above.

Implementing all these undocumented special cases in DirXML Script would have added a lot of bloat and unnecessary complexity to the default AD driver preconfigs, so instead NetIQ wraps this logic in an opaque JAR.

Conversion between Active Directory FILETIME and eDirectory CTIME

function: translateFileTime2Epoch()
function: translateEpoch2FileTime()

As mentioned earlier, this pair of functions convert converts between:
Windows FileTime (Signed 64-bit integer with an interval precision of of 100-nanoseconds (also called ticks) and an epoch of Jan 1, 1601)
and
eDirectory CTIME (Unsigned 32-bit integer with an interval precision of 1 second and an epoch of Jan 1, 1970)

Actually, eDirectory for some time now has had the capability to store date/times internally as a 64-bit integer (still with an interval precision of 1 second and an epoch of Jan 1, 1970). However, few tools (even those from NetIQ) support this yet.

As mentioned above, translateFileTime2Epoch() performs a similar conversion to the token-convert-time in DirXML-Script.

However, that conversion is not aware of the magic numbers used by Microsoft in Active Directory. These are numbers re-purposed by Microsoft and have a special meaning in some situations.

Known Active Directory Magic Numbers

• 9223372036854775807

• 0

• -1

9223372036854775807 can also be written as 2^63 – 1, which makes this the largest signed integer that can be saved as a 64-bit value. Obviously, this is so far in the future (the year 28,814 AD if my calculations are correct) that Microsoft has determined that they can safely re-interpret this value as a magic number.

The real-world usage of magic numbers in Active Directory varies from attribute to attribute. The following list is not exhaustive.

• accountExpires: 9223372036854775807 or 0 are magic numbers, they means that in AD that the account should never expire. The value 9223372036854775807 is a logical choice for never expires as this represents a real date that is so far in the future it can be easily considered as never. The usage of 0 to also indicate "never expires" makes far less sense, but is an unfortunate hangover from the Windows NT era.

• pwdLastSet: 0 and -1 are magic numbers. However, Active Directory immediately interprets the value -1. Active directory replaces this with the current FILETIME. This means that -1 will never be sent to IDM (either via query/instance or in an add/modify publisher channel operation). The value 0 means that the user is forced to change their password on the next logon.

• lastLogon: The last time the user logged on (not replicated, per DC). 0 is the magic number here, it means that the last logon time is unknown. In other words, never.

• lastLogonTimeStamp: Approximately, the last time the user logged on (domain wide). No defined magic number, but an absent value means that the last logon time is unknown. In other words, never.

• lockoutTimeThe date and time (UTC) that this account was locked out. 0 is the magic number here, it means that the account is not currently locked out.

• badPasswordTime: The last time and date that an attempt to log on to this account was made with a password that is not valid. 0 is the magic number here, it means that the last time a incorrect password was used is unknown. In other words, never.

• maxPwdAge: (Delta attribute) The maximum amount of time, a password is valid for. This value combines with the value from pwdLastSet to determine when the password expires. 0 is the magic number here, it means that the password has already expired.

• minPwdAge: (Delta attribute) The minimum amount of time one must wait before one is permitted to set a new password. Combined with pwdLastSet to determine the first permitted date/time when a user could choose a new password. No magic numbers. 0 should stay as zero

Also, for some of these attributes, eDirectory indicates never or not enabled by clearing the attribute. Active Directory requires the same attribute to have a value, thus the need for these magic numbers. All of which make the conversion logic more complex.
Handling the scenario of a not set (in eDirectory) attribute is generally more suited to DirXML-Script. That is the approach used in the current driver policies.

When examining the list above and the various usage of magic numbers a pattern does appear. The current JAR only accounts for two magic numbers, it does not try to adjust the conversion logic depending on which attribute the supplied value came from.

Additionally, one needs to take into account other factors during the conversion, in particular date/times before and after each respective epoch.

Tests via simulator show that the function translateFileTime2Epoch() clips to zero any FILETIME values which fall before the unix epoch (FILETIME starts at 1601, CTIME starts at 1970) and also clips FILETIME values at the upper limit to 4294967294 (the second largest unsigned 32-bit integer possible).

For example: 9223372036854775807 (max FILETIME) converts to 4294967294 (the second largest possible value for 32-bit CTIME)

            <add-attr attr-name="accountExpires">
                <value type="string">9223372036854775807</value>
            </add-attr>
        </add>
    </input>
</nds>
AD402 :Applying policy: % CCadutil-test%-C.
AD402 :  Applying to add #1.
AD402 :    Evaluating selection criteria for rule 'accountExpires: Convert to Identity Vault time format'.
AD402 :    Rule selected.
AD402 :    Applying rule 'accountExpires: Convert to Identity Vault time format'.
AD402 :      Action: do-reformat-op-attr("accountExpires",token-xpath("jadutil:translateFileTime2Epoch($current-value)")).
AD402 :        arg-string(token-xpath("jadutil:translateFileTime2Epoch($current-value)"))
AD402 :          token-xpath("jadutil:translateFileTime2Epoch($current-value)")
AD402 :            Token Value: "4294967294".
AD402 :          Arg Value: "4294967294".
AD402 :Policy returned:
AD402 :
<nds dtdversion="4.0" ndsversion="8.x">
  <source>
    <product version="?.?.?.?">DirXML</product>
    <contact>Novell, Inc.</contact>
  </source>
  <input>
    <add class-name="User" qualified-src-dn="o=dirXML Test\ou=Users\cn=User1" src-dn="o=dirXML Test\ou=Users\cn=User1">
      <association>o=dirXML Test\ou=Users\cn=User1</association>
      <add-attr attr-name="accountExpires">
        <value type="time">4294967294</value>

Simulator tests also show that the known FILETIME magic number 0 is converted to a CTIME of 4294967295 (the largest possible value for 32-bit CTIME)

            <add-attr attr-name="accountExpires">
                <value type="string">0</value>
            </add-attr>
        </add>
    </input>
</nds>
AD402 :Applying policy: % CCadutil-test%-C.
AD402 :  Applying to add #1.
AD402 :    Evaluating selection criteria for rule 'accountExpires: Convert to Identity Vault time format'.
AD402 :    Rule selected.
AD402 :    Applying rule 'accountExpires: Convert to Identity Vault time format'.
AD402 :      Action: do-reformat-op-attr("accountExpires",token-xpath("jadutil:translateFileTime2Epoch($current-value)")).
AD402 :        arg-string(token-xpath("jadutil:translateFileTime2Epoch($current-value)"))
AD402 :          token-xpath("jadutil:translateFileTime2Epoch($current-value)")
AD402 :            Token Value: "4294967295".
AD402 :          Arg Value: "4294967295".
AD402 :Policy returned:
AD402 :
<nds dtdversion="4.0" ndsversion="8.x">
  <source>
    <product version="?.?.?.?">DirXML</product>
    <contact>Novell, Inc.</contact>
  </source>
  <input>
    <add class-name="User" qualified-src-dn="o=dirXML Test\ou=Users\cn=User1" src-dn="o=dirXML Test\ou=Users\cn=User1">
      <association>o=dirXML Test\ou=Users\cn=User1</association>
      <add-attr attr-name="accountExpires">
        <value type="time">4294967295</value>

Also, the pair of functions are not truly reversible, as there is a significant difference in both range of date/times that can be represented and also the difference in precision (seconds versus hundreds of nanoseconds).

Finally, as outlined in the list above - depending on the specific Active Directory attribute, some additional logic may be required. Looking at the default driver policies, this logic (where required) is implemented using DirXML-Script on an attribute-by-attribute basis after first transforming the raw timestamp using the relevant function.


Conversion of permitted login hours (between Active Directory and eDirectory)

function: translateTimeMap2eDir()
function: translateTimeMap2ADLenient()

These functions are aware of some inbuilt limitations/exemptions in the Microsoft and eDirectory representations and try to account for this.

Both eDirectory and AD have an attribute that indicates the permitted logon hours of the day/week for a user.

However, this is challenging because of the way this information is actually stored in the attribute differs between the two directories.

In Active Directory, this information is stored as a byte array – 3 bytes per day with 1 bit per hour (24 bits per day, 21 bytes in total). Therefore, the minimum precision is 1 hour.

In contrast, eDirectory uses 6 bytes per day and thus has a precision of a half-hour period (so you could say that a user can log in from 08:30 to 17:00 Monday to Friday). Windows is limited to setting 09:00 to 17:00 Monday to Friday (or 08:00 to 17:00 if lenient) in this same case.

Additionally, on the Active Directory side, there is a local time zone offset from UTC factor, which requires checking a local windows registry value that defines the offset from UTC (irrespective of local daylight savings time). As the engine can run on non-windows platforms, I cannot see how this logic could possibly be in this JAR.

These function seems to know how to translate between the Active Directory and eDirectory binary formats and seems to have some fuzzy logic that maps a half hour to full hour discrepancy.


Convert Line Endings

function: fixLines()

This function is used in the output transform to convert Unix style line endings to Active Directory style line endings for attributes like streetAddress which often have multiple lines. The reverse version of this function does not exist (as far as I can tell), as stripping carriage-returns (but leaving new-lines) is trivial to implement with pure DirXML-Script tokens.

Historically, there has been a lot of confusion related to this particular function. It was present in earlier pre-configs. However, at some point, it was replaced by a pure DirXML-Script solution. The irony is that replacement does not actually work!

Even the current Active Directory Default Configuration driver package (version 2.3.0.20131218144845) contains this not-actually-working rule!
There is a bug raised for this issue.

Disclaimer
The information in this article derives only from the following sources:

• Reviewing default Active Directory default driver policies that have shipped with Identity Manager.
• 
  Publicly available information regarding the relevant Active Directory and eDirectory attributes.
• 
  Simulating different input values to these functions, and then reviewing the trace to observe what has changed (Black-Box testing)