Hi, I just wanted to say this is very nicely written. The like just was not enough.
Within any one of the rules covered above (Input Transform, Command Transform, etc.) are zero or more policies. Some of these may come from the “preconfigured” driver import used as a starting point. Others may be local customizations of the driver. These are where the magic of IDM happens.
A policy can be written in a language called DirXML Script. Or it can be written in XSLT. DirXML Script has a nice GUI front end called Policy Builder, which guides you and helps you. XSLT assumes that you know what you're doing, and doesn't provide much help at all. For almost anything that can be done in IDM, Policy Builder can do it (there are a couple of exceptions), and is faster, easier, and more maintainable over time. Other than the Input and Output transforms of the Delimited Text driver, and the parts of the other drivers like SAP/HR and SOAP that require it, I do not recommend working in XSLT.
So, what's in a Policy?
A Policy takes a general form of Rule: Conditions/Actions. More than one rule can be in a policy, and each rule has conditions that have to be met, and actions to be carried out when the conditions are true. The grammar of the conditions is meant to be human readable, and generally to make sense. So a condition of “if object class equal user” would be True if the object being described in the current document is a User object, and would be False if the object is a Group. These conditions can be combined with And and Or logic to build complicated questions, but reading them and a bit of thought should unravel even the most complicated groups of conditions in to something that makes sense.
Conditions are made up of Condition Groups. Within a Condition Group, all of the conditions can be combined with And or Or, and the result must be True for the Condition Group to evaluate to True, otherwise it evaluates to False.
Multiple Condition Groups can also be combined, using And and Or. When using multiple Conditions in a Condition Group, and using multiple Condition Groups, the And and Or are exclusive, so that if the Condition Groups are using And, then the Conditions within each Condition Group are using Or. Alternately, if the Condition Groups are using Or, then the Conditions within each Condition Group are using And.
Conditions and Condition Groups are where you ask the question "should I be doing something here?"
Once the Condition Group(s) have evaluated to True, then the Rule's Actions are performed. Actions are where you answer the question “what should I be doing about it?” after the conditions have been satisfied.
The Actions part of a rule consists of one or more things to go and do. Actions can act on the current document, and in many cases this is sufficient. But Actions can also query the source or destination (which could be eDirectory or the connected system, depending on which channel you are on) for additional information. Actions can change the current document in to a modified version of itself, or can block it entirely. Actions can be used to make different documents. A document describing a delete event in a connected system could be tested by a Condition (if operation equal delete), and acted on by a set of Actions to prevent the associated object from being deleted in eDirectory (veto), but to modify that object to remove its association value for this application (remove-association), and to disable it (set destination attribute value Login Disabled = True).
The Association value is IDM's way of keeping track of which object in the connected system matches up with (is associated with) an object in eDirectory. In almost all cases, this should be a 1:1 match, so that it is possible to say that "Herman Munster, employee number 1234567" in the HR system matches up exactly with the User object "hmunster13" in eDirectory, with "Munster, Herman" in Active Directory, and "email@example.com" in the email system.
Most computer systems have some sort of internal unique identifier, even if it's not the one that you usually see in the system's management tools. eDirectory and Active Directory have a “globally unique identifier” or GUID. Many HR systems have an employee number. Email systems usually have a unique email address value for each person. These are the values used to build the Association used by IDM.
Associations are stored in eDirectory only. On the Subscriber channel, the IDM engine uses this value to allow the shim to modify the correct object in the connected system. On the Publisher channel, the shim supplies the association value, allowing the IDM engine to quickly and easily find the correct object in eDirectory to work with. So picking a good association value is important. Fortunately, most of the preconfigured driver imports have already done this for you.
A document listing some of the preconfigured driver's default Association values is here: Open Call - IDM Association Values for eDirectory Objects
When a "modify" document gets to the Add Processor, if it does not contain an Association value that resolves to an actual object, the IDM engine does something interesting. It assumes that since you are trying to modify something that does not exist, obviously you want it to exist, so it attempts to create it for you. This is the Synthetic Add process, and it can happen on either the Publisher or the Subscriber channel.
The IDM engine takes the "modify" to figure out which object to work with. It then uses the Filter to query back to get all attributes that are available for that object that are set to Sync or Notify on the current channel. It then throws away the "modify" document and builds an "add" document to replace it. This "add" document is then forwarded through the Matching, Create, Placement, and Command Transform (on the Subscriber it also goes through the Schema Map and Output Transform).
This has interesting side effects, especially if you have any policies on the rules not named in that list, because they will not have executed on this "add" document. As an example of what not to do, if you were to create a rule on the Event Transform that looks for an object being created (if operation = add), and creates a data attribute (set destination attribute value Telephone Number = 555-1212) on that object, so that all newly created objects get a phone number, then a "modify" that goes through the Synthetic Add process will create a destination object with no Telephone Number. Why?
Because the Event Transform does not get a chance to work on the document. The document came in with an operation type of "modify", so the condition was evaluated to False. Then the engine turned it in to an "add", because the "modify" referred to a non-existent object. This is why this is a bad example, and why I said earlier that most of your policies will be on the Command Transform.
The other interesting side effect of this is that the business logic it enforces is that the id vault is the authoritative source for the existence of objects in the connected system. This is not required, as anything in IDM can be customized, but it can be very powerful as well.
As an example, if your IDM environment contained an HR system, eDirectory as the vault, and a connected Active Directory system, and your business rules are such that the Active Directory administrators can (technically) but are not supposed to (business rule) delete accounts from the Active Directory domain, what happens when they do (accidently, or not)?
In IDM, the way to handle this is that the Event Transform on the Active Directory driver's Publisher channel would be used to handle the "delete”" event, by blocking it. That does not stop the delete from happening, as the event is being described here and has already happened. But what it does do is leave the object in the vault in place. Except that it has an Association (see above) that references an object in Active Directory that no longer exists. If you handle that, again on the Event Transform, with a remove-association action, that cleans up the vault object nicely. Now the object in the vault represents the true state of affairs: the object does not exist in Active Directory, and has no association value for the Active Directory driver.
Some time later, the HR system sends a change to this person down its Publisher channel. Something that also would be updated in Active Directory, like the person's Surname. This event updates the vault object in eDirectory, which causes an event to be sent to the Active Directory driver's Subscriber channel (assuming that the MAD driver has Surname in its Filter).
The MAD driver then takes this "modify" event, and tries to process it. It gets to the Add Processor, and finds that the "modify" is not for an associated object. The "modify" event is converted to an "add" event. Assuming that all of the conditions are met (no matching objects found, create rule finds that there is sufficient information to proceed, etc.), then the shim gets an "add" document to create an object in Active Directory for this person (the one that was just deleted...). And it does. Because the business logic says that this person should have an account in MAD, and does not, IDM fixes it for you.
A merge happens when the IDM engine converts an "add" operation in to a "modify". This happens most commonly during an initial migration, as the migrate sends objects down a channel, and the Matching Rule finds an object it can use to link up with the object being migrated. This also happens frequently on the Delimited Text driver, since all events (by default) come through as "add" documents, but with a valid association, so that the engine can turn them in to "modify" documents.
What happens in a Merge is that the current document is discarded (again, like the Synthetic Add), and the Filter is used to query to both the connected system and to eDirectory for all values. Then the" setting for each attribute in the Filter is used to decide what to do with the data. The options include overwriting the source information with the information from the destination, overwriting the destination with the source, combining the two and updating both with the results, or doing nothing.