Another Toolkit Rule Use Example, Bad Attribute Value Cleanup
I have been writing a long series of articles (first four parts written, plans for the next 4 in hand, a usage example and now this article) about writing Toolkit rules in Novell Identity Manager.
You can read all about them at:
- Toolkit Rules in Identity Manager Part 1
- Toolkit Rules in Identity Manager Part 2
- Toolkit Rules in Identity Manager Part 3
- Toolkit Rules in Identity Manager Part 4
- Example of a Use for a Toolkit Rule in Identity Manager
with more to follow as they get published over the next couple of weeks.
I have written some other tutorial rules, walking through a problem or task or concept, which you can see here:
- How to GCVize a Driver, Part 1: Subscriber Channel
- How to GCVize a Driver - Part 2
- Password Transformation Rule Sets
The first two, how to GCV-ize a driver, takes the Active Directory driver, and modifies the default configuration that ships with IDM 3.5, and replaces all hard coded references to the container in Active Directory or eDirectory for user placement with Global Configuration Variables. Thus when you need to change the placement of users, you can change one value, instead of hunting down a dozen different instances where the string is used.
The third rule walks through the Command transform rules for Password handling that all drivers have as part of the default configuration. (As it happens I used the trace from an Identity manager 3.01 Lotus Notes driver in that one, but it is basically the same for most drivers). That is a great exercise for a beginner in Identity Manager, because it explains what the heck is going on with Password events in IDM. (Sometimes there is a node, sometimes there is a <modify-attr @name="nspmDistributionPassword> and yet depending where you are in the rules, one comes from the other!). Password events are non-obvious, and this article explains why you always need nspmDistributionPassword in the Subscriber channel set as Notify for it to work. Seems like an odd way to set the filter. Read the article for the explanation of why!
In the Toolkit rule series, I have been looking for really nice examples of how to leverage the notion. I started with the several examples I ran into at a client site. The case of the HR system giving me the Employee ID of a manager, and I wanted to store the Distinguished name (DN) of the manager in eDirectory, because of all the benefits DN gives me. I next moved on to talking about ACL compares.
Then I ran into a great example as someone posted a way to retrieve the Full Names of all users who are members of a group into a CSV file. Now he suggested you do it in VBScript I think, and I decided this would be a fun example to do in a Toolkit rule. So I did: Example of a Use for a Toolkit Rule in Identity Manager
I was at a client, and in the Development system I realized I had screwed up with a typo a while ago, and it was really bugging me.
I was working on setting Group Membership attributes (remember there are four involved in eDirectory, which is actually a GOOD thing! Lots of good articles on the why of the four attributes, if someone is interested in a focused article on it, let me know and I could whip it off!) and I did something dumb. I was setting the attribute Group Membership on Users and some other objects in the tree, inside a loop through the query results for all Users, like in all these toolkit rules. The dumb thing was, I used XPATH of @src-dn as the value to be added.
Look at that for a second and tell me what is wrong with it.
Well the @src-dn XPATH will return the Source DN of the current operational document. Well that document was triggered by a modify of our triggering attribute on a single user. Thus for every object in my query results, I added a Group Membership attribute with a value of a User object. That looks kind of dumb in the tree, where every object looks like it is the Member of a group that is really a User.
What I should have done, was of course an XPATH of $current-node/@src-dn which would be Source DN of the current node we are looping through. Oh well live and learn. Be fun to get comments from people of similar dumb things they might have done (just don't log in when you post it, so it comes across as anonymous in the comments to protect your good name!)
Well turns out eDirectory does not really care if you do that, it just looks really ugly and annoyed me. It was a development system, and in case you ever wonder why you do not develop in the production system, this is a great example!
However, having done the dumb thing. lets try the smart thing and fix it.
I could probably do this via LDAP, and that was my first approach. I thought, ok, I will query for all objects with a Group Membership attribute of cn=BSmith,ou=Users,dc=acme,dc=com in the filter, and then globally search and replace the modify to a delete or something like that. Turns out I could not get the filter to only return objects with that attribute. I should probably look at the schema definition, and figure out what the actual filter that would work is, but it gave me enough of an excuse to try a different approach.
Probably could have figured it out, but instead I decided to do it in a Toolkit rule, because I can reuse it for any arbitrary value in any arbitrary attribute in any arbitrary tree, which would be much more valuable than manually futzing around with LDIF exports. (I suppose I could have written a Perl script to convert the LDIF from one format to another as well).
This is a pretty easy thing in Novell Identity Manager.
Here is the simple overview:
Basically query for all objects with the attribute we care about, loop through the returned values, test for the cases that have the bad value, and then do something with it.
I will use the basic body of the toolkit rule developed in the previous set of articles, and expand it from there again.
The trigger is the same, I used 1234 as the value, could be anything you want as usual.
<if-op-attr mode="nocase" name="SomeTriggerAttribute" op="changing-to">1234</if-op-attr>
First thing that needs to be done is to set the bad value into a variable so we can test for it. Next, lets make it even more generic and set the name of the attribute it might be stored in, in a variable. This way, we should be able to reuse this rule with just two modifies to fix most problem cases! Change the value that is bad, and change the attribute the bad value is stored in. Heck we could probably just use a pair of Global Configuration Values as well. In that case, rather than edit the rule itself, we would edit the GCV's on the driver.
Normally I would be all for using GCV's, but in this case it would mean a second category of changes in order to use this rule to try and explain so I think I will wimp out and use the easier approach. But the difference in usage is that any time we say $VariableName$ we would say ~GCVName~ instead. Also instead of a token Local Variable, we would have used a token Global Configuration Value.
Now, to be fair, it is probably not possible to completely generalize this, since we would need to handle structured attribute syntaxes (Like Object ACL, used by Access Control Lists, or Path syntax used by DirXML-Associations. Take a look at this series on eDirectory schema syntaxes for a better understanding of what I mean:
- Interesting Schema Syntaxes in eDirectory from an Identity Manager Perspective - Part 1
- Interesting Schema Syntaxes in eDirectory from an Identity Manager Perspective - Part 2
If you have an approach to handling the VERY generic case of this value, anywhere, I would love to see it, as that would be a very powerful tool! I suspect we would need a custom approach for each syntax type. That might not be too bad to implement, and might be workable.
Until I see a better way, we need to store the bad value, and the attribute storing it in variables, that look something like this:
<do-set-local-variable name="BAD-VALUE" scope="policy">
<do-set-local-variable name="BAD-ATTR-NAME" scope="policy">
<token-text xml:space="preserve">Group Membership</token-text>
On to the next part of the rule, and we keep all the fun stuff for tracking time elapsed, and the counter for numbers found and removed, and all that jazz.
Our first major step as always is to get the list of objects we want to walk through in our for each loop. This time, I actually did not do it to just users, for a variety of reasons I had screwed up on the Organizational structure objects used in the SAP HR driver. Thus I actually had screwed up User, CommExec, Organizational Roles, and Organizational Units all at once. If you are going to screw up, screw up big time I say!
Thus the query starts small with Organizational Role objects, since there are only 700 of those in my example tree, which keeps the scope small enough for testing.
<do-set-local-variable name="CURR-BAD-LIST" notrace="true" scope="policy">
<token-query class-name="Organizational Role" datastore="src">
We store the values in the local variable CURR-BAD-LIST (note the notrace="true" since that is a big performance boost as we talked about in earlier articles in this series), searching with a Query token for all objects of Class Organizational Role, from the root of the tree.
One of the neat things is that the when we specify the attributes to return, we can use the Token local variable to specify our BAD-ATTR-NAME, which in this case is set to the value of Group Membership.
Now for the good stuff. As usual, it is done inside a for each loop, iterating through the node set we retrieved and doing the magic on each loop through. As usual, I set it to notrace="true" for the loop itself, since this is a killer on performance if you let it trace. The difference in time it takes is amazing! You can easily use this type of rule to prove this to yourself. Pick a reasonably large set of objects, say 5000 or so, run a rule like this on it with trace enabled on the for each loop, then look at the time it takes in the email report. Do it again with trace disabled, and see how long it takes. The difference can be quite astonishing! Several minutes to hours depending on what you are doing.
<do-set-local-variable name="REMOVED-COUNT" scope="policy">
<token-xpath expression="$REMOVED-COUNT + 1"/>
Inside the for each loop, we have an IF condition, that tests an XPATH statement:
Lets dissect that XPATH for a moment.
$current-node means our current iteration of the for each loop, it is an automatic variable that gets assigned by the engine itself.
Then we want the node ($current-node/attr), whose XML attribute is attr-name, and has the value of our variable BAD-ATTR-NAME ($current-node/attr[@attr-name="$BAD-ATTR-NAME"]). Yes, I know it is confusing, but in this case the IDM attribute means one thing (something like Group Membership, Surname, etc) but the XML attribute means stuff inside the node, and might looks like timestamp="200812121229Z" or src-dn="\ACME-TREE\com\Acme\Users\BSmith" and the like. We use the @ symbol to specify one of the XML attributes, like when selecting @src-dn or $current-node/@src-dn (the original cause of my screw up!).
Next inside the XPATH test, we take the node from the specific node ($current-node/attr[@attr-name="$BAD-ATTR-NAME"]/value), and check if it is equal to the $BAD-VALUE local variable ($current-node/attr[@attr-name="$BAD-ATTR-NAME"]/value=$BAD-VALUE). This relies on the fact that if you compare with an equals test, a string to a node set in XPATH, it compares all the values to the string and it returns true if ANY of the values in the node set are the same. This is great, since it basically is a contains() style test, but much easier than looping through all the value nodes and figuring it out.
When we hit one of these cases, we then send a Remove Source Attribute action, removing the specific value (since we do not want to remove any other Group Membership attributes just the one bad value) and finally we increment our counter (REMOVED-COUNT) so we can report on it in the email.
Finally we do some reporting variables, END-TIME, calculate ELAPSED-TIME, and QUERY-TIME and use them in the email message to let us know how long the process took, and how many values we removed.
This rule really made my life easier, and once I realized I could use it on pretty much most attribute types, I was really happy. In that same system, I had to clean up the manager, isManager, and directReports attributes on about 15,000 objects. Well, a simple modification of this basic rule, and away it went! Nice and easy, and when I need to do it again because I did not quite get the process perfect and want to try again, just trigger the rule again!
Very powerful, simple to use, and easy to understand. Just the way I like it! Hope this is helpful to others out there!