What’s new in Validator for Identity Manager 1.4 – Part 2
Validator for Identity Manager version 1.4 was recently released and it adds a bunch of interesting new features. I like writing about new features in products to share the news, but also makes me think about them in detail so I end up learning new things I might otherwise have missed.
I have written a number of articles about new features in IDM recently:
In part one of this series I discussed Scheduler, the GUI tester (Selenium plugin), contextual right click menus, and wildcards in Delete operations.
Let's continue on through the enhancements in this article.
* Implemented collapsible test groups
From a User Interface perspective, this is my favorite new feature. During the beta I wanted to start using 1.4 exclusively just for this one feature.
In earlier versions, they added the ability when adding a test, to call it a "Group" which made it a header, that grouped the tests below it, visibly at least. This was a great add on to make organizing a large number of tests easier. Sort them by groups. Additionally, if you selected the Group test line, you would select all the tests under it. Thus you could easily run a set of tests in sequence, and select them by group.
With version 1.4 there is now an arrow that rotates when you click on it, and it folds and unfolds the group of tests into one line. This is immensely helpful in decluttering your list of tests. When you are working on one set of tests, you can fold all the rest and ignore them, without wasting valuable screen height.
This is really my favorite feature added, since every time you use Validator you will use it. The other features are great, but this is the most useful so far.
* Replaced NOT actions and Compare and Equal actions with general actions where a compare operator and mode are specified.
This is a huge change in how Validator behaves and a very good change. The simplest way to explain it to an IDM familiar person is that the Assert statements are all now similar to how the IDM tokens do compares. There is a different Assert for each type of "thing" (attributes, objects, etc) and each one now has a mode selector. This is sometimes simply, Assert exists or Assert not exists. Othertimes (depending on context) there is a Compare Operator, and a With Mode selector.
Compare Operator has the following options:
greater than or equals
less than or equals
not greater than
not greater than or equals
not less than
not less than or equals
not in subtree
The With Mode selector has the following options:
These should be pretty familiar as they match what the IDM engine mostly supports. Though missing of course is the Source DN/Destination DN compare modes, because there is only ever one kind of DN format in each connector, unlike in the IDM engine where is always potentially two modes.
This is one of the main reasons that the format for the JSON file is changed when you first open it in the new Validator. This is important, since once you open a test suite's JSON file in Validator 1.4 the format is converted so that file cannot be used in Validator 1.3 afterwards. What this means is COPY your test suite to the 1.4 instance, and backup your 1.3 test file. Do not MOVE the test suite JSON file!
Anyway, this simplifies the number of Asserts in each connector, so at some levels it looks like there are less options in 1.4 vs 1.3 but that is incorrect. There is much more available now, and simpler to use.
What I also like is that while in the dream world, the people writing the tests are different from the people designing the system. This way you have two perspectives working on the subject, making it easier to catch issues, rarely is that ever the case. This is sort of why you should not test with the system you use to do the stuff. I.e. You could write tests in IDM to prove everything was working, but it is better to have an outside system do the testing.
However you rarely get that dream world, and usually the guys doing the work design the tests as well. By making the Assert tokens act like the tokens in IDM it is easier to wrap your head around it, and make the tests do what is needed, since you know what the code is doing, and can more simply replicate it in the testing tool.
* Removed the Regex checkbox and replaced it with compare operator and mode select boxes.
This one is pretty much part of the above change. Once you have the Compare Operator and With Mode selections, there is no need for a Regular Expression tick box, you just select it from the drop down as your compare mode.
Now since this is a web application, it would be kind of cool if they were to find some toolkit that does Regex testing to allow a Regex tester in Validator as well. For example, perhaps you could enter your Regex, hit a tester button, and then a field to put sample data in would show up and you could then test your Regex against it. That would be a great way to make this work better.
Personally I use a commercial tool called Regex Buddy that I quite like, since it has an "Explain" mode, which parses the Regex and shows you what each 'token' in the Regex is doing. That does not always explain what it is doing, but can give you good hints to decode someone else's regular expressions.
I wrote about Regex Buddy here, in case that interests you:
* Added retry to LDAP "Delete Objects" and "Rename Or Move" to handle case where directory is busy (Error 51)
These are nice add ons, since in the past if the system could not delete, rename, or move the object in time, it usually would return an error which is not what you want really. This now adds the usual, retry X times every Y intervals (in milliseconds).
It is nice that they thought to add it to Move, Rename, and Delete since these are often, slow operations compared to other things.
* Added Generic Actions Increment Variable to increment/decrement the value of a variable
This called Calculate Variable in Generic Actions connector and it lets you select a source and destination variable, and then how much you wish to increment it. There is an up/down selector and you go down to negative numbers to decrement, and up for positives to increment it.
This is clever, since you could now store a testing password for use, and just increment it by a 2 digit number to reuse it on each test run, so as to avoid unique password history collisions. It looks like you can use very large intergers in the increment/decrement field, so if your password policy requires that the password differ by X number of characters, then you could just increment by a number with sufficient digits to beat that rule.
What I used to do was set a variable to the current time in CTIME, which a 10 digit number, and append it to some test string that meets the rest of the password complexity rules. This way I always got a random looking number. (It wasn't really random, just always different). This just makes it much simpler. I still think a generate random password would be great and I know the reason why they are avoiding it. They would need to allow you to specify all the different kinds of password rules, and that is a ton of complexity to implement for minimal benefit that would be used elsewhere. Personally I would prefer if they could use the eDirectory connector to read the password policy out of eDirectory and use those rules to generate a password.
Or even simpler, in the eDirectory connector there is an API call to generate a random password, and NMAS generates it for you, based on the password policy specified. This would not work in Active Directory or any other connector but would be helpful for eDirectory. Validators primary target is IDM, which always has eDirectory at its heart, so one such approach would cover most cases. If you needed a strange password policy for some other connector, you could always just set up an unassigned Password Policy in eDirectory for Validator to read to get the configuration.
* Run Template - when selecting a template to run it will update the description automatically
This one is a bit subtle but clever, and makes things look better quickly. When you use the Run Template connector action from the Generic Actions connector you get to select any test you marked as a template. This is really useful to reuse code. Not quite true functions, but getting closer. (I doubt you can recurse them, though I wonder... If a Run Template calls a Run Template, will it ever end?)
Anyway when you select the Template to run, the Description field changes from "Run Template" to "Run Template - Template Name" which means if you do not touch the Description to save time, it still looks good and is descriptive. I noticed this in a number of other fields, that when I chose a Action of some kind, in a connector, Instead of just naming it "Assert Object Exists" it named it "Assert Object Exists in Connector Name" which also makes it look nicer.
I like using the Description field to remind me what I am testing or doing in that step, and this nicely makes it easier to see in when everything is folded up in the Setup/Test/Cleanup tasks.
It is subtle, but makes things better quickly and reduces the effort you need to make. All great things. I am glad to see what I think of a "quality of life" enhancements coming into the product. Like I said in the first section, this is a great product because the folk who write it, use it daily. They live in the same world we live in, and if they can fix something to make their lives easier, it is worth doing, and then we get to benefit.
This is a trick I learned for getting features I want added to products. I find a bug and submit it, but I note that if they ALSO fixed this other thing, it would make fixing the issue they committed faster. There was a memory leak in Designer 4.0 that was related to Packaged content. but every time you opened packaged content it would pop up a warning. But to find the leak you had to do this 50 to a 100 times. I noted and submitted a bug about that silly popup and the dev added in the "Remember this decision" button to fix it, so he could fix the real problem faster. So remember that when you have an issue that has a related issue.
That is about it for this segment of the series, stay tuned for the next article.