Walking through an Integration Activity in User Application - Part 1

Identity Manager has several important components. The original synchronization engine that makes all the magic possible has been very stable for many years now and just keeps working and working well.

The Roles Based Provisioning Module (RBPM) added in some user facing components with the User Application. This allows you to create Workflows (real name is a PRD, Provisioning Request Definition) that has a form to begin, that can have data entered, a possible approval flow (my manager, and my manager's manager have to approve) with the same or different forms for approvals. Finally it can enact some change. There is a toolkit of functions available when designing a PRD that you can see in the Workflow tab of a PRD in Designer (The main development tool for Identity Manager, based on Eclipse).

I have written a fair bit about this aspect of Identity Manager. Let me offer some quick links to help you get up to speed, and offer hints about possible options before I begin on this topic.
Introduction to Workflows and PRDs in Identity Manager – Part 1
Introduction to Workflows and PRDs in Identity Manager – Part 2

If you are not yet familiar with workflows and where all the 'stuff' is, in Designer, these two articles above will explain where most of the interesting stuff is hidden. I say hidden on purpose, because sometimes you wonder who designed the interface, and why they made it so cryptic on purpose! I joke, but in honesty it seems like it was designed in pieces, and each piece makes sense, just when mushed together there is a level of consistency I would have liked to see that is missing.
Using SOAP to terminate a running workflow – Part 1
Using SOAP to terminate a running workflow – Part 2

In the two article above I discussed that while there is a Start Workflow token in the DirXML Policy side in the engine, which is very helpful, there is no matching Stop Workflow token. But you can fix that by starting a workflow, that stops the workflow you request. I described how you could do that in the article above. In order to do that, you need to pass in the Correlation ID from the workflow that was started, then you have to look up the Process ID that goes with it via a SOAP call, and finally with that piece of information you can make a SOAP call to terminate a running workflow. Both those checks, get the process ID and terminate the workflow are SOAP calls, that you make using an Integration Activity. This is one of the tools in the palette on the Workflow tab of the PRD editor in Designer.

Looking at the new Integration Activity approach – Part 1
Looking at the new Integration Activity approach – Part 2

They changed the way the Integration Activity works midway through IDM 4.0 in a service pack, and I discussed what has changed in the two articles above.

With that as a starting point, I wanted to take an example trace I saved from when I was trying to get the User Application to run a workflow that would create a Role object. Ironically there is a Add Role and Add Resource token, but they mean assign that Role or Resource to a user. Not create a new nrfRole or nrfResource object. In theory you could create the object in Policy by making the proper object, attribute by attribute directly, however officially that is not supported. What is the difference? I remain unconvinced you can tell the difference, nonetheless, the proper way is to make the SOAP call. This is probably the most common PRD I do at a client. Make a Resource, make a Role, then associate the two via SOAP. These do seem like functions that would be useful to build into the engine. If you look at the PCRS (Permissions Collection and Reconciliation Service) add ons that came in IDM 4.5 (Coming soon to a driver Package near you!) you will see they call a utility Java class that is part of the IDM 4.5 install. There, they use Java to make the SOAP calls, which saves all this work and the asynchronous nature of it all. However they do not expose this in a clear way, but it is pretty easy to infer and figure it out.

I should probably review the functions in that JAR for fun and profit, but that is an article for another day. (Or maybe I will stick to just stealing underpants!)

I do hope that they strongly consider adding more of the SOAP functions as part of the engine toolkit. Or even a generic SOAP approach in the engine, to remove the need for these workarounds. But that is a hope for the future.

First of all, the odds of you succeeding the very first try with an Integration Activity is fairly close to zero. You can get pretty close, but even after having done it a lot, I always seem to miss one of the myriad steps.

This can be as simple as, the WSDL (Web Services Description Language) file, says a field can have 1 or more values, and you have no value for it. If it says 0 or more values, on the Workflow tab, if you click on the Integration Activity icon, and right click, to enable Data Item Mappings view, there is a pre-activity and post-activity button, as well as a Mapping button. If you hit the Mapping button you will see a tree representation of the XML document that is to be sent. There you can disable specific nodes that you just do not want to include at all.

Some endpoints are very picky about empty nodes or not, so this can often be an issue. (You would think you could edit it on the Integration Activity Tab, in the Messages session at the top, but that is just for this session, and for the Simulator really, not for the way it runs normally. Confused? Ya, me too.).

If it needs to be accepted you cannot just drop this in, minimal configuration and go, then you need to know how to troubleshoot it. No doubt you will just go to the documentation for it to explain it. If you happen to find it there, please let me know, since I would love to read it.

What you need to do, is enable logging, for the specific classes of interest. I used to maintain a Wiki page with all the known logging classes listed, and was trying to collect information about when to use each class. However that page was taken down by NetIQ. I maintain a copy personally so if you ever need to know you are welcome to ask me, but apparently not ask NetIQ. If this makes sense to you, please explain it to me.

In this apparently secret cache of knowledge, you will find that the class you really need to enable for troubleshooting logging is:

It helps if you know what the acronyms in there stand for. SOA is Service Oriented Architecture, which is fancy speak for a model, in which Web Services are used to talk between components of a system. Less obvious is AF, which stands for Activity Flow, which is what the Workflow stuff used to be called. (You can actually see an occasional id tag in the XML of a DirXML-EntitlementRef of AF, which means a workflow granted to Entitlement (deprecated now) or NRF for a Resource granted it).

So lets get started. I dropped the Integration Activity onto the workflow in my Designer project, and it prompted me for some info. I am not explaining how to write this workflow rather how to debug it, so no screen shots for you! (Money out, say your request, move to the right! Workflow SOAP nutsi, not Soup Nutsi).

When you import it requests the workflow, and gives you a chance to define the URL, username, and password, see the articles about the New Integration Activity stuff for screen shots and a detailed explanation.

So I triggered my rule by tweaking an object in the directory, my Start Workflow token was called, made the SOAP call and I got this message:

2016-06-28 15:37:36,280 [http-bio-8443-exec-5] DEBUG com.novell.soa.af.impl.core.EngineImpl- [RBPM] startProcess() process id =cn=Create Role,cn=RequestDefs,cn=AppConfig,cn=UserApplication,cn=dset,ou=idm,ou=system,o=acme
2016-06-28 15:37:36,280 [http-bio-8443-exec-5] DEBUG com.novell.soa.util.CacheUtil- [RBPM] Object was found in cache: IDM_ACTIVATION_INFORMATION cache holder: AdvancedEdition
2016-06-28 15:37:36,280 [http-bio-8443-exec-5] DEBUG com.novell.soa.util.CacheUtil- [RBPM] Object RETRIEVED from cache: IDM_ACTIVATION_INFORMATION from cache holder: AdvancedEdition
2016-06-28 15:37:36,280 [http-bio-8443-exec-5] TRACE com.novell.soa.af.impl.core.ProcessImpl- [RBPM] start() requestId=316b78aa667942459f5691b05eb16ad0, type=cn=Create Role,cn=RequestDefs,cn=AppConfig,cn=UserApplication,cn=dset,ou=idm,ou=system,o=acme
2016-06-28 15:37:36,282 [http-bio-8443-exec-5] TRACE com.novell.soa.af.impl.link.Link- [RBPM] Link() FORWARD[Start, Activity]

It is interesting that it is looking to see if IDM is activated, for Advanced Edition, at this level, but just informational. Then it starts its, shows the requestId (which is what is returned to the Start Workflow token, to be used in my Stop workflow example, that is the process Id), shows the DN of the requested workflow.

If you made a typo in the DN you sent via the Start Workflow token, it will be glaringly obvious at this point.

All workflow's start with a Start Activity, and low and behold that is the last line, we made it, to at least start.

On a side note it seems like a Start activity is literally just a placeholder, like the root of a tree, or the root node of an XML document. I.e. There has to be a starting point, but in workflow's there is actual useful work required here.

If you right click on the Start item, and make sure Data Item Mappings view is showing, you will see a pre and post activity button.

It helps if you realize that the interface between the Form that the user sees, works on, and submits (All of which executes in the browser environment, only the results get sent in) and the Workflow process itself is either side of this Activity.

In the pre-activity, you can actually define 'stuff' to be done (JavaScript being your toolset here). I did not realize this till some one pointed it out (Hey Joel!) that you can actually read some GCV's and set them into the form, before it even renders at this point if you wanted to. This is a great way to inject some configuration data into the form. This happens, after the user chooses to run the PRD but before the form is instantiated or loaded.

Post Activity is just after the form is submitted, but before anything else happens in the workflow. The key step here is to copy (Called Map in the UI) the Form data, from the form fields into the Flowdata XML structure that the workflow uses.

I am torn if this is a clever idea (so you can process the form data) or if it is a sign they mushed together two different things and you need to copy the info over. I tend towards the latter, since if the former, you would just use an activity to clean up the data if you needed to at any point.

Now this is pretty tedious to do by hand, since you might have a fair number of form fields to map. Luckily they realized this and gave us a Map All button, that makes simple assumptions that works 95% of the time, so a pretty good helper function.

2016-06-28 15:37:36,297 [http-bio-8443-exec-5] DEBUG com.novell.soa.af.impl.core.DataItemEvaluator- [RBPM] evaluating target expression: wi.createXPath("flow-data/IDM_COMPLETED_APPROVAL_STATUS").setValue("approved")

This I think is a built in function, reflecting the button on the first page of a PRD labeled, "Set Default Completion Status to Approved".

2016-06-28 15:37:36,298 [http-bio-8443-exec-5] DEBUG com.novell.soa.af.impl.core.DataItemEvaluator- [RBPM] result: com.novell.soa.script.mozilla.javascript.Undefined@1b36f5f6

While this sort of looks like an error, 'Undefined' cannot be good, right, I think it means in this context that the resulting value of the previous action, a wx.createXPath, returns a result, that has no defined display value, so that is undefined. You will see this a lot when the Mapping happens.

2016-06-28 15:37:36,299 [http-bio-8443-exec-5] TRACE com.novell.soa.af.impl.activity.ActivityNode- [RBPM] persistStatus() activity=Start[Start], newStatus=Arriving
2016-06-28 15:37:36,303 [http-bio-8443-exec-5] TRACE com.novell.soa.af.impl.core.ProcessImpl- [RBPM] startActivity() activity=Start[Start], newThread=true, pooled = true
2016-06-28 15:37:36,305 [pool-4-thread-10] TRACE com.novell.soa.af.impl.activity.ActivityNode- [RBPM] notifyArrive(), activity=Start[Start]
2016-06-28 15:37:36,305 [pool-4-thread-10] TRACE com.novell.soa.af.impl.activity.ActivityNode- [RBPM] process(), activity=Start[Start]
2016-06-28 15:37:36,306 [pool-4-thread-10] TRACE com.novell.soa.af.impl.activity.ActivityNode- [RBPM] persistStatus() activity=Start[Start], newStatus=Running
2016-06-28 15:37:36,308 [pool-4-thread-10] INFO com.novell.soa.af.impl.LogEvent- [RBPM] [Workflow_Started] Initiated by cn=uaadmin,ou=admins,ou=system,o=acme, Process ID: 316b78aa667942459f5691b05eb16ad0, Process Name: cn=Create Role,cn=RequestDefs,cn=AppConfig,cn=Us
erApplication,cn=dset,ou=idm,ou=system,o=acme:25, Activity: Start, Recipient: cn=uaadmin,ou=admins,ou=system,o=acme, Secondary User: null
2016-06-28 15:37:36,311 [pool-4-thread-10] TRACE com.novell.soa.af.impl.activity.ActivityNode- [RBPM] forward(), activity=Start[Start], link=FORWARD[Start, Activity]
2016-06-28 15:37:36,311 [pool-4-thread-10] TRACE com.novell.soa.af.impl.link.Link- [RBPM] Link() SUCCESS[Activity, Finish]
2016-06-28 15:37:36,311 [pool-4-thread-10] TRACE com.novell.soa.af.impl.core.ProcessImpl- [RBPM] forward() current=Start[Start], next=Call CreateRole[Activity]

I think here, we are still in the process of starting the workflow, so we are still in the Start Activity. You can see the steps it goes through in the starting process. Finally, we are showing the link, the line in Designer, that says, current is Start, next is Call CreateRole.

That seems like a good place to take a break point and continue this in the next article. There is so much trace to talk about, that this may take a while.

My generic hope in these style of articles is that by getting the text into the Google cache, when someone runs into an error they do not understand, pasting a relevant snippet of it will pull up one of these articles and hopefully offer some insight to better diagnose their problem.

Thus my request to those of you reading this, to please collect similar traces, errors, and interesting messages and write about them in a similar way. This way we can all help each other figure out the low level stuff, that is not really talked about anywhere else.


How To-Best Practice
Comment List
Related Discussions