ddnicholls1 Absent Member.
Absent Member.
829 views

Parsing REST XML data

I'm using Geoff's code example (https://www.netiq.com/communities/cool-solutions/example-parsing-json-rest-driver/) to grab REST data and process through the driver. I am doing this through a Migrate In from the web service.

Everything working well (thanks Geoff) to a point and I receive the same data format as in his example, and have a node set process that is outputting data such as:

<modify-attr attr-name="is_overridden">
<add-value>
<value type="string">false</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_modified_by">
<add-value>
<value type="string">adminns</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_modified_on">
<add-value>
<value type="string">2018-06-07T11:53:16.000Z</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_name">
<add-value>
<value type="string">Employee</value>
</add-value>
</modify-attr>
<modify-attr attr-name="marital_status">
<add-value>
<value type="string">M</value>
</add-value>
</modify-attr>

however following this parse I get errors for each node:
<output>
<status event-id="0" level="error">Code(-9039) Element <modify> does not have a valid association.<application>DirXML</application>
<module>HCM</module>
<object-dn></object-dn>
<component>Subscriber</component>
</status>
</output

The transaction errors before the schema map.

I've tried setting destination associations as part of the policy but makes no difference. I've also looked at earlier threads suggesting Merge Authority settings however these have not helped either.

I'm assuming there is a best practice of setting an association/association value in a scenario like this that I'm not adhering to.

Any suggestions appreciated.

Thanks

Dean
Labels (1)
0 Likes
17 Replies
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

ddnicholls wrote:

> I've tried setting destination associations as part of the policy but
> makes no difference.


You need an association on every event that comes from the application side, so
IDM can identify the object. This is usually some kind of GUID or primary key
that the REST service has to deliver together with the payload and that you
can feed into

<do-set-op-association>
<arg-association>
<token-text xml:space="preserve">my_association_value</token-text>
</arg-association>
</do-set-op-association>

If you use the rs:jsonToXDS() function as in the default driver config and the
association is provided as
{
"association" : "my_association_value",
...
}

it will even be set as operation association automatically, IIRC.

--
http://www.is4it.de/en/solution/identity-access-management/

(If you find this post helpful, please click on the star below.)
______________________________________________
https://www.is4it.de/identity-access-management
ddnicholls1 Absent Member.
Absent Member.

Re: Parsing REST XML data

Thanks Lothar. I was using a unique value from the app as an association - one of my rules not the package rule you have pointed out - but wasn't having any luck.

Possibly I wasn't embedding it correctly in the nodeset, but I will use your code snippet and play around and see how I go.

Thanks

Dean
0 Likes
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

On 7/17/2018 4:34 AM, ddnicholls wrote:
>
> Thanks Lothar. I was using a unique value from the app as an association
> - one of my rules not the package rule you have pointed out - but wasn't
> having any luck.
>
> Possibly I wasn't embedding it correctly in the nodeset, but I will use
> your code snippet and play around and see how I go.


If all else fails, you might need to use the Append XML Node association
to a target of . (period, current node).

Then set XML text of target XPATH of association and your value.


0 Likes
ddnicholls1 Absent Member.
Absent Member.

Re: Parsing REST XML data

Cheers Geoff. Would that be part of the same nodeset rule or would I need to parse the data a second time to do that?
0 Likes
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

On 7/17/2018 9:04 AM, ddnicholls wrote:
>
> Cheers Geoff. Would that be part of the same nodeset rule or would I
> need to parse the data a second time to do that?


Well where is the association data stored? I.e. What are you using.

I woould do this outside and after the nodeset loop, since it only needs
to be done once. If you map it to some value in the JSON/XML then you
can simply use the value of Operation Attribute AttrName, if you have
already handled it in the loop earlier.


0 Likes
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

geoffc;2484161 wrote:
On 7/17/2018 9:04 AM, ddnicholls wrote:
>
> Cheers Geoff. Would that be part of the same nodeset rule or would I
> need to parse the data a second time to do that?


Well where is the association data stored? I.e. What are you using.

I woould do this outside and after the nodeset loop, since it only needs
to be done once. If you map it to some value in the JSON/XML then you
can simply use the value of Operation Attribute AttrName, if you have
already handled it in the loop earlier.


Our good friend Fernando continuously working on JSON processing library (current version v1.0.2).
It help a lot with parsing of REST/JSON documents.

https://www.netiq.com/communities/cool-solutions/cool_tools/easier-json-processing-idm-driver-policies/
https://github.com/fchierad/IDM-ECMAlib
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

On 7/17/2018 10:34 AM, al b wrote:
>
> geoffc;2484161 Wrote:
>> On 7/17/2018 9:04 AM, ddnicholls wrote:
>>>
>>> Cheers Geoff. Would that be part of the same nodeset rule or would I
>>> need to parse the data a second time to do that?

>>
>> Well where is the association data stored? I.e. What are you using.
>>
>> I woould do this outside and after the nodeset loop, since it only
>> needs
>> to be done once. If you map it to some value in the JSON/XML then you
>> can simply use the value of Operation Attribute AttrName, if you have
>> already handled it in the loop earlier.

>
> Our good friend Fernando continuously working on JSON processing library
> (current version v1.0.2).
> It help a lot with parsing of REST/JSON documents.
>
> https://www.netiq.com/communities/cool-solutions/cool_tools/easier-json-processing-idm-driver-policies/
> https://github.com/fchierad/IDM-ECMAlib


I forgot about Fernando's stuff. It is both useful and simple. Wraps the
calls to make it easier to use.


ddnicholls1 Absent Member.
Absent Member.

Re: Parsing REST XML data

Much appreciated thanks guys. I'll rejig my parsing logic to incorporate your advice and revert.

In this particular instance the RESTful service we're talking to is capable of communicating with XML directly, so may not need Fernando's tools, but I can see how useful they will be in other implementations.
0 Likes
ddnicholls1 Absent Member.
Absent Member.

Re: Parsing REST XML data

Made a bit of progress here but still no joy. As seen on the trace below an association is added using the timestamp element, but still data does not get past the schema map.

Every user object coming in from the web service results in an input like this, so I think the data itself is parsing fine.

I still get the no association error but I would expect this would result in a synthetic add.

I do get this on startup, I think it is normal for a REST driver but I'm not sure whether it is referring to the process I'm attempting, is it worthwhile implementing this class?

The REST driver doesn't return any application schema by default. If there is application specific schema you want the shim to report, you can write your own Java class that implements the SchemaReporter interface and then configure the driver to load your class as a Java extension. See the driver documentation for more details.



<nds dtdversion="4.0" ndsversion="8.x">
<source>
<product edition="Advanced" version="4.6.2.0">DirXML</product>
<contact>NetIQ Corporation</contact>
</source>
<input>
<modify class-name="User" event-id="0">
<association>1532325489384</association>
<modify-attr attr-name="created_by">
<remove-all-values/>
<add-value>
<value type="string">admin.bc</value>
</add-value>
</modify-attr>
<modify-attr attr-name="created_on_timestamp">
<remove-all-values/>
<add-value>
<value type="string">2018-07-19T00:56:14.000Z</value>
</add-value>
</modify-attr>
<modify-attr attr-name="end_date">
<remove-all-values/>
<add-value>
<value type="string">9999-12-31</value>
</add-value>
</modify-attr>
<modify-attr attr-name="first_name">
<remove-all-values/>
<add-value>
<value type="string">Doris</value>
</add-value>
</modify-attr>
<modify-attr attr-name="formal_name">
<remove-all-values/>
<add-value>
<value type="string">Doris Day</value>
</add-value>
</modify-attr>
<modify-attr attr-name="gender">
<remove-all-values/>
<add-value>
<value type="string">F</value>
</add-value>
</modify-attr>
<modify-attr attr-name="is_overridden">
<remove-all-values/>
<add-value>
<value type="string">false</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_modified_by">
<remove-all-values/>
<add-value>
<value type="string">admin.bc</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_modified_on">
<remove-all-values/>
<add-value>
<value type="string">2018-07-19T00:56:14.000Z</value>
</add-value>
</modify-attr>
<modify-attr attr-name="last_name">
<remove-all-values/>
<add-value>
<value type="string">Day</value>
</add-value>
</modify-attr>
<modify-attr attr-name="marital_status">
<remove-all-values/>
<add-value>
<value type="string">M</value>
</add-value>
</modify-attr>
<modify-attr attr-name="nationality">
<remove-all-values/>
<add-value>
<value type="string">AUS</value>
</add-value>
</modify-attr>
<modify-attr attr-name="native_preferred_lang">
<remove-all-values/>
<add-value>
<value type="string">en</value>
</add-value>
</modify-attr>
<modify-attr attr-name="preferred_name">
<remove-all-values/>
<add-value>
<value type="string">Doris</value>
</add-value>
</modify-attr>
<modify-attr attr-name="salutation">
<remove-all-values/>
<add-value>
<value type="string">MR</value>
</add-value>
</modify-attr>
<modify-attr attr-name="start_date">
<remove-all-values/>
<add-value>
<value type="string">2018-07-19</value>
</add-value>
</modify-attr>
<modify-attr attr-name="personal_information_aus">
<remove-all-values/>
<add-value>
<value type="string"/>
</add-value>
</modify-attr>
</modify>
</input>
</nds>
[07/23/18 15:58:09.833]:HCMApplication HCM ST: Pumping XDS to eDirectory.
[07/23/18 15:58:09.833]:HCMApplication HCM ST: Performing operation modify for .
[07/23/18 15:58:09.833]:HCMApplication HCM ST: Processing returned document.
[07/23/18 15:58:09.833]:HCMApplication HCM ST: Processing operation <status> for .
[07/23/18 15:58:09.833]:HCMApplication HCM ST:
DirXML Log Event -------------------
Driver: \CMDIR-SIT\admin\IDM\DriverSet\HCMApplication HCM
Channel: Subscriber
Status: Error
Message: Code(-9039) Element <modify> does not have a valid association.
[07/23/18 15:58:09.834]:HCMApplication HCM ST: Direct command from policy result
[07/23/18 15:58:09.834]:HCMApplication HCM ST:
<nds dtdversion="4.0" ndsversion="8.x">
<source>
<product edition="Advanced" version="4.6.2.0">DirXML</product>
<contact>NetIQ Corporation</contact>
</source>
<output>
<status event-id="0" level="error">Code(-9039) Element <modify> does not have a valid association.<application>DirXML</application>
<module>HCMApplication HCM</module>
<object-dn></object-dn>
<component>Subscriber</component>
</status>
</output>
</nds>
[07/23/18 15:58:09.835]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.835]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.836]:HCMApplication HCM ST:Applying policy: %+C%14CNETQRESTJSON-itp-JSONtoXDS%-C.
[07/23/18 15:58:09.836]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.836]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.836]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.837]:HCMApplication HCM ST:Applying policy: %+C%14CNETQRESTPCRS-itp-StripAssociationRef%-C.
[07/23/18 15:58:09.837]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.837]:HCMApplication HCM ST: Evaluating selection criteria for rule 'Strip association-ref attribute on User modify'.
[07/23/18 15:58:09.838]:HCMApplication HCM ST: (if-class-name equal "User") = TRUE.
[07/23/18 15:58:09.838]:HCMApplication HCM ST: (if-operation equal "modify") = FALSE.
[07/23/18 15:58:09.838]:HCMApplication HCM ST: Rule rejected.
[07/23/18 15:58:09.838]:HCMApplication HCM ST: Evaluating selection criteria for rule 'Strip association-ref attribute on User Migration'.
[07/23/18 15:58:09.838]:HCMApplication HCM ST: (if-class-name equal "User") = TRUE.
[07/23/18 15:58:09.839]:HCMApplication HCM ST: (if-op-attr 'Group Membership' changing) = FALSE.
[07/23/18 15:58:09.839]:HCMApplication HCM ST: Rule rejected.
[07/23/18 15:58:09.839]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.839]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.840]:HCMApplication HCM ST:Applying policy: %+C%14CNETQRESTPCRS-itp-DoPermissionAssignment%-C.
[07/23/18 15:58:09.840]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.840]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.840]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.840]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.841]:HCMApplication HCM ST:Applying policy: %+C%14CNETQRESTDCFG-itp-AddAssociation%-C.
[07/23/18 15:58:09.841]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.841]:HCMApplication HCM ST: Evaluating selection criteria for rule 'Check for association -ADD'.
[07/23/18 15:58:09.841]:HCMApplication HCM ST: (if-operation match "status") = TRUE.
[07/23/18 15:58:09.842]:HCMApplication HCM ST: (if-xml-attr 'level' match "success") = TRUE.
[07/23/18 15:58:09.842]:HCMApplication HCM ST: (if-op-property 'association' available) = FALSE.
[07/23/18 15:58:09.842]:HCMApplication HCM ST: Rule rejected.
[07/23/18 15:58:09.842]:HCMApplication HCM ST: Evaluating selection criteria for rule 'Check for association - MODIFY'.
[07/23/18 15:58:09.842]:HCMApplication HCM ST: (if-operation match "modify") = FALSE.
[07/23/18 15:58:09.843]:HCMApplication HCM ST: Rule rejected.
[07/23/18 15:58:09.843]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.843]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.843]:HCMApplication HCM ST:Applying policy: %+C%14CNOVLPWDSYNC-itp-EmailOnFailedPwdSub%-C.
[07/23/18 15:58:09.844]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.844]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.844]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.844]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.844]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.845]:HCMApplication HCM ST:Applying policy: %+C%14CNETQPWDSYNC-itp-TransformPassword%-C.
[07/23/18 15:58:09.845]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.845]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.845]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.845]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.846]:HCMApplication HCM ST:Applying policy: %+C%14CNETQRESTPCRS-itp-BuildEntitlementQueryResult%-C.
[07/23/18 15:58:09.846]:HCMApplication HCM ST: Applying to status #1.
[07/23/18 15:58:09.846]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.846]:HCMApplication HCM ST: -- trace suppressed --
[07/23/18 15:58:09.847]:HCMApplication HCM ST:Policy returned:
[07/23/18 15:58:09.847]:HCMApplication HCM ST:
<nds dtdversion="3.0">
<source>
<product build="20180326_0715" version="1.0.1.0">Identity Manager REST Driver</product>
<contact>NetIQ Corporation.</contact>
</source>
<output>
<status class-name="User" event-id="0" level="success" type="driver-general"/>
</output>
</nds>
[07/23/18 15:58:09.847]:HCMApplication HCM ST:Applying schema mapping policies to input.
[07/23/18 15:58:09.847]:HCMApplication HCM ST:Applying policy: %+C%14CHCMApplication-smp%-C.
[07/23/18 15:58:09.848]:HCMApplication HCM ST: No mapping for class-name 'User'.
[07/23/18 15:58:09.848]:HCMApplication HCM ST:Resolving association references.
[07/23/18 15:58:09.851]:HCMApplication HCM ST:No objects found.
[07/23/18 15:58:09.851]:HCMApplication HCM ST:No document returned.
[07/23/18 15:58:09.851]:HCMApplication HCM ST:End transaction.
[07/23/18 15:58:42.682]:HCMApplication HCM PT:Receiving DOM document from application.
[07/23/18 15:58:42.683]:HCMApplication HCM PT:
<nds dtdversion="3.5" ndsversion="8.x">


Thanks

Dean
0 Likes
Knowledge Partner
Knowledge Partner

Re: Parsing REST XML data

ddnicholls wrote:

> an association is added using the timestamp element


Not sure why you are doing it that way, but an association has to be static and
unique, i.e. every event for the same object coming in must have the same
association value, and two different objects must never have the same
association. A timestamp implements the later but not the first (unless you
would only ever get one single event per object).

--
http://www.is4it.de/en/solution/identity-access-management/

(If you find this post helpful, please click on the star below.)
______________________________________________
https://www.is4it.de/identity-access-management
0 Likes
ddnicholls1 Absent Member.
Absent Member.

Re: Parsing REST XML data

Hi Lothar.

We are testing this connector while waiting for a unique attribute to be seeded in the app, and are using timestamp as a temporary measure. The way we are implementing does result in a unique and static association value for each user:
<input>
<modify class-name="User" event-id="0">
<association>1532325489318</association>
.
.
.
<input>
<modify class-name="User" event-id="0">
<association>1532325489332</association>


Cheers
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.