Generic REST Collector supporting odata

I am considering to connect an application offering an odata compliant REST API to IG (3.7.3), and I am wondering if it is possible - and/or suggested - to modify the shippir REST collector for GiHub.

Unfortunately, without modification the REST GitHub Collectors cannot be used for this challenge, since they have some "hard" coded stuff in them.

After exporting the collector I found only the following two functions:

  • getRestResource(connectionParm, token, pageSize)
  • getNextPageURL(response) 

I believe, the responsibility of the first one is to implement the REST request and compose the JSON object representing the collection.

The code seams to utilize a paged search (token = '/users?per_page=' + pageSize;) which can not work for two reasons when working with odata:

1) the schema (/users) is not correct

2) odata does not support paged searches in this way.

The schema can be possibly simply set to the one used in the odata implementation, but the paged search has to be implemented differently.

 In addition to these findings, the first function uses some "HelpperFunctions" provided by com.netiq.daas.restconnector.internal.HelperFunctions!

Can anybody provide some information regarding the HelperFunctions class, and the structure of the final JSON object, IG is able to process?

In case a paged search can not be implemented for odata, is there a technical limitation on the data received in one REST response?

To replace a paged search odata offers the parameters &size and $skip - so a paged search could be replaced by a loop running several requests with in increasing value in the $skip parameter, if needed!

I am not sure, regarding the purpose of the function getNextPageURL(response)  - is this needed, or does this need to be adapted as well?

Kind regards

Thorsten

  • Finally, I got it working!

    In my last attempt I set the "token" element in the result JSON the same way, the original script did:

    result['token'] =  helpperFunctions.getNextPageURL(connectionParm, JSON.stringify(httpRes['Response_Headers']));

    Now there is no error anymore, and IG reports the correct number of object (users) to be imported.

    Since there are only two functions in the original collector template and the undocumented two? functions of the helpperFunctions class, I am wondering, If OpenText could provide some details on the functions defined in the java-script and pagination-script of the template as well as the functions contained in the helpperFunctions class!

    This would make it more convenient to customize the existing (REST) collectors.

    Kind regards

    Thorsten