Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
Rohan001 Respected Contributor.
Respected Contributor.
868 views

Extremely slow search performance & Pagination alternative

Hi All,

We have a web-application which is extracting data from HPRM 8.3 using RM NET SDK.

The applciation is build on Asp.net MVC & we are sending text queries to get the results.

The issue is with HPRM instance while limited records (< 5000) the query executuon code takes about few seconds to retreive records. But on another system which has 50000+ records the performance of the same degrades substantially. To give example, to execute query which returns 900 records in first HPRM instance it takes 5 milli-seconds; where as to retreive 2500 records from second instance takes in excess of 8 seconds.

So my question is, Is there any specific care we need to take to ensure the instances with large data also works comparably ?

Another option which I was wondering was in case getting 2500 records in a single go is a issue, then can we split the output in smaller chunk (something like pagination). I went through the documentation and did not find any relevent info on such feature. Does HPRM 8.3 has such feature ?

  

Tags (1)
0 Likes
7 Replies
Micro Focus Expert
Micro Focus Expert

Re: Extremely slow search performance & Pagination alternative

Paging is the way to go.

The main thing you need when paging is SkipCount (which I believe exists in 8.3).

Also, don't use TrimMainObjectSearch.FastCount, ironically is can be slow.  Instead return a property indicating whether there are more Records.

So, your web request includes a start parameter which becomes SkipCount and then you use a foreach through the TrimMainObjectSearch, stopping when you have done 20.  Always prefer iterating over using the various get Uri methods in TrimMainObjectSearch as the iterator has database pre-fetch optimisation built in.

Example

int counter = 0;

SDK.TrimMainObjectSearch search = new SDK.TrimMainObjectSearch(_database, TRIMSDK.BaseObjectTypes.Record);
search.SkipCount = myStartParameter;

foreach (SDK.Record record in search)
{
    if (counter < pageSize)
    {
        //return Record
    }
    else
    {
        response.HasMoreItems = true;
        break;
    }
    counter++;
}

 

 


Blog | Samples | CM SDK Docs
**Any opinions expressed in this forum are my own personal opinion and should not be interpreted as an official statement on behalf of MicroFocus**
0 Likes
Highlighted
Rohan001 Respected Contributor.
Respected Contributor.

Re: Extremely slow search performance & Pagination alternative

Hi David,

Thanks for the reply. I'll try the method mentioned and will update the thread.

On a side note, In your experience, Do yo think the result payload has any impact on performance.

What I was thinking was instead of getting all properties I would get only some specific record properties, this would make the resultset much lighter. But in the HPRm doucmentaton does not mention any property / method which helps us to return only specific properties of records. Do you think this is a good line of investgation ? or its  someting that won't make much difference ? 

0 Likes
Rohan001 Respected Contributor.
Respected Contributor.

Re: Extremely slow search performance & Pagination alternative

Hi David,

I implimented the SkipCount property and it did skip first 10 records and returned the 11th record.

But, the performance issues I have is the line "

foreach (SDK.Record record in search)

(I guess this is the line where the query actually gets executed and returns the results.)

The line takes the most amount of time.

The time required for executing this line will not change if I impliment 'SkipCount' rite? (as it still returns all the records)

Please correct me if my understanding is incorrect.

PS.. thanks again for the reply and warning about the getUri methods; I think I have used some which I will review again.

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Extremely slow search performance & Pagination alternative

SkipCount will only improve performance on subsequent pages.  If you do not use SkipCount then when you open page 100 you have to iterate past the first 100 pages.  For the first page though SkipCount has zero effect.


Blog | Samples | CM SDK Docs
**Any opinions expressed in this forum are my own personal opinion and should not be interpreted as an official statement on behalf of MicroFocus**
0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Extremely slow search performance & Pagination alternative

The commonly used properties are always returned but the less commonly only when you request them.  You should definately only return the properties you need as there are some that are quite expensive.  The way the SDK works is to make the database call as you access the property 'getter'.  So you should implement a property on your service request that allows the consumer to specify the properties they need.  


Blog | Samples | CM SDK Docs
**Any opinions expressed in this forum are my own personal opinion and should not be interpreted as an official statement on behalf of MicroFocus**
0 Likes
Rohan001 Respected Contributor.
Respected Contributor.

Re: Extremely slow search performance & Pagination alternative

Hi David,

Thanks for the explanation. So I have implimeted the code as given before, but I don't see any overloads or properties where I can set which properties values should be returned (source - sdk.hpecm.xyz), 

So my current code is:

...
...
TrimMainObjectSearch searchObject = new TrimMainObjectSearch(hpConnect, BaseObjectTypes.Record);
searchObject.SetSearchString(searchString);

//Cannot find any Set method to specify which properties should be filled in Record object 
//(e.g. only title) foreach (Record item in searchObject) { //when I add breakpoint here the 'item' object has all the default properties (which is a bit bulky) resultList.Add(item.Title); //just a example, adding to some list } ... ...

Is there an alternate way to query to get the specific (i.e. 'trimmed-down') version of Record object ?

regards,

Rohan W.

 

0 Likes
Micro Focus Expert
Micro Focus Expert

Re: Extremely slow search performance & Pagination alternative

There is no facility to determine which properties are returned from the database, other than whether you choose to access the property.  In your sample the only property returned is Record.Title.

My recollection is that this is a web service?  If so then you will want an intermediate model to return your data, not the SDK Record object.  In my code below we return Title and Number.  You could add a parameter to your web service to allow the consumer to send a list of field names and then use the PropertyDef (and Record.GetPropertyAsString() method) to return only the specified fields.

If you want to discuss further send me a private message and we can arrange a phone call to discuss web service design.

foreach (Record item in searchObject)
{
   //when I add breakpoint here the 'item' object has all the default properties (which is a bit bulky)
   MyRecord myRecord = new MyRecord() { Title = item.Title, Number = item.Number};
   resultList.Add(myRecord); //just a example, adding to some list
}

Blog | Samples | CM SDK Docs
**Any opinions expressed in this forum are my own personal opinion and should not be interpreted as an official statement on behalf of MicroFocus**
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.