mdirkse Absent Member.
Absent Member.
1883 views

Serious resource utlization problem using SOAP and 8.0.2

Hi,
I'm running into a fairly serious issues whilst trying to download large attachments from a Groupwise 8.0.2 server using SOAP.

Basically, the problem is as follows: at a client site, there's an inbox with an email that has an attached avi file of about 330mb. Our SOAP client tries to retrieve this file in chunks of 1mb in size (if you try to set the size of the chunks any larger, Groupwise will give you a 59922 error, as 1mb is apparently some sort of hard coded limit). So that attachment corresponds to roughly 330 AttachmentRequestMessage's and their responses, each with a 1mb payload.

When we run our client, the cpu utilization of the Groupwise server takes about a dozen seconds to climb to 100%, every single time we try. Since 100% cpu utilization renders all GW clients more or less inoperable (or at least extremely slow) this is a non-starter.

We changed our client to wait 2 secs. in between requesting each chunk of the attachment. This seemed to help initially, but by the time we had transferred 100mb of the file, cpu utilization climbed to 95% and stayed there until the file transfer was completed.

This is a little disconcerting. We're requesting a megabyte of data from an attachment every 2 seconds, and Groupwise seems to have all sorts of problems in trying to service these requests. As you can imagine, client performance isn't that great at 95% server utilization either. I suppose we could increase the pause to something like 5 seconds, but it seems ridiculous that such simple requests are generating this amount of load.

So my question is 2-fold:
1. can I increase the maximum size of an attachment part chunk? 1mb is not very much at all, seeing as our server is on the same gigabit lan segment as the GW server
2. why does cpu utilization shoot up to 95% while servicing what seem to be fairly simple requests, and how do I avoid this by some means other than increasing my pause time?

Thanks in advance,
Maarten

PS I spent some time looking for the 59922 error code in the docs, but turned up nothing
Labels (1)
0 Likes
6 Replies
Anonymous_User Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2


You can download the complete attachment
in 1 go over HTTP using a SOAP login session.

It's all in the docs...

Am 19.08.2010 16:06, schrieb mdirkse:
>
> Hi,
> I'm running into a fairly serious issues whilst trying to download
> large attachments from a Groupwise 8.0.2 server using SOAP.
>
> Basically, the problem is as follows: at a client site, there's an
> inbox with an email that has an attached avi file of about 330mb. Our
> SOAP client tries to retrieve this file in chunks of 1mb in size (if you
> try to set the size of the chunks any larger, Groupwise will give you a
> 59922 error, as 1mb is apparently some sort of hard coded limit). So
> that attachment corresponds to roughly 330 AttachmentRequestMessage's
> and their responses, each with a 1mb payload.
>
> When we run our client, the cpu utilization of the Groupwise server
> takes about a dozen seconds to climb to 100%, every single time we try.
> Since 100% cpu utilization renders all GW clients more or less
> inoperable (or at least extremely slow) this is a non-starter.
>
> We changed our client to wait 2 secs. in between requesting each chunk
> of the attachment. This seemed to help initially, but by the time we had
> transferred 100mb of the file, cpu utilization climbed to 95% and stayed
> there until the file transfer was completed.
>
> This is a little disconcerting. We're requesting a megabyte of data
> from an attachment every 2 seconds, and Groupwise seems to have all
> sorts of problems in trying to service these requests. As you can
> imagine, client performance isn't that great at 95% server utilization
> either. I suppose we could increase the pause to something like 5
> seconds, but it seems ridiculous that such simple requests are
> generating this amount of load.
>
> So my question is 2-fold:
> 1. can I increase the maximum size of an attachment part chunk? 1mb is
> not very much at all, seeing as our server is on the same gigabit lan
> segment as the GW server
> 2. why does cpu utilization shoot up to 95% while servicing what seem
> to be fairly simple requests, and how do I avoid this by some means
> other than increasing my pause time?
>
> Thanks in advance,
> Maarten
>
> PS I spent some time looking for the 59922 error code in the docs, but
> turned up nothing
>
>


0 Likes
mdirkse Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2

Hmm, could you perhaps point me to where in the docs?

Because as far as I know you have to download an attachment using a getAttachmentRequest (Novell Doc: NDK: GroupWise Web Services - getAttachmentRequest), which takes a length parameter, and if you set that to > 1mb, Groupwise returns a 59922 error which, I'm pretty sure, *isn't* in the docs.

Ray;2013273 wrote:
You can download the complete attachment
in 1 go over HTTP using a SOAP login session.

It's all in the docs...

Am 19.08.2010 16:06, schrieb mdirkse:
>
> Hi,
> I'm running into a fairly serious issues whilst trying to download
> large attachments from a Groupwise 8.0.2 server using SOAP.
>
> Basically, the problem is as follows: at a client site, there's an
> inbox with an email that has an attached avi file of about 330mb. Our
> SOAP client tries to retrieve this file in chunks of 1mb in size (if you
> try to set the size of the chunks any larger, Groupwise will give you a
> 59922 error, as 1mb is apparently some sort of hard coded limit). So
> that attachment corresponds to roughly 330 AttachmentRequestMessage's
> and their responses, each with a 1mb payload.
>
> When we run our client, the cpu utilization of the Groupwise server
> takes about a dozen seconds to climb to 100%, every single time we try.
> Since 100% cpu utilization renders all GW clients more or less
> inoperable (or at least extremely slow) this is a non-starter.
>
> We changed our client to wait 2 secs. in between requesting each chunk
> of the attachment. This seemed to help initially, but by the time we had
> transferred 100mb of the file, cpu utilization climbed to 95% and stayed
> there until the file transfer was completed.
>
> This is a little disconcerting. We're requesting a megabyte of data
> from an attachment every 2 seconds, and Groupwise seems to have all
> sorts of problems in trying to service these requests. As you can
> imagine, client performance isn't that great at 95% server utilization
> either. I suppose we could increase the pause to something like 5
> seconds, but it seems ridiculous that such simple requests are
> generating this amount of load.
>
> So my question is 2-fold:
> 1. can I increase the maximum size of an attachment part chunk? 1mb is
> not very much at all, seeing as our server is on the same gigabit lan
> segment as the GW server
> 2. why does cpu utilization shoot up to 95% while servicing what seem
> to be fairly simple requests, and how do I avoid this by some means
> other than increasing my pause time?
>
> Thanks in advance,
> Maarten
>
> PS I spent some time looking for the 59922 error code in the docs, but
> turned up nothing
>
>
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2

The 59922 error is just a warning.
You should use the HTTP GET construct to get large
attachments. It is orders of magnitude faster than
using getAttachmentRequest.

>>> On Friday, August 20, 2010 at 11:06 AM,

mdirkse<mdirkse@no-mx.forums.novell.com> wrote:

> Hmm, could you perhaps point me to where in the docs?
>
> Because as far as I know you have to download an attachment using a
> getAttachmentRequest ('Novell Doc: NDK: GroupWise Web Services ‑
> getAttachmentRequest'
>

(http://developer.novell.com/documentation/gwsoap/gwwbserv/data/b7m3i5b.html

> )),
> which takes a length parameter, and if you set that to > 1mb, Groupwise
> returns a 59922 error which, I'm pretty sure, *isn't* in the docs.
>
> Ray;2013273 Wrote:
>> You can download the complete attachment
>> in 1 go over HTTP using a SOAP login session.
>>
>> It's all in the docs...
>>
>> Am 19.08.2010 16:06, schrieb mdirkse:
>> >
>> > Hi,
>> > I'm running into a fairly serious issues whilst trying to download
>> > large attachments from a Groupwise 8.0.2 server using SOAP.
>> >
>> > Basically, the problem is as follows: at a client site, there's an
>> > inbox with an email that has an attached avi file of about 330mb.

>> Our
>> > SOAP client tries to retrieve this file in chunks of 1mb in size (if

>> you
>> > try to set the size of the chunks any larger, Groupwise will give you

>> a
>> > 59922 error, as 1mb is apparently some sort of hard coded limit). So
>> > that attachment corresponds to roughly 330

>> AttachmentRequestMessage's
>> > and their responses, each with a 1mb payload.
>> >
>> > When we run our client, the cpu utilization of the Groupwise server
>> > takes about a dozen seconds to climb to 100%, every single time we

>> try.
>> > Since 100% cpu utilization renders all GW clients more or less
>> > inoperable (or at least extremely slow) this is a non‑starter.
>> >
>> > We changed our client to wait 2 secs. in between requesting each

>> chunk
>> > of the attachment. This seemed to help initially, but by the time we

>> had
>> > transferred 100mb of the file, cpu utilization climbed to 95% and

>> stayed
>> > there until the file transfer was completed.
>> >
>> > This is a little disconcerting. We're requesting a megabyte of data
>> > from an attachment every 2 seconds, and Groupwise seems to have all
>> > sorts of problems in trying to service these requests. As you can
>> > imagine, client performance isn't that great at 95% server

>> utilization
>> > either. I suppose we could increase the pause to something like 5
>> > seconds, but it seems ridiculous that such simple requests are
>> > generating this amount of load.
>> >
>> > So my question is 2‑fold:
>> > 1. can I increase the maximum size of an attachment part chunk? 1mb

>> is
>> > not very much at all, seeing as our server is on the same gigabit

>> lan
>> > segment as the GW server
>> > 2. why does cpu utilization shoot up to 95% while servicing what

>> seem
>> > to be fairly simple requests, and how do I avoid this by some means
>> > other than increasing my pause time?
>> >
>> > Thanks in advance,
>> > Maarten
>> >
>> > PS I spent some time looking for the 59922 error code in the docs,

>> but
>> > turned up nothing
>> >
>> >

0 Likes
Highlighted
mdirkse Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2

Hi Preston,
Thanks for the tip. So would you recommend ditching getAttachmentRequest for the streaming API entirely? Or should I still use it for smaller attachments, say under 100k or something.
Regards,
Maarten
0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2

It is simpler to have just one way to get attachments.
The is one problem. If you log in as a proxy, you can't
use the HTTP GET method, you have to use getAttachmentRequest.
You would also need to use getAttachmentRequest if you
are talking to a POA that doesn't support HTTP GET.

I believe the logic went in 7.0.2.

Preston

>>> On Tuesday, August 24, 2010 at 2:36 AM,

mdirkse<mdirkse@no-mx.forums.novell.com> wrote:

> Hi Preston,
> Thanks for the tip. So would you recommend ditching
> getAttachmentRequest for the streaming API entirely? Or should I still
> use it for smaller attachments, say under 100k or something.
> Regards,
> Maarten

0 Likes
Anonymous_User Absent Member.
Absent Member.

Re: Serious resource utlization problem using SOAP and 8.0.2


http://developer.novell.com/documentation/gwsoap/gwwbserv/index.html?page=/documentation/gwsoap/gwwbserv/data/bfdhq1z.html

I suggest you read the whole thing up until the Method chapters...

Many good things there.

Ray.
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.