Highlighted
Absent Member.
Absent Member.
2638 views

GWIA operation failing

At this point, I'm just logging a problem. I've no real insight into it yet. The problem is that email stops being sent through GWIA. Until 2012 SP1, GWIA would actually crash and I simply had a cron job checking if it had crashed and restarted it. However, since 2012 SP1 has been installed, it is still failing but not always crashing. The process is still running and I can still access the web interface, however, on the web interface the MTP is closed and Queue counts show -1.

The strange thing is that the failures seem to be synchronous. At UCBA (work) we have two GWIA's running on different server. I also run Groupwise on my home servers, totally unrelated to UCBA. Within 24 hours, all three GWIA's have failed. In the last 24 hours, my home GWIA had to be restarted, and since then to two interfaces at UBCA failed over night. One was still running, the other had aborted.

From the logs, this seems to be common,


09:46:49 EF90 MSG 2716513 Processing inbound message: /gwsystem/domain/wpgate/gwia/receive/3ed2a905.412
09:46:49 EF90 Path create failed (8210)


06:26:41 F481 The agent is attempting to reestablish the GroupWise link.

I see no other symptoms on the servers, plenty of available disk space and memory. A simple restart of GWIA will keep it going a few more weeks.
At this stage, I'll be running a cron job to stop and restart GWIA on a regular basis for the work servers. I'll leave my home one to fail to see if I can get a better handle on the problem.

All servers are SLES 10 OES2, latest service packs (up until about Sept 2012), running GW 2012 SP1

Regards
Eric.
Labels (2)
0 Likes
12 Replies
Highlighted
Absent Member.
Absent Member.

Re: GWIA operation failing

* ucba wrote, On 11/07/2012 01:46 AM:
> 09:46:49 EF90 Path create failed (8210)


Without looking into details, that error indicates a problem accessing the file system.
Is that GWIA running on a different machine than the domain it belongs to?

Uwe

--
Novell Knowledge Partner (NKP)
Please don't send me support related e-mail unless I ask you to do so.
0 Likes
Highlighted
Visitor.

Re: GWIA operation failing

Its unlikely that the same remote file access issue impacts your home test environment and your production work environment.

Do you have something in common between the two? Like similar anti-virus software running on the GWIA host which is doing on-access scanning of the queues?

-- Bob
0 Likes
Highlighted
Absent Member.
Absent Member.

Re: GWIA operation failing

Thanks for the comment. On all system, everything is local. I'm not seeing any file access issues anywhere else. I have separate servers for each domain.
I think at this stage it is a symptom rather than a cause.
0 Likes
Highlighted
Absent Member.
Absent Member.

Re: GWIA operation failing

Thanks Bob,
The only common thing I can think of is all machines are running as XEN virtuals. All are backed up using LVM snapshots. This is done every day however, and I'm only seeing GWIA go down every few weeks. Just all at the one time!

The only thing that makes sense to me is a memory leak of some sort.
The traffic loads are vastly different, we don't run virus protection at the OS level on any machines

I had wondered about a mal formed email that might get sent to both home and work, but both GWIA have a different antispam server and one GWIA is internal traffic only (Network scanners etc)
It's still bewildering at this point.

I'll be interested to see if anyone else report it.
Regards
Eric.
0 Likes
Highlighted
Trusted Contributor.
Trusted Contributor.

Re: GWIA operation failing

Same issue here. This is the first time it happens, but it never happened before and we've updated to 2012sp1 last month. In our case there is a separate domain for the gwia, and they run on NSS filesystem (OES11sp1).

16:10:26 EF77 Accepted POP3 connection with: 192.168.128.98
16:10:27 EF77 POP3 session ended: 192.168.128.98
16:10:27 F071 Accepted POP3 connection with: 192.168.129.40
16:10:27 F071 GroupWise login failed: 8503
16:10:30 EF6F DMN: MSG 969289 SMTP session ended: [192.168.128.28] ()
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 F071 POP3 session ended: 192.168.129.40
16:10:42 EF77 Accepted POP3 connection with: 192.168.129.40
16:10:42 EF7F Path create failed (8210)
16:10:42 EF7F MSG 969291 Processing inbound message: /media/nss/IAVOL1/iado/wpgate/gwia/receive/00c54e05.569
16:10:42 EF7F Path create failed (8210)

Eric, have you enabled POP3 on the gwia?
0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Re: GWIA operation failing

Do you have purge immediate enable on the volume or are you sitting with a massive amount of purgeable disk space?

Cheers,
Laura Buckley

Views/comments expressed here are entirely my own.
If you find this post helpful, please show your appreciation and click on "Like" below...
0 Likes
Highlighted
Trusted Contributor.
Trusted Contributor.

Re: GWIA operation failing

laurabuckley;2239062 wrote:
Do you have purge immediate enable on the volume or are you sitting with a massive amount of purgeable disk space?

Cheers,


Salvage is disabled on the volume.
0 Likes
Highlighted
Micro Focus Expert
Micro Focus Expert

Re: GWIA operation failing

Thanks for confirming that. I asked as I have seen that exact error in the past when, although there is free space on the volume, there's been a large amount of purgeable space and that's caused the problem.

Cheers,
Laura Buckley

Views/comments expressed here are entirely my own.
If you find this post helpful, please show your appreciation and click on "Like" below...
0 Likes
Absent Member.
Absent Member.

Re: GWIA operation failing

Hi there,

Have you found a solution for this problem yet?
I have a customer (a very small one, so not that much traffic) where I experience the same thing after upgrading to gw2012 sp1 on OES11 SP1. GroupWise running on NSS volume.

GWIA stops processing email and the only thing in the logs are about GroupWise link:
12:34:00 F475 The agent is attempting to reestablish the GroupWise link.
12:34:10 F475 GroupWise link recovery successful.

Does not reporte "path create failed" as others in this thread.

Only place I see abnormal behaviour is on GWIA http monitoring:
All queues in "Queue Information" shows Count to "-1".

Restarting GWIA and everything is working again.

Plenty of diskspace on all partitions, not using POP nor IMAP - only SMTP.

Patrick Frontéri # GroupWiseR of Norway # Active part of the OpenHorizons and NetworkPartners.no Collaboration
0 Likes
Highlighted
Trusted Contributor.
Trusted Contributor.

Re: GWIA operation failing

Hi,

Nope, no resolution yet, however the "path create failed" hasn't happened again (yet).

The other day however, I had the counters on -1 , and of course restarted gwia and it got back to normal operation. The log was showing a lot of "exceeded maximum number of open files" or something similar. I think I saw a TID stating that the issue should be fixed in the next service pack for GW.

Cheers

pfronteri;2249649 wrote:
Hi there,

Have you found a solution for this problem yet?
I have a customer (a very small one, so not that much traffic) where I experience the same thing after upgrading to gw2012 sp1 on OES11 SP1. GroupWise running on NSS volume.

GWIA stops processing email and the only thing in the logs are about GroupWise link:
12:34:00 F475 The agent is attempting to reestablish the GroupWise link.
12:34:10 F475 GroupWise link recovery successful.

Does not reporte "path create failed" as others in this thread.

Only place I see abnormal behaviour is on GWIA http monitoring:
All queues in "Queue Information" shows Count to "-1".

Restarting GWIA and everything is working again.

Plenty of diskspace on all partitions, not using POP nor IMAP - only SMTP.
0 Likes
Highlighted
Absent Member.
Absent Member.

Re: GWIA operation failing

This problem sounds like the one we fixed that has been addressed in the next release of 12.01 hot patch 1

It was a File descriptor / socket leak issue:
We found it when we enabled more of the IP v6 and update our DNS host was returning alot of IP v6 queries with not found driving up Sockets
we were use on linux. I believe linux maps a file descriptor = socket, this could be why we are seeing it error out over time when creating a file.
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.