HareB Absent Member.
Absent Member.
1418 views

GW 8.0.3 hp1 Abends Randomly

Hello, GroupWise gurus. I recently upgraded our GroupWise 7.0.1 up to GW 8.0.3 hp1. Since then the server has been experiencing an ABEND that has me concerned; the server will lock and display a blue screen that shows a stack trace in yellow letters. I have to manually power-cycle the server. No abend log is left behind. I've taken a photo of the latest abend with my smartphone (link here.)

Here is a config report from the server. Here is the SYS$LOG.ERR file. Hardware is a DL380G7 with 25gb RAM. RAID 6 on an SA 410i controller.

Here are the facts.

1) Updated to GW 8.0.3 on Sunday, Dec. 16
2) On Tuesday, Dec. 18 I applied "GroupWise 8.0 SP3 Hot Patch 1 Netware Viewers x86-64 x86"
3) Server abended on the morning of Dec. 20
4) Server abended on the morning of Dec. 26
5) Server abended midday on Jan 03. Took a picture of the abend screen.
6) No abend.log file has been left by any of these abends

Quite frankly I'm baffled as to how to proceed to troubleshoot this issue and need some direction from others more qualified them myself. Any suggestions would be greatly appreciated.


-Benjamin N. Hare
Labels (2)
Tags (1)
0 Likes
11 Replies
Bob-O-Rama
Visitor.

Re: GW 8.0.3 hp1 Abends Randomly

This is a "Multiple Abends Are Occurring" state where, as its says, you are hosed. Someone threw a grenade into the china shop. It appears that the initial abend occured when loading a module. This can be any module. If the abend occurs at a specific time(s) then it may well be something like backups or log rolling events.

You need to identify where the crash is coming from. So be sure to

SET AUTO RESTART AFTER ABEND = 0

This will prompt you, and require manual intervention, when an ABEND occurs. You will want to "Update ABEND.LOG and Exit." Then restart the server. The ABEND.LOG should indicate what happened, you can post the whole log using pastebin or similar.

And, since you have neglected to mention it, what version and patch level of NetWare are you running? Are your agents running in protected memory? The converters do, and older versions of NetWare did not necessarily protect the kernel from crashes as well.

-- Bob
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

Bob, thanks so much for your reply. My apologies for leaving out that critical piece of information. This server is running NetWare 6.5 Support Pack Revision 08. Agents are NOT running in protected memory.

As you recommended I've set AUTO RESTART AFTER ABEND = 0.
0 Likes
Bob-O-Rama
Visitor.

Re: GW 8.0.3 hp1 Abends Randomly

HareB;2238808 wrote:
Bob, thanks so much for your reply. My apologies for leaving out that critical piece of information. This server is running NetWare 6.5 Support Pack Revision 08. Agents are NOT running in protected memory.

As you recommended I've set AUTO RESTART AFTER ABEND = 0.


Unfortunately this means further inconveniencing your user population. But hopefully you can tell whats happening. You may want to revert to the original viewers to see if this is causing the issue.

-- Bob
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

Finally managed to get an abend log (link here.)

I found multiple errors in the SYS$LOG.ERR (link here) regarding the short term memory allocator. The error suggested that I should add -u671252480 to server.exe, which I did. I've since restarted the server. Other than that change to server.exe no other change has been made.
0 Likes
Bob-O-Rama
Visitor.

Re: GW 8.0.3 hp1 Abends Randomly

Uh, tell me about the server hardware, cores and so on....

The abend log, if it can be believed, says this server has 24 CPUs? Really?

Abend 1 on P23:

Anyway, the crash is in handling IMAP, and falls into the TCPIP stack. So I would apply the post NW 6.5 SP 8 patches.... like TCPIP for NW65SP8 29012010


As for the memory. Yeah, that's a problem. If this server was, at some point "tuned" for memory, you should undo that and let the server deal with it. As SP8 handled it very well. There should be no need on SP8 to manually tune. How much RAM does this system have?

-- Bob
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

No, not really.

The server is an HP ProLiant DL380 G7, dual Intel X5680 cpus. This cpu has 6 cores and 12 threads; I'm assuming these threads are seen as individual processors by NetWare, which would explain why there are 24 of them. This server has 25gb of RAM.

Thanks for your advice, Bob. I made two changes to the server.

* Applied the TCPIP for NW65SP8 29012010 patch that you pointed out
* Removed the -u671252480 from the AUTOEXEC.BAT file as you recommended

Having restarted the server I'm waiting to see how it behaves. Thanks for pointing out the post SP8 patches; I'm chagrined to admit that I had not even considered looking at those.
0 Likes
Bob-O-Rama
Visitor.

Re: GW 8.0.3 hp1 Abends Randomly

I'm still concerned about the P23: business. That means you have hyper-threading enabled, which is a no-no for NetWare. Next time you reboot, disable HT.

Running NetWare on this server is, to be honest, a complete waste. If an opportunity presents itself, you could install OES Linux, possibly without having to re-do the disks holding the PO. The stability and performance difference for us was well worth the move.

The problem with NetWare is that significant pieces of the kernel code need to run on CPU 0, as a consequence application threads are frequently marshaled to CPU 0, creating a bottleneck. Linux has ( a lot ) less of this, which allows more efficient use of the cores. We have 4 POA, 4 MTA, Webaccess, GWIA, .... running on the same server.

-- Bob
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

Moving to OES11 is a project I am working on at present; migrating GW 8 over to GW 2012 running on OES11. Personally, I'm finding the Linux learning curve is quite steep as a result I'm not confident in my ability to support GW running on Linux. However, this current situation will certainly accelerate that process out of necessity.

I will disable HT during my next maintenance window, unless the server abends again and I'm forced to restart it before that time.
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

Server locked up again last night but thanks to the SET AUTO RESTART AFTER ABEND = 0 we got an abend log; this time the abend was caused by a thread owned by GWINTER.NLM. (Link here.) The server was not up long enough for me to get to a maintenance window, thus I have not yet disabled HT. Is there a possibility that HT is affecting the stability of the server?

Judging from the change in the abend log from 2013-01-11 and 2013-01-16 it would appear that the TCPIP for NW65SP8 29012010 patch resolved our GWIA.NLM abend. Now we've got an issue with GWINTER.NLM. Any suggestions on how to tackle that one?
0 Likes
HareB Absent Member.
Absent Member.

Re: GW 8.0.3 hp1 Abends Randomly

Problem has been resolved. Here is what I did.


  1. SET AUTO RESTART AFTER ABEND = 0, allowing me to get an abend log
  2. Applied the TCPIP for NW65SP8 29012010 patch, as Bob suggested
  3. Removed the -u671252480 from the AUTOEXEC.BAT file, as Bob suggested
  4. Abends after the patch pointed to NIC driver, so I updated to BX2.LAN v5.01 which is the latest release
  5. Moved GWIA into protected memory, isolating the fault and stabilizing the server


    I then opened a support ticket with Novell. Tech had me add the following into the GWIA.CFG file:

    /imapreadlimit 2
    /imapreadnew
    /udpslap

    Both GWIA and WebAccess are stable. Problem resolved. Thanks so much for your help, Bob. Hopefully this thread will prove helpful to others in the future.
0 Likes
Bob-O-Rama
Visitor.

Re: GW 8.0.3 hp1 Abends Randomly

Good to hear things are better. The /udpslap ( --udpslap for Linux ) is a "cure all" for a set of issues with the POA. I usually want to see the IMAP and POP stuff doe via the GWIA to insulate the POA and this is your touch point with the Internet.

FWIW, for us, installing GroupWise on SLES it becomes more or less an appliance. Ypu don't need to be a Linux genius. We rarely have to mess with it. If agents die, they restart ( without killing the Universe ). Its much faster. ( I mean MUCH faster ). The move from NW to Linux for the same version of GroupWise is pretty much of a non-event which you can accomplish with a few hours of down time if you plan it right.

-- Bob
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.