Highlighted
Absent Member.
Absent Member.
2865 views

[SOLVED] Upgrade from SLES10 SP3 & OES2 SP2 to SP4/SP3 breaks NCS

Hi folks,

(This post was originally meant to be a rant and a request for help, but
while writing the final paragraph i found the solution. It's still a
rant, but i figured i'd post my solution here in case someone else runs
into the same issue.)

I've just spent several hours banging my head against a broken cluster
node. My system is a 32-bit SLES 10 VM running on VMware ESX 3.5.x.

I upgraded from SLES10 SP3 and OES2 SP2 to the next service packs for
each (using the move-to-oes-sp3 script in yast2 online_update).
Everything went well for the first few update/reboot sequences, then
after the final reboot on SLES10 SP4 & OES2 SP3, cluster services would
not load or join the cluster on restart.

I checked dmesg and found errors about "Loading module compiled for
kernel version 2.6.16.60-0.54.5-vmi" into a previous kernel version, so
i tried downgrading to that kernel version, only to find that it was
older than the one i had just upgraded from (it's the original SLES10
SP3 kernel). So i tried upgrading back to the same kernel which is
running on the other cluster node (2.6.16.60-0.77.1-vmi), but that did
not work any better.

<preaching>
I have to say that i'm not impressed that OES2 SP3 isn't even compiled
against the appropriate kernel, and because of SUSE's kernel RPM
overwrite policy there's no way i can select to boot from a previous
kernel to see if that fixes things. Note to SUSE and other distro
builders: if you're not doing kernel package upgrades like Red Hat or
Ubuntu (so that we can select to boot from the previous kernel from the
boot menu), you're doing it *WRONG*.
</preaching>

I then upgraded again to the latest recommended kernel for SLES10 SP4,
and still no joy. Dmesg shows this error before the rot starts:

allocation failed: out of vmalloc space - use vmalloc=<size> to increase
size.

When searching for this error i stumbled across
http://ubuntuforums.org/showthread.php?t=1613132
which pointed me to
http://www.mythtv.org/wiki/Common_Problem:_vmalloc_too_small

Adding vmalloc=192M to /boot/grub/menu.lst and rebooting solved the
problem for me.

Regards,
Paul
Labels (1)
0 Likes
19 Replies
Highlighted
Knowledge Partner
Knowledge Partner

On 12/10/2011 16:46, magic31 wrote:

> With SLES AFAIK you can do an inplace upgrade from 32 to 64 bit


That may be true for SLES but OES is different. Whilst I've heard reports
that it works I think I'm right in saying that in-place upgrades from
32-bit OESx (on SLESx) to 64-bit OESx are not currently supported.

HTH.
--
Simon
Novell Knowledge Partner (NKP)
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

Paul Gear;2146004 wrote:
On 13/10/11 08:46, magic31 wrote:
> ...
> Somewhat offtopic to what the OP is reporting....


I am the OP... 😉


Oeps... 🙂
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

smflood;2146052 wrote:
On 12/10/2011 16:46, magic31 wrote:

> With SLES AFAIK you can do an inplace upgrade from 32 to 64 bit


That may be true for SLES but OES is different. Whilst I've heard reports
that it works I think I'm right in saying that in-place upgrades from
32-bit OESx (on SLESx) to 64-bit OESx are not currently supported.

HTH.
--
Simon
Novell Knowledge Partner (NKP)


Hey Simon,

Yes... I intended to split the two out in seperate lines (SLES vs OES options in howto). To further clarify, with migration matrix for OES2/11 I meant looking into miggui which can do 32bit to 64bit and indeed is not an inplace.

Cheers,
Willem
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

Paul Gear;2146015 wrote:
On 13/10/11 09:26, magic31 wrote:
> ...
> I'd go with Kevin on that. Have only had one production cluster run
> inside VMware and that ran ok with that parameter...
> With all the recovery options virtualization brings and also with Linux
> in general, have been mostly using standalone vm's. The only 'pain'
> there compared to clustering, is the extra downtime needed when patching
> the OS as you don't have the option of migrating resources. For us that
> has not really been an issue.


If patching were as easy on SLES/OES as it is on Debian, that might be
an option, but if we've only got 30 minutes of downtime, we can't
possibly complete a service pack application in that time, even on a
fast server with a local SMT mirror. Even the "smooth" service pack i
did two days ago on our non-clustered OES2 master replica took well over
90 minutes.

Paul


I'm not familiar with Debian, but I patch my servers "live" all the time. No user interruption UNTIL I reboot the server, and then it's only a 5 minute (or thereabouts) downtime. With clustering it doesn't really matter as you migrate to another server and patch it then.

But i've got about 35 WAN locations where we patch across the WAN link, so we'll start the patching in the morning, it'll take several hours to download the patches and maybe 30 minutes to apply (it really depends on how long our last patch run was). then we reboot it "later" at 5:00 p.m.
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

magic31;2146089 wrote:
Hey Simon,

Yes... I intended to split the two out in seperate lines (SLES vs OES options in howto). To further clarify, with migration matrix for OES2/11 I meant looking into miggui which can do 32bit to 64bit and indeed is not an inplace.

Cheers,
Willem


Hmmm.. I just checked the OES 2 migration docs... but 32>64 in not mentioned ....
I'll check with Novell what the current state actually is, as I have this in my head as a fact since OES 2 SP3... 😕
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

It's one of those "SUSE" things. Meaning SLES doesn't support 32 to 64-bit "inplace" upgrade. Although I'm not aware of any other Linux distro that does either (RHEL, etc.)

Supposedly it CAN be done, but use at your own risk and there's some issues with java stuff.

Joe D. has tested one.
0 Likes
Highlighted
Knowledge Partner
Knowledge Partner

kjhurni;2146096 wrote:
It's one of those "SUSE" things. Meaning SLES doesn't support 32 to 64-bit "inplace" upgrade. Although I'm not aware of any other Linux distro that does either (RHEL, etc.)

Supposedly it CAN be done, but use at your own risk and there's some issues with java stuff.

Joe D. has tested one.


Yes, when talking inplace.... but I am specifically talking about miggui to migrate from server to server.

-Willem
0 Likes
Highlighted
Absent Member.
Absent Member.

On 14/10/11 01:26, kjhurni wrote:
> ...
> I'm not familiar with Debian, but I patch my servers "live" all the
> time. No user interruption UNTIL I reboot the server, and then it's
> only a 5 minute (or thereabouts) downtime. With clustering it doesn't
> really matter as you migrate to another server and patch it then.
>
> But i've got about 35 WAN locations where we patch across the WAN link,
> so we'll start the patching in the morning, it'll take several hours to
> download the patches and maybe 30 minutes to apply (it really depends on
> how long our last patch run was). then we reboot it "later" at 5:00
> p.m.


That might work for minor patch upgrades, but service packs need all the
latest updates for the previous service pack applied (sometimes needing
a reboot), then the service pack installed (needing a reboot and an eDir
schema upgrade), then the latest updates for the new service pack
applied (sometimes needing a reboot). And there can be 30-45 minute
lags between each reboot.

I consider this a fundamental limitation of SUSE's update system,
because they don't just install the latest version when it's time to
install - you have to install, then upgrade. Having worked on
Debian/Ubuntu, it's like chalk and cheese - apt is much faster and
installs the latest version by default. Even if you do a new install
from CD, if you give it an online repo during OS installation it will
install the latest versions as it goes.

Paul
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.