Remote Manager not working after 23.4 Upgrade

Question

Hi. Anybody with a working remote manager aka novell-httpstk after upgrading OES2018 to 23.4? On all servers I upgraded, novell-httpstkd won't start. Logs are really not helpful: massi:~ # journalctl -xeu novell-httpstkd.service
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit novell-httpstkd.service has entered the 'failed' state with result 'timeout'.
Dec 22 00:12:58 massi systemd[1]: novell-httpstkd.service: Unit process 1726 (httpstkd) remains running after unit stopped.
Dec 22 00:12:58 massi systemd[1]: Failed to start Novell httpstkd Daemon.
░░ Subject: A start job for unit novell-httpstkd.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit novell-httpstkd.service has finished with a failure.
░░
░░ The job identifier is 933025 and the job result is failed.
Dec 22 00:12:58 massi systemd[1]: novell-httpstkd.service: Scheduled restart job, restart counter is at 5386.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ Automatic restarting of the unit novell-httpstkd.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Dec 22 00:12:58 massi systemd[1]: Stopped Novell httpstkd Daemon.
░░ Subject: A stop job for unit novell-httpstkd.service has finished
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A stop job for unit novell-httpstkd.service has finished.
░░
░░ The job identifier is 933118 and the job result is done.
Dec 22 00:12:58 massi systemd[1]: Starting Novell httpstkd Daemon...
░░ Subject: A start job for unit novell-httpstkd.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit novell-httpstkd.service has begun execution.
░░
░░ The job identifier is 933118.
Dec 22 00:12:58 massi systemd[1]: novell-httpstkd.service: Can't open PID file /run/httpstkd.pid (yet?) after start: Operation not permitted

marvhuffaker · Answer

I found a workable solution to this issue that I have tested on several different systems and it seems to do the trick. I have numerous more I can test on and will do so to confirm. I tried to compare a working New build vs a non-working upgrade and while there were some file differences I couldn't find anything that fixed it. So as someone else mentioned it seemed like a timing issue on startup. I shifted my efforts to this and found that I had success. So I took the novell-httpstkd.service and I forced it to load later in the boot process. This file contains the startup parameters for the novell-httpstkd service: 
 /usr/lib/systemd/system/novell-httpstkd.service. I edited this file, and below is my edited version. The highlighted parts are the changes: 
 [Unit] Documentation=man:systemd-sysv-generator(8) Description=Novell httpstkd Daemon Before= After=network-online.target multi-user.target graphical.target Wants=network-online.target multi-user.target graphical.target 
 [Service] Type=forking Restart=on-failure TimeoutSec=180 IgnoreSIGPIPE=no KillMode=process GuessMainPID=no PIDFile=/var/run/httpstkd.pid ExecStart=/opt/novell/httpstkd/sbin/httpstkd ExecStop=/bin/kill -TERM $MAINPID 
 [Install] WantedBy=multi-user.target 
 ----------------------------------------------------------------------------------------------------------- 
 To explain what this does.. The lines to note are in the [unit] section and are the "Before=", "After=", and "Wants" directives. I moved all the targets to the "After=" statement. Below is the original config file: 
 [Unit] Documentation=man:systemd-sysv-generator(8) Description=Novell httpstkd Daemon Before=multi-user.target multi-user.target graphical.target After=network-online.target Wants=network-online.target 
 Based on my understanding of this, the original configuration allows the service to start once the network is active, but before the system is fully booted. The change I made tells the service to wait until after the multi-user.target and the graphical.target are also loaded. This seems to avoid whatever collision is happening with the original config. If it was known what service it was colliding with, it could be tweaked to just list that service instead. 
 Here you can see the service status as well as the PID, which confirms it did activate correctly on startup rather than being started manually later (which would have resulted in a higher PID). 
 ------------------------------------

Other Notes to Point Out

Yes, there are still some errors displayed on startup but those seem to be cosmetic. 
 Because the service is delayed until after the GUI loads, if you putty in shortly after a reboot, it's possible that the service has not even tried to load yet. Wait until the GUI is fully loaded and at a login prompt, and then you should see the service loading finally. 
 I have not seen or observed any negative side effects for making this change, and would not expect any. 
 On the servers I have tested, I have the GUI installed. I have not tested this on a system without the graphical target.

Georg H. Weber · Answer

a strace of the httpstkd shows the behavior of the start of the service. It is also important to know that from a SLES version that I do not currently know by heart, the tmpfs is mounted to /var and thus in some cases no pid file dir must be specified for services. The systems I have access to come from OES 11 and have been upgraded to 24.2. The one or other system sometimes makes problems, this can be fixed quickly if it is clear where to intervene. At the moment there is actually a maloc in ndsd, but it can be intercepted with nds tuning. you need to have experience to analyze the maloc. to the httptkd itself. 
 please make sure that the lines in which the 32 bit plugins are loaded are removed and only the necessary 64 bit plugins are loaded. The plugins reload functions in the Remote Manager. Assuming that the certs are correct and the symlinks are also correct, it can be said that the application has been written in such a way that it checks itself for errors at startup and performs a kill hub and restarts after a standard 180 seconds. 
 
 A correct start looks like this: systemctl status novell-httpstkd.service novell-httpstkd.service - Novell httpstkd Daemon Loaded: loaded (/usr/lib/systemd/system/novell-httpstkd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2024-06-27 12:05:54 CEST; 20s ago Docs: man:systemd-sysv-generator(8) Process: 8830 ExecStart=/opt/novell/httpstkd/sbin/httpstkd (code=exited, status=0/SUCCESS) Tasks: 23 CGroup: /system.slice/novell-httpstkd.service 8834 /opt/novell/httpstkd/sbin/httpstkd Jun 27 12:05:54 server systemd[1]: Starting Novell httpstkd daemon... Jun 27 12:05:54 server systemd[1]: Starting Novell httpstkd daemon... 
 sorry, I made a mistake at this point, my fingers were faster than my head. In SLES 15 the tmpfs is mounted to /run. A service can store a pid file via the tmpfs in /run without knowing where its pid file is. This is the reason why the pid file dir has been omitted below. After starting, the pid file is created correctly by the service or removed again when it is stopped. 
 
 pid file after start service in /run 
 -rw-r--r-- 1 root root 5 Jun 27 12:05 httpstkd.pid the pid file is written under /run at startup Here is the config for it vi /usr/lib/systemd/system/novell-httpstkd.service [Unit] Documentation=man:systemd-sysv-generator(8) Description=Novell httpstkd Daemon Before=multi-user.target multi-user.target graphical.target After=network-online.target Wants=Netzwerk-online.target [Service] Type=forking Restart=on-failure TimeoutSec=180 IgnoreSIGPIPE=no KillMode=process GuessMainPID=no #PIDFile=/var/run/httpstkd.pid ExecStart=/opt/novell/httpstkd/sbin/httpstkd ExecStop=/bin/kill -TERM $MAINPID [Install] WantedBy=multi-user.target 
 if you want to know more: man 8 systemd-sysv-generator. Keywords here are SysV init LSB headers, init scripts /etc/init.d/ and, above all, you can read how what is put together. This knowledge is important. 
 
 George

lcmartines · Answer

Hello guys, 
 I was following this thread and my OES 23.4 /OES24.3 Remote Manager still was not working, then I discovered that: 
 Solution: A working OES 23.4 server's Remote Manager certificate files: 
 oes-mailsrv:/opt/novell/lib # ls -la /etc/opt/novell/httpstkd/ total 32 drwxr-xr-x 2 root root 4096 Feb 22 20:25 ./ drwxr-xr-x 23 root root 4096 Jul 26 12:21 ../ -rwxr-xr-x 1 root root 24570 Sep 6 2023 mimetyp.txt* lrwxrwxrwx 1 root root 34 Aug 31 2020 server.key -> /etc/ssl/servercerts/serverkey.pem lrwxrwxrwx 1 root root 35 Aug 31 2020 server.pem -> /etc/ssl/servercerts/servercert.pem 
 A non working OES 23.4 server's Remote Manager certificate files: 
 oes18sp3:/opt/novell/lib64 # ls -la /etc/opt/novell/httpstkd/ total 40 drwxr-xr-x 2 root root 4096 Jul 25 17:49 . drwxr-xr-x 30 root root 4096 Jul 26 12:16 .. -rw-rw-rw- 1 root root 31 Jul 24 23:05 httpstkdCertReqIn -rwxr-xr-x 1 root root 24570 Mar 27 10:19 mimetyp.txt -r-------- 1 root root 1220 Jul 24 23:05 server.key lrwxrwxrwx 1 root root 35 Nov 10 2021 server.pem -> /etc/ssl/servercerts/servercert.pem 
 On the problem server, the server.key was incorrect, it should be a symlink to the /etc/ssl/servercerts/serverkey.pem file. After correcting that recreating the symlink with the command ln -s /etc/ssl/servercerts/serverkey.pem server.key, Remote Manager loads and works correctly. 
 Hope that helps. Seeya. Luiz M.

Menu

OES User Discussions

Verified Answers

Remote Manager not working after 23.4 Upgrade

Top Replies

Resources