Our vBulletin migration is complete.
Welcome vBulletin users! All content and user information from the Micro Focus Forums (vBulletin) site has been migrated to this site. READ MORE.
Highlighted
Madddy Respected Contributor.
Respected Contributor.
516 views

SMAX kubelet services are not coming up

Hi All,

I have just stated working on SMAX.

We are using single master and 2 workers.

kube-status status is showing inactive status and most of the PODS are not running, Stopped Kubelet services on worker and master, then started the services on Master followed by workers, still no luck. 

space & system resources are good, kindly suggest as what need to be done.

I have attached output of below commands
kube-staus.sh
kubectl get pods --all-namespaces
kuberctl get nodes

Tags (1)
0 Likes
7 Replies
Misaq Acclaimed Contributor.
Acclaimed Contributor.

Re: SMAX kubelet services are not coming up

Depending on version of SMAX  it takes a lot of resources and time to come up especially in multi-node scenario.

Your attachements are missing.

In the meantime make sure that core services are up (mini CDF first then Full CDF) otherwise  suite services will not start.

0 Likes
Misaq Acclaimed Contributor.
Acclaimed Contributor.

Re: SMAX kubelet services are not coming up

0 Likes
Madddy Respected Contributor.
Respected Contributor.

Re: SMAX kubelet services are not coming up

Please find the attachment and out put of kubectl get nodesget nodes.jpg

0 Likes
Misaq Acclaimed Contributor.
Acclaimed Contributor.

Re: SMAX kubelet services are not coming up

Did you check resources on master node

free -g

df -kh

Full CDF  db and pods are still not running so one of them has problems. It might be some leftover from earlier instances.

can do a kube-stop.sh       and then make sure everything is stop by ps -ef| grep dock      and gerp kube to make sure no other instances are running.

 

 

0 Likes
Francis Feugue Outstanding Contributor.
Outstanding Contributor.

Re: SMAX kubelet services are not coming up

It looks like this system has been working since more than 90 days and suddenly stop working? In most of the cases I have seen till now it could happen when the disk space is full or the rabbitmq is crashed (this is not your case because even CDF is not up) or DNS issue due to OS auto update.

  • Check the disk space on each of the server using df -h
  • Check if it is working with the proxy-mode parameter set to userspace instead of iptables

Run

*vi /usr/lib/systemd/system/kube-proxy.service

*change from proxy-mode=iptables to proxy-mode=userspace

*save

Then run:

systemctl daemon-reload

systemctl restart kubelet

 

And run: ./kube-restart.sh

   If issue persists after all then I would strongly suggest to open a support case and share the ID

Madddy Respected Contributor.
Respected Contributor.

Re: SMAX kubelet services are not coming up

Thanks for your response.

After a Day with out untouched all the services were up, what I have noticed is difference b/t worker server configuration. we have 2 worker,

Master: RAM 16 & CPU 32

Worker - 1: RAM 8 & CPU 8

Worker -2: RAM 16 & CPU 32

most of the time my worker CPU has high utilization and I could see high memory consumtion, this is a demo setup with just 1 or 2 users still SMAX services goes down after a day. is this because of mismatch in worker configuration? other wise I see enough resource for DEMO setup.

Solution: Restart the SMAX services from all 3 server and wait for aprox 24 hours for the services to start. I am suprised with the time it is consuming to start complete setup.

Regards,
Madhan

0 Likes
Francis Feugue Outstanding Contributor.
Outstanding Contributor.

Re: SMAX kubelet services are not coming up

Hello,

Yes 24 hours is too long

I would suggest to strickly follow the sizing guide documented on the URL below

https://docs.microfocus.com/itom/Service_Management_Automation_-_X:2018.08/Sizing-for-on-premises-deployment_19895859

btw, 8 GB RAM seems to be very very low

 

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.