When executing a reboot/shutdown
k8s pods are not receiving the SIGTERM
signal which leads some of them to
unexpected behaviour such as generating
huge coredumps.
There is an upstream issue regarding this:
https://github.com/kubernetes/kubernetes/issues/107158
The problem seems to be systemd related
but this commit addresses the problem
with a workaround.
This commit introduces a new script that
will cleanup all the remaing pods and will
be run after kubelet is stopped.
The script is executed successfully when
kubelet stops and the pods are stopped
before the system shuts down.
Closes-bug: 1964111
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: Ia0376aa510dd0dc3983e16cd89840726c15d6c92
The script will run everytime before the kubelet service is started.
It reads the reserved-cpus list for the kubelet from the service
environment file and sanitizes it on the basis of online CPUs.
If none of the reserved cpus is online, it removes the
--reserved-cpus flag from the environment file which allows
the kubelet to choose CPUs itself.
Sanitizing the reserved-cpus list everytime before the kubelet starts
assures that the kubelet will not fail to start due to unavailability
of one or more CPUs in the list.
By enabling or disabling CPU hyperthreading, available CPUs change.
This change will make sure changing CPU hyperthreading setting will
not lead to kubelet start failure after the system boots up.
Test Plan: (On AIO-SX)
PASS:
Initial Hyperthreading state: enabled
Host-lock->Reboot->Disable CPU hyperthreading and reboot->Host-unlock
Observe kubelet does not fail to start before host-unlock.
All pods states are as expected. Host-unlock succeeds.
PASS:
Initial Hyperthreading state: disabled
Host-lock->Reboot->Enable CPU hyperthreading and reboot->Host-unlock
Observe kubelet does not fail to start before host-unlock.
All pods states are as expected. Host-unlock succeeds.
PASS:
Manually restart the Kubelet service.
Observe that the kubelet does not fail to start.
All pods states are as expected.
PASS:
Host-lock->Host unlock (without any config change).
Observe that the kubelet does not fail to start.
All pods states are as expected.
PASS:
Packages built successfully on both Debian and CentOS.
Closes-Bug: 1955608
Change-Id: I699c5c36a56a50d4c48faa816edad69c17058079
Signed-off-by: Kaustubh Dhokte <kaustubh.dhokte@windriver.com>
Here are the changes needed for adding k8s v1.22.5
in StarlingX alongside with the changes needed
for the build environment to find and build the package.
The package builds successfully.
Deployed an iso with k8s 1.22.5 on
AIO-SX and AIO-DX. The deployment phase
works and the pods are up and running after
the upgrade completes.
Story: 2009789
Task: 44305
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: Ibb9be075fa0b1491b9ab1854ebb1fddf4df53461
