=== RUN TestMultiControlPlane/serial/DegradedAfterClusterRestart
ha_test.go:390: (dbg) Run: out/minikube-linux-amd64 profile list --output json
ha_test.go:413: expected profile "ha-767488" in json of 'profile list' to have "Degraded" status but have "HAppy" status. got *"{\"invalid\":[],\"valid\":[{\"Name\":\"ha-767488\",\"Status\":\"HAppy\",\"Config\":{\"Name\":\"ha-767488\",\"KeepContext\":false,\"EmbedCerts\":false,\"MinikubeISO\":\"https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso\",\"KicBaseImage\":\"gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7\",\"Memory\":2200,\"CPUs\":2,\"DiskSize\":20000,\"Driver\":\"kvm2\",\"HyperkitVpnKitSock\":\"\",\"HyperkitVSockPorts\":[],\"DockerEnv\":null,\"ContainerVolumeMounts\":null,\"InsecureRegistry\":null,\"RegistryMirror\":[],\"HostOnlyCIDR\":\"192.168.59.1/24\",\"HypervVirtualSwitch\":\"\",\"HypervUseExternalSwitch\":false,\"HypervExternalAdapter\":\"\",\"KVMNetwork\":\"default\",\"KVMQemuURI\":\"qemu:///system\",\"KVMGPU\":false,\"KVMHidden\":false,\"KVMNUMACount\":1,\"
APIServerPort\":8443,\"DockerOpt\":null,\"DisableDriverMounts\":false,\"NFSShare\":[],\"NFSSharesRoot\":\"/nfsshares\",\"UUID\":\"\",\"NoVTXCheck\":false,\"DNSProxy\":false,\"HostDNSResolver\":true,\"HostOnlyNicType\":\"virtio\",\"NatNicType\":\"virtio\",\"SSHIPAddress\":\"\",\"SSHUser\":\"root\",\"SSHKey\":\"\",\"SSHPort\":22,\"KubernetesConfig\":{\"KubernetesVersion\":\"v1.30.3\",\"ClusterName\":\"ha-767488\",\"Namespace\":\"default\",\"APIServerHAVIP\":\"192.168.39.254\",\"APIServerName\":\"minikubeCA\",\"APIServerNames\":null,\"APIServerIPs\":null,\"DNSDomain\":\"cluster.local\",\"ContainerRuntime\":\"crio\",\"CRISocket\":\"\",\"NetworkPlugin\":\"cni\",\"FeatureGates\":\"\",\"ServiceCIDR\":\"10.96.0.0/12\",\"ImageRepository\":\"\",\"LoadBalancerStartIP\":\"\",\"LoadBalancerEndIP\":\"\",\"CustomIngressCert\":\"\",\"RegistryAliases\":\"\",\"ExtraOptions\":null,\"ShouldLoadCachedImages\":true,\"EnableDefaultCNI\":false,\"CNI\":\"\"},\"Nodes\":[{\"Name\":\"\",\"IP\":\"192.168.39.217\",\"Port\":8443,\"Kubernet
esVersion\":\"v1.30.3\",\"ContainerRuntime\":\"crio\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m02\",\"IP\":\"192.168.39.45\",\"Port\":8443,\"KubernetesVersion\":\"v1.30.3\",\"ContainerRuntime\":\"crio\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m03\",\"IP\":\"192.168.39.210\",\"Port\":8443,\"KubernetesVersion\":\"v1.30.3\",\"ContainerRuntime\":\"crio\",\"ControlPlane\":true,\"Worker\":true},{\"Name\":\"m04\",\"IP\":\"192.168.39.181\",\"Port\":0,\"KubernetesVersion\":\"v1.30.3\",\"ContainerRuntime\":\"crio\",\"ControlPlane\":false,\"Worker\":true}],\"Addons\":{\"ambassador\":false,\"auto-pause\":false,\"cloud-spanner\":false,\"csi-hostpath-driver\":false,\"dashboard\":false,\"default-storageclass\":false,\"efk\":false,\"freshpod\":false,\"gcp-auth\":false,\"gvisor\":false,\"headlamp\":false,\"helm-tiller\":false,\"inaccel\":false,\"ingress\":false,\"ingress-dns\":false,\"inspektor-gadget\":false,\"istio\":false,\"istio-provisioner\":false,\"kong\":false,\"kubeflow\":false,\"kubevirt\":false
,\"logviewer\":false,\"metallb\":false,\"metrics-server\":false,\"nvidia-device-plugin\":false,\"nvidia-driver-installer\":false,\"nvidia-gpu-device-plugin\":false,\"olm\":false,\"pod-security-policy\":false,\"portainer\":false,\"registry\":false,\"registry-aliases\":false,\"registry-creds\":false,\"storage-provisioner\":false,\"storage-provisioner-gluster\":false,\"storage-provisioner-rancher\":false,\"volcano\":false,\"volumesnapshots\":false,\"yakd\":false},\"CustomAddonImages\":null,\"CustomAddonRegistries\":null,\"VerifyComponents\":{\"apiserver\":true,\"apps_running\":true,\"default_sa\":true,\"extra\":true,\"kubelet\":true,\"node_ready\":true,\"system_pods\":true},\"StartHostTimeout\":360000000000,\"ScheduledStop\":null,\"ExposedPorts\":[],\"ListenAddress\":\"\",\"Network\":\"\",\"Subnet\":\"\",\"MultiNodeRequested\":true,\"ExtraDisks\":0,\"CertExpiration\":94608000000000000,\"Mount\":false,\"MountString\":\"/home/jenkins:/minikube-host\",\"Mount9PVersion\":\"9p2000.L\",\"MountGID\":\"docker\",\"MountI
P\":\"\",\"MountMSize\":262144,\"MountOptions\":[],\"MountPort\":0,\"MountType\":\"9p\",\"MountUID\":\"docker\",\"BinaryMirror\":\"\",\"DisableOptimizations\":false,\"DisableMetrics\":false,\"CustomQemuFirmwarePath\":\"\",\"SocketVMnetClientPath\":\"\",\"SocketVMnetPath\":\"\",\"StaticIP\":\"\",\"SSHAuthSock\":\"\",\"SSHAgentPID\":0,\"GPUs\":\"\",\"AutoPauseInterval\":60000000000},\"Active\":false,\"ActiveKubeContext\":true}]}"*. args: "out/minikube-linux-amd64 profile list --output json"
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p ha-767488 -n ha-767488
helpers_test.go:244: <<< TestMultiControlPlane/serial/DegradedAfterClusterRestart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestMultiControlPlane/serial/DegradedAfterClusterRestart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-amd64 -p ha-767488 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-amd64 -p ha-767488 logs -n 25: (1.870985664s)
helpers_test.go:252: TestMultiControlPlane/serial/DegradedAfterClusterRestart logs:
-- stdout --
==> Audit <==
|---------|----------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|----------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
| cp | ha-767488 cp ha-767488-m03:/home/docker/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04:/home/docker/cp-test_ha-767488-m03_ha-767488-m04.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m03 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-767488 ssh -n ha-767488-m04 sudo cat | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | /home/docker/cp-test_ha-767488-m03_ha-767488-m04.txt | | | | | |
| cp | ha-767488 cp testdata/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04:/home/docker/cp-test.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| cp | ha-767488 cp ha-767488-m04:/home/docker/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | /tmp/TestMultiControlPlaneserialCopyFile2644279276/001/cp-test_ha-767488-m04.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| cp | ha-767488 cp ha-767488-m04:/home/docker/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488:/home/docker/cp-test_ha-767488-m04_ha-767488.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-767488 ssh -n ha-767488 sudo cat | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | /home/docker/cp-test_ha-767488-m04_ha-767488.txt | | | | | |
| cp | ha-767488 cp ha-767488-m04:/home/docker/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m02:/home/docker/cp-test_ha-767488-m04_ha-767488-m02.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-767488 ssh -n ha-767488-m02 sudo cat | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | /home/docker/cp-test_ha-767488-m04_ha-767488-m02.txt | | | | | |
| cp | ha-767488 cp ha-767488-m04:/home/docker/cp-test.txt | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m03:/home/docker/cp-test_ha-767488-m04_ha-767488-m03.txt | | | | | |
| ssh | ha-767488 ssh -n | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | ha-767488-m04 sudo cat | | | | | |
| | /home/docker/cp-test.txt | | | | | |
| ssh | ha-767488 ssh -n ha-767488-m03 sudo cat | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | /home/docker/cp-test_ha-767488-m04_ha-767488-m03.txt | | | | | |
| node | ha-767488 node stop m02 -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:25 UTC |
| | --alsologtostderr | | | | | |
| node | ha-767488 node start m02 -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:25 UTC | 29 Jul 24 12:26 UTC |
| | --alsologtostderr | | | | | |
| node | list -p ha-767488 -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:26 UTC | |
| | --alsologtostderr | | | | | |
| stop | -p ha-767488 -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:26 UTC | |
| | --alsologtostderr | | | | | |
| start | -p ha-767488 --wait=true -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:28 UTC | |
| | --alsologtostderr | | | | | |
| node | list -p ha-767488 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:37 UTC | |
| node | ha-767488 node delete m03 -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:37 UTC | |
| | --alsologtostderr | | | | | |
| stop | ha-767488 stop -v=7 | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:37 UTC | |
| | --alsologtostderr | | | | | |
| start | -p ha-767488 --wait=true | ha-767488 | jenkins | v1.33.1 | 29 Jul 24 12:40 UTC | 29 Jul 24 12:48 UTC |
| | -v=7 --alsologtostderr | | | | | |
| | --driver=kvm2 | | | | | |
| | --container-runtime=crio | | | | | |
|---------|----------------------------------------------------------------------------------|-----------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/07/29 12:40:51
Running on machine: ubuntu-20-agent-5
Binary: Built with gc go1.22.5 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0729 12:40:51.329866 260472 out.go:291] Setting OutFile to fd 1 ...
I0729 12:40:51.329974 260472 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0729 12:40:51.329984 260472 out.go:304] Setting ErrFile to fd 2...
I0729 12:40:51.329990 260472 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0729 12:40:51.330183 260472 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19341-233093/.minikube/bin
I0729 12:40:51.330779 260472 out.go:298] Setting JSON to false
I0729 12:40:51.331755 260472 start.go:129] hostinfo: {"hostname":"ubuntu-20-agent-5","uptime":8594,"bootTime":1722248257,"procs":187,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0729 12:40:51.331823 260472 start.go:139] virtualization: kvm guest
I0729 12:40:51.334313 260472 out.go:177] * [ha-767488] minikube v1.33.1 on Ubuntu 20.04 (kvm/amd64)
I0729 12:40:51.335770 260472 out.go:177] - MINIKUBE_LOCATION=19341
I0729 12:40:51.335784 260472 notify.go:220] Checking for updates...
I0729 12:40:51.338199 260472 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0729 12:40:51.339561 260472 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19341-233093/kubeconfig
I0729 12:40:51.340932 260472 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19341-233093/.minikube
I0729 12:40:51.342165 260472 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0729 12:40:51.343840 260472 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0729 12:40:51.345700 260472 config.go:182] Loaded profile config "ha-767488": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0729 12:40:51.346109 260472 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0729 12:40:51.346170 260472 main.go:141] libmachine: Launching plugin server for driver kvm2
I0729 12:40:51.362742 260472 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:34745
I0729 12:40:51.363165 260472 main.go:141] libmachine: () Calling .GetVersion
I0729 12:40:51.363711 260472 main.go:141] libmachine: Using API Version 1
I0729 12:40:51.363735 260472 main.go:141] libmachine: () Calling .SetConfigRaw
I0729 12:40:51.364108 260472 main.go:141] libmachine: () Calling .GetMachineName
I0729 12:40:51.364327 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:40:51.364586 260472 driver.go:392] Setting default libvirt URI to qemu:///system
I0729 12:40:51.365000 260472 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0729 12:40:51.365043 260472 main.go:141] libmachine: Launching plugin server for driver kvm2
I0729 12:40:51.379978 260472 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:42067
I0729 12:40:51.380389 260472 main.go:141] libmachine: () Calling .GetVersion
I0729 12:40:51.380778 260472 main.go:141] libmachine: Using API Version 1
I0729 12:40:51.380814 260472 main.go:141] libmachine: () Calling .SetConfigRaw
I0729 12:40:51.381158 260472 main.go:141] libmachine: () Calling .GetMachineName
I0729 12:40:51.381323 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:40:51.415931 260472 out.go:177] * Using the kvm2 driver based on existing profile
I0729 12:40:51.417174 260472 start.go:297] selected driver: kvm2
I0729 12:40:51.417189 260472 start.go:901] validating driver "kvm2" against &{Name:ha-767488 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVer
sion:v1.30.3 ClusterName:ha-767488 Namespace:default APIServerHAVIP:192.168.39.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.217 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m02 IP:192.168.39.45 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m03 IP:192.168.39.210 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m04 IP:192.168.39.181 Port:0 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:fals
e efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersi
on:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0729 12:40:51.417335 260472 start.go:912] status for kvm2: {Installed:true Healthy:true Running:true NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0729 12:40:51.417664 260472 install.go:52] acquiring lock: {Name:mk900956b073697a4aa6c80a27c6bb0742a99a53 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0729 12:40:51.417770 260472 install.go:117] Validating docker-machine-driver-kvm2, PATH=/home/jenkins/minikube-integration/19341-233093/.minikube/bin:/home/jenkins/workspace/KVM_Linux_crio_integration/out/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/local/go/bin:/home/jenkins/go/bin:/usr/local/bin/:/usr/local/go/bin/:/home/jenkins/go/bin
I0729 12:40:51.432545 260472 install.go:137] /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2 version is 1.33.1
I0729 12:40:51.433500 260472 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0729 12:40:51.433539 260472 cni.go:84] Creating CNI manager for ""
I0729 12:40:51.433548 260472 cni.go:136] multinode detected (4 nodes found), recommending kindnet
I0729 12:40:51.433631 260472 start.go:340] cluster config:
{Name:ha-767488 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.3 ClusterName:ha-767488 Namespace:default APIServerHAVIP:192.168.39
.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.217 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m02 IP:192.168.39.45 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m03 IP:192.168.39.210 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m04 IP:192.168.39.181 Port:0 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-til
ler:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPo
rt:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0729 12:40:51.433831 260472 iso.go:125] acquiring lock: {Name:mk286127eebdb995eadd3e9d10023ce1d15dc938 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0729 12:40:51.435545 260472 out.go:177] * Starting "ha-767488" primary control-plane node in "ha-767488" cluster
I0729 12:40:51.436699 260472 preload.go:131] Checking if preload exists for k8s version v1.30.3 and runtime crio
I0729 12:40:51.436735 260472 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19341-233093/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.3-cri-o-overlay-amd64.tar.lz4
I0729 12:40:51.436747 260472 cache.go:56] Caching tarball of preloaded images
I0729 12:40:51.436866 260472 preload.go:172] Found /home/jenkins/minikube-integration/19341-233093/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.3-cri-o-overlay-amd64.tar.lz4 in cache, skipping download
I0729 12:40:51.436877 260472 cache.go:59] Finished verifying existence of preloaded tar for v1.30.3 on crio
I0729 12:40:51.437012 260472 profile.go:143] Saving config to /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/config.json ...
I0729 12:40:51.437194 260472 start.go:360] acquireMachinesLock for ha-767488: {Name:mk080f325f153c5dcf3942d95ca198ed89cc6e64 Clock:{} Delay:500ms Timeout:13m0s Cancel:<nil>}
I0729 12:40:51.437233 260472 start.go:364] duration metric: took 21.45µs to acquireMachinesLock for "ha-767488"
I0729 12:40:51.437247 260472 start.go:96] Skipping create...Using existing machine configuration
I0729 12:40:51.437253 260472 fix.go:54] fixHost starting:
I0729 12:40:51.437521 260472 main.go:141] libmachine: Found binary path at /home/jenkins/workspace/KVM_Linux_crio_integration/out/docker-machine-driver-kvm2
I0729 12:40:51.437552 260472 main.go:141] libmachine: Launching plugin server for driver kvm2
I0729 12:40:51.451341 260472 main.go:141] libmachine: Plugin server listening at address 127.0.0.1:35705
I0729 12:40:51.451741 260472 main.go:141] libmachine: () Calling .GetVersion
I0729 12:40:51.452191 260472 main.go:141] libmachine: Using API Version 1
I0729 12:40:51.452220 260472 main.go:141] libmachine: () Calling .SetConfigRaw
I0729 12:40:51.452535 260472 main.go:141] libmachine: () Calling .GetMachineName
I0729 12:40:51.452723 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:40:51.452885 260472 main.go:141] libmachine: (ha-767488) Calling .GetState
I0729 12:40:51.454319 260472 fix.go:112] recreateIfNeeded on ha-767488: state=Running err=<nil>
W0729 12:40:51.454350 260472 fix.go:138] unexpected machine state, will restart: <nil>
I0729 12:40:51.456154 260472 out.go:177] * Updating the running kvm2 "ha-767488" VM ...
I0729 12:40:51.457351 260472 machine.go:94] provisionDockerMachine start ...
I0729 12:40:51.457369 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:40:51.457584 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:51.459878 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.460266 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:51.460296 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.460395 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:40:51.460553 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.460704 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.460782 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:40:51.460935 260472 main.go:141] libmachine: Using SSH client type: native
I0729 12:40:51.461114 260472 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.217 22 <nil> <nil>}
I0729 12:40:51.461124 260472 main.go:141] libmachine: About to run SSH command:
hostname
I0729 12:40:51.569205 260472 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-767488
I0729 12:40:51.569235 260472 main.go:141] libmachine: (ha-767488) Calling .GetMachineName
I0729 12:40:51.569499 260472 buildroot.go:166] provisioning hostname "ha-767488"
I0729 12:40:51.569524 260472 main.go:141] libmachine: (ha-767488) Calling .GetMachineName
I0729 12:40:51.569693 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:51.572499 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.572988 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:51.573033 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.573160 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:40:51.573358 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.573548 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.573648 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:40:51.573898 260472 main.go:141] libmachine: Using SSH client type: native
I0729 12:40:51.574069 260472 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.217 22 <nil> <nil>}
I0729 12:40:51.574089 260472 main.go:141] libmachine: About to run SSH command:
sudo hostname ha-767488 && echo "ha-767488" | sudo tee /etc/hostname
I0729 12:40:51.701204 260472 main.go:141] libmachine: SSH cmd err, output: <nil>: ha-767488
I0729 12:40:51.701229 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:51.703986 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.704423 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:51.704461 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.704639 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:40:51.704824 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.704975 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:51.705089 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:40:51.705288 260472 main.go:141] libmachine: Using SSH client type: native
I0729 12:40:51.705507 260472 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.217 22 <nil> <nil>}
I0729 12:40:51.705531 260472 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sha-767488' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 ha-767488/g' /etc/hosts;
else
echo '127.0.1.1 ha-767488' | sudo tee -a /etc/hosts;
fi
fi
I0729 12:40:51.817644 260472 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0729 12:40:51.817684 260472 buildroot.go:172] set auth options {CertDir:/home/jenkins/minikube-integration/19341-233093/.minikube CaCertPath:/home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19341-233093/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19341-233093/.minikube}
I0729 12:40:51.817700 260472 buildroot.go:174] setting up certificates
I0729 12:40:51.817709 260472 provision.go:84] configureAuth start
I0729 12:40:51.817719 260472 main.go:141] libmachine: (ha-767488) Calling .GetMachineName
I0729 12:40:51.818054 260472 main.go:141] libmachine: (ha-767488) Calling .GetIP
I0729 12:40:51.820835 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.821225 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:51.821246 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.821413 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:51.823391 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.823759 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:51.823788 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:51.823928 260472 provision.go:143] copyHostCerts
I0729 12:40:51.823969 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem -> /home/jenkins/minikube-integration/19341-233093/.minikube/ca.pem
I0729 12:40:51.824015 260472 exec_runner.go:144] found /home/jenkins/minikube-integration/19341-233093/.minikube/ca.pem, removing ...
I0729 12:40:51.824028 260472 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19341-233093/.minikube/ca.pem
I0729 12:40:51.824106 260472 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19341-233093/.minikube/ca.pem (1078 bytes)
I0729 12:40:51.824213 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/cert.pem -> /home/jenkins/minikube-integration/19341-233093/.minikube/cert.pem
I0729 12:40:51.824238 260472 exec_runner.go:144] found /home/jenkins/minikube-integration/19341-233093/.minikube/cert.pem, removing ...
I0729 12:40:51.824248 260472 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19341-233093/.minikube/cert.pem
I0729 12:40:51.824287 260472 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19341-233093/.minikube/cert.pem (1123 bytes)
I0729 12:40:51.824345 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/key.pem -> /home/jenkins/minikube-integration/19341-233093/.minikube/key.pem
I0729 12:40:51.824376 260472 exec_runner.go:144] found /home/jenkins/minikube-integration/19341-233093/.minikube/key.pem, removing ...
I0729 12:40:51.824384 260472 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19341-233093/.minikube/key.pem
I0729 12:40:51.824417 260472 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19341-233093/.minikube/key.pem (1679 bytes)
I0729 12:40:51.824477 260472 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19341-233093/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca-key.pem org=jenkins.ha-767488 san=[127.0.0.1 192.168.39.217 ha-767488 localhost minikube]
I0729 12:40:52.006332 260472 provision.go:177] copyRemoteCerts
I0729 12:40:52.006418 260472 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0729 12:40:52.006452 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:52.009130 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:52.009520 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:52.009546 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:52.009704 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:40:52.009964 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:52.010156 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:40:52.010326 260472 sshutil.go:53] new ssh client: &{IP:192.168.39.217 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/ha-767488/id_rsa Username:docker}
I0729 12:40:52.094644 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem -> /etc/docker/ca.pem
I0729 12:40:52.094738 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0729 12:40:52.119444 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/machines/server.pem -> /etc/docker/server.pem
I0729 12:40:52.119509 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/machines/server.pem --> /etc/docker/server.pem (1200 bytes)
I0729 12:40:52.143660 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/machines/server-key.pem -> /etc/docker/server-key.pem
I0729 12:40:52.143716 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0729 12:40:52.167324 260472 provision.go:87] duration metric: took 349.60091ms to configureAuth
I0729 12:40:52.167355 260472 buildroot.go:189] setting minikube options for container-runtime
I0729 12:40:52.167557 260472 config.go:182] Loaded profile config "ha-767488": Driver=kvm2, ContainerRuntime=crio, KubernetesVersion=v1.30.3
I0729 12:40:52.167627 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:40:52.170399 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:52.170750 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:40:52.170769 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:40:52.170976 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:40:52.171205 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:52.171383 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:40:52.171515 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:40:52.171707 260472 main.go:141] libmachine: Using SSH client type: native
I0729 12:40:52.171890 260472 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.217 22 <nil> <nil>}
I0729 12:40:52.171904 260472 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %!s(MISSING) "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0729 12:42:30.662176 260472 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0729 12:42:30.662209 260472 machine.go:97] duration metric: took 1m39.204842674s to provisionDockerMachine
I0729 12:42:30.662225 260472 start.go:293] postStartSetup for "ha-767488" (driver="kvm2")
I0729 12:42:30.662240 260472 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0729 12:42:30.662263 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:30.662582 260472 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0729 12:42:30.662612 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:42:30.665494 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.666063 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:30.666088 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.666235 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:42:30.666474 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:30.666633 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:42:30.666847 260472 sshutil.go:53] new ssh client: &{IP:192.168.39.217 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/ha-767488/id_rsa Username:docker}
I0729 12:42:30.752735 260472 ssh_runner.go:195] Run: cat /etc/os-release
I0729 12:42:30.757792 260472 info.go:137] Remote host: Buildroot 2023.02.9
I0729 12:42:30.757820 260472 filesync.go:126] Scanning /home/jenkins/minikube-integration/19341-233093/.minikube/addons for local assets ...
I0729 12:42:30.757900 260472 filesync.go:126] Scanning /home/jenkins/minikube-integration/19341-233093/.minikube/files for local assets ...
I0729 12:42:30.757994 260472 filesync.go:149] local asset: /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem -> 2403402.pem in /etc/ssl/certs
I0729 12:42:30.758009 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem -> /etc/ssl/certs/2403402.pem
I0729 12:42:30.758096 260472 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0729 12:42:30.768113 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem --> /etc/ssl/certs/2403402.pem (1708 bytes)
I0729 12:42:30.793284 260472 start.go:296] duration metric: took 131.040886ms for postStartSetup
I0729 12:42:30.793328 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:30.793694 260472 ssh_runner.go:195] Run: sudo ls --almost-all -1 /var/lib/minikube/backup
I0729 12:42:30.793729 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:42:30.796515 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.796959 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:30.796985 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.797155 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:42:30.797360 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:30.797508 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:42:30.797632 260472 sshutil.go:53] new ssh client: &{IP:192.168.39.217 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/ha-767488/id_rsa Username:docker}
W0729 12:42:30.883560 260472 fix.go:99] cannot read backup folder, skipping restore: read dir: sudo ls --almost-all -1 /var/lib/minikube/backup: Process exited with status 2
stdout:
stderr:
ls: cannot access '/var/lib/minikube/backup': No such file or directory
I0729 12:42:30.883593 260472 fix.go:56] duration metric: took 1m39.446338951s for fixHost
I0729 12:42:30.883619 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:42:30.886076 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.886458 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:30.886483 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:30.886633 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:42:30.886829 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:30.886996 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:30.887140 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:42:30.887303 260472 main.go:141] libmachine: Using SSH client type: native
I0729 12:42:30.887526 260472 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x82da80] 0x8307e0 <nil> [] 0s} 192.168.39.217 22 <nil> <nil>}
I0729 12:42:30.887541 260472 main.go:141] libmachine: About to run SSH command:
date +%!s(MISSING).%!N(MISSING)
I0729 12:42:30.997876 260472 main.go:141] libmachine: SSH cmd err, output: <nil>: 1722256950.957387407
I0729 12:42:30.997906 260472 fix.go:216] guest clock: 1722256950.957387407
I0729 12:42:30.997917 260472 fix.go:229] Guest: 2024-07-29 12:42:30.957387407 +0000 UTC Remote: 2024-07-29 12:42:30.883602483 +0000 UTC m=+99.589379345 (delta=73.784924ms)
I0729 12:42:30.997948 260472 fix.go:200] guest clock delta is within tolerance: 73.784924ms
I0729 12:42:30.997986 260472 start.go:83] releasing machines lock for "ha-767488", held for 1m39.560717836s
I0729 12:42:30.998041 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:30.998327 260472 main.go:141] libmachine: (ha-767488) Calling .GetIP
I0729 12:42:31.000905 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.001304 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:31.001335 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.001531 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:31.002184 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:31.002392 260472 main.go:141] libmachine: (ha-767488) Calling .DriverName
I0729 12:42:31.002499 260472 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0729 12:42:31.002576 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:42:31.002622 260472 ssh_runner.go:195] Run: cat /version.json
I0729 12:42:31.002652 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHHostname
I0729 12:42:31.005308 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.005500 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.005704 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:31.005737 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.005887 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:42:31.006092 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:31.006208 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:31.006233 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:31.006272 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:42:31.006395 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHPort
I0729 12:42:31.006459 260472 sshutil.go:53] new ssh client: &{IP:192.168.39.217 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/ha-767488/id_rsa Username:docker}
I0729 12:42:31.006551 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHKeyPath
I0729 12:42:31.006697 260472 main.go:141] libmachine: (ha-767488) Calling .GetSSHUsername
I0729 12:42:31.006864 260472 sshutil.go:53] new ssh client: &{IP:192.168.39.217 Port:22 SSHKeyPath:/home/jenkins/minikube-integration/19341-233093/.minikube/machines/ha-767488/id_rsa Username:docker}
I0729 12:42:31.115893 260472 ssh_runner.go:195] Run: systemctl --version
I0729 12:42:31.122469 260472 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0729 12:42:31.297345 260472 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
W0729 12:42:31.304517 260472 cni.go:209] loopback cni configuration skipped: "/etc/cni/net.d/*loopback.conf*" not found
I0729 12:42:31.304592 260472 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0729 12:42:31.316445 260472 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0729 12:42:31.316475 260472 start.go:495] detecting cgroup driver to use...
I0729 12:42:31.316547 260472 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0729 12:42:31.333639 260472 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0729 12:42:31.349241 260472 docker.go:217] disabling cri-docker service (if available) ...
I0729 12:42:31.349303 260472 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0729 12:42:31.364204 260472 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0729 12:42:31.378300 260472 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0729 12:42:31.534355 260472 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0729 12:42:31.684660 260472 docker.go:233] disabling docker service ...
I0729 12:42:31.684748 260472 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0729 12:42:31.700676 260472 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0729 12:42:31.715730 260472 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0729 12:42:31.862044 260472 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0729 12:42:32.012656 260472 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0729 12:42:32.026627 260472 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0729 12:42:32.048998 260472 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.9" pause image...
I0729 12:42:32.049086 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.9"|' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.060466 260472 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0729 12:42:32.060565 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.071761 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.082721 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.094732 260472 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0729 12:42:32.106637 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *"net.ipv4.ip_unprivileged_port_start=.*"/d' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.117985 260472 ssh_runner.go:195] Run: sh -c "sudo grep -q "^ *default_sysctls" /etc/crio/crio.conf.d/02-crio.conf || sudo sed -i '/conmon_cgroup = .*/a default_sysctls = \[\n\]' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.131937 260472 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^default_sysctls *= *\[|&\n "net.ipv4.ip_unprivileged_port_start=0",|' /etc/crio/crio.conf.d/02-crio.conf"
I0729 12:42:32.142195 260472 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0729 12:42:32.151406 260472 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0729 12:42:32.160525 260472 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0729 12:42:32.305601 260472 ssh_runner.go:195] Run: sudo systemctl restart crio
I0729 12:42:40.307724 260472 ssh_runner.go:235] Completed: sudo systemctl restart crio: (8.002069181s)
I0729 12:42:40.307768 260472 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0729 12:42:40.307825 260472 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0729 12:42:40.312866 260472 start.go:563] Will wait 60s for crictl version
I0729 12:42:40.312915 260472 ssh_runner.go:195] Run: which crictl
I0729 12:42:40.316658 260472 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0729 12:42:40.356691 260472 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.29.1
RuntimeApiVersion: v1
I0729 12:42:40.356775 260472 ssh_runner.go:195] Run: crio --version
I0729 12:42:40.385190 260472 ssh_runner.go:195] Run: crio --version
I0729 12:42:40.417948 260472 out.go:177] * Preparing Kubernetes v1.30.3 on CRI-O 1.29.1 ...
I0729 12:42:40.419401 260472 main.go:141] libmachine: (ha-767488) Calling .GetIP
I0729 12:42:40.422540 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:40.422892 260472 main.go:141] libmachine: (ha-767488) DBG | found host DHCP lease matching {name: "", mac: "52:54:00:f5:5c:b8", ip: ""} in network mk-ha-767488: {Iface:virbr1 ExpiryTime:2024-07-29 13:20:37 +0000 UTC Type:0 Mac:52:54:00:f5:5c:b8 Iaid: IPaddr:192.168.39.217 Prefix:24 Hostname:ha-767488 Clientid:01:52:54:00:f5:5c:b8}
I0729 12:42:40.422937 260472 main.go:141] libmachine: (ha-767488) DBG | domain ha-767488 has defined IP address 192.168.39.217 and MAC address 52:54:00:f5:5c:b8 in network mk-ha-767488
I0729 12:42:40.423110 260472 ssh_runner.go:195] Run: grep 192.168.39.1 host.minikube.internal$ /etc/hosts
I0729 12:42:40.427910 260472 kubeadm.go:883] updating cluster {Name:ha-767488 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.3 Cl
usterName:ha-767488 Namespace:default APIServerHAVIP:192.168.39.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.217 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m02 IP:192.168.39.45 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m03 IP:192.168.39.210 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m04 IP:192.168.39.181 Port:0 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false fre
shpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L Mou
ntGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0729 12:42:40.428052 260472 preload.go:131] Checking if preload exists for k8s version v1.30.3 and runtime crio
I0729 12:42:40.428107 260472 ssh_runner.go:195] Run: sudo crictl images --output json
I0729 12:42:40.473605 260472 crio.go:514] all images are preloaded for cri-o runtime.
I0729 12:42:40.473627 260472 crio.go:433] Images already preloaded, skipping extraction
I0729 12:42:40.473677 260472 ssh_runner.go:195] Run: sudo crictl images --output json
I0729 12:42:40.600040 260472 crio.go:514] all images are preloaded for cri-o runtime.
I0729 12:42:40.600073 260472 cache_images.go:84] Images are preloaded, skipping loading
I0729 12:42:40.600100 260472 kubeadm.go:934] updating node { 192.168.39.217 8443 v1.30.3 crio true true} ...
I0729 12:42:40.600218 260472 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.30.3/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=ha-767488 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.39.217
[Install]
config:
{KubernetesVersion:v1.30.3 ClusterName:ha-767488 Namespace:default APIServerHAVIP:192.168.39.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0729 12:42:40.600301 260472 ssh_runner.go:195] Run: crio config
I0729 12:42:40.713091 260472 cni.go:84] Creating CNI manager for ""
I0729 12:42:40.713114 260472 cni.go:136] multinode detected (4 nodes found), recommending kindnet
I0729 12:42:40.713124 260472 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0729 12:42:40.713150 260472 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.39.217 APIServerPort:8443 KubernetesVersion:v1.30.3 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:ha-767488 NodeName:ha-767488 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.39.217"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.39.217 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernete
s/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0729 12:42:40.713297 260472 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.39.217
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/crio/crio.sock
name: "ha-767488"
kubeletExtraArgs:
node-ip: 192.168.39.217
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.39.217"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.30.3
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/crio/crio.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0729 12:42:40.713315 260472 kube-vip.go:115] generating kube-vip config ...
I0729 12:42:40.713354 260472 ssh_runner.go:195] Run: sudo sh -c "modprobe --all ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack"
I0729 12:42:40.731149 260472 kube-vip.go:167] auto-enabling control-plane load-balancing in kube-vip
I0729 12:42:40.731283 260472 kube-vip.go:137] kube-vip config:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: kube-vip
namespace: kube-system
spec:
containers:
- args:
- manager
env:
- name: vip_arp
value: "true"
- name: port
value: "8443"
- name: vip_nodename
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: vip_interface
value: eth0
- name: vip_cidr
value: "32"
- name: dns_mode
value: first
- name: cp_enable
value: "true"
- name: cp_namespace
value: kube-system
- name: vip_leaderelection
value: "true"
- name: vip_leasename
value: plndr-cp-lock
- name: vip_leaseduration
value: "5"
- name: vip_renewdeadline
value: "3"
- name: vip_retryperiod
value: "1"
- name: address
value: 192.168.39.254
- name: prometheus_server
value: :2112
- name : lb_enable
value: "true"
- name: lb_port
value: "8443"
image: ghcr.io/kube-vip/kube-vip:v0.8.0
imagePullPolicy: IfNotPresent
name: kube-vip
resources: {}
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
volumeMounts:
- mountPath: /etc/kubernetes/admin.conf
name: kubeconfig
hostAliases:
- hostnames:
- kubernetes
ip: 127.0.0.1
hostNetwork: true
volumes:
- hostPath:
path: "/etc/kubernetes/admin.conf"
name: kubeconfig
status: {}
I0729 12:42:40.731354 260472 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.30.3
I0729 12:42:40.745678 260472 binaries.go:44] Found k8s binaries, skipping transfer
I0729 12:42:40.745771 260472 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube /etc/kubernetes/manifests
I0729 12:42:40.756067 260472 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (309 bytes)
I0729 12:42:40.779511 260472 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0729 12:42:40.802104 260472 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2153 bytes)
I0729 12:42:40.819400 260472 ssh_runner.go:362] scp memory --> /etc/kubernetes/manifests/kube-vip.yaml (1441 bytes)
I0729 12:42:40.835924 260472 ssh_runner.go:195] Run: grep 192.168.39.254 control-plane.minikube.internal$ /etc/hosts
I0729 12:42:40.840719 260472 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0729 12:42:40.986870 260472 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0729 12:42:41.001565 260472 certs.go:68] Setting up /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488 for IP: 192.168.39.217
I0729 12:42:41.001593 260472 certs.go:194] generating shared ca certs ...
I0729 12:42:41.001614 260472 certs.go:226] acquiring lock for ca certs: {Name:mk35442f684e402d7c9b18ad971a254e9ba86fe0 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0729 12:42:41.001819 260472 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19341-233093/.minikube/ca.key
I0729 12:42:41.001875 260472 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19341-233093/.minikube/proxy-client-ca.key
I0729 12:42:41.001890 260472 certs.go:256] generating profile certs ...
I0729 12:42:41.001972 260472 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/client.key
I0729 12:42:41.002032 260472 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/apiserver.key.310e5293
I0729 12:42:41.002065 260472 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/proxy-client.key
I0729 12:42:41.002076 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/ca.crt -> /var/lib/minikube/certs/ca.crt
I0729 12:42:41.002091 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/ca.key -> /var/lib/minikube/certs/ca.key
I0729 12:42:41.002113 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/proxy-client-ca.crt -> /var/lib/minikube/certs/proxy-client-ca.crt
I0729 12:42:41.002131 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/proxy-client-ca.key -> /var/lib/minikube/certs/proxy-client-ca.key
I0729 12:42:41.002148 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/apiserver.crt -> /var/lib/minikube/certs/apiserver.crt
I0729 12:42:41.002165 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/apiserver.key -> /var/lib/minikube/certs/apiserver.key
I0729 12:42:41.002182 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/proxy-client.crt -> /var/lib/minikube/certs/proxy-client.crt
I0729 12:42:41.002198 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/proxy-client.key -> /var/lib/minikube/certs/proxy-client.key
I0729 12:42:41.002263 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/240340.pem (1338 bytes)
W0729 12:42:41.002296 260472 certs.go:480] ignoring /home/jenkins/minikube-integration/19341-233093/.minikube/certs/240340_empty.pem, impossibly tiny 0 bytes
I0729 12:42:41.002305 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca-key.pem (1675 bytes)
I0729 12:42:41.002328 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/ca.pem (1078 bytes)
I0729 12:42:41.002348 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/cert.pem (1123 bytes)
I0729 12:42:41.002370 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/key.pem (1679 bytes)
I0729 12:42:41.002406 260472 certs.go:484] found cert: /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem (1708 bytes)
I0729 12:42:41.002434 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/certs/240340.pem -> /usr/share/ca-certificates/240340.pem
I0729 12:42:41.002446 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem -> /usr/share/ca-certificates/2403402.pem
I0729 12:42:41.002458 260472 vm_assets.go:164] NewFileAsset: /home/jenkins/minikube-integration/19341-233093/.minikube/ca.crt -> /usr/share/ca-certificates/minikubeCA.pem
I0729 12:42:41.003070 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0729 12:42:41.027259 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0729 12:42:41.050547 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0729 12:42:41.074374 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0729 12:42:41.097416 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1440 bytes)
I0729 12:42:41.120537 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0729 12:42:41.143944 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0729 12:42:41.166548 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/profiles/ha-767488/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0729 12:42:41.189375 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/certs/240340.pem --> /usr/share/ca-certificates/240340.pem (1338 bytes)
I0729 12:42:41.212392 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/files/etc/ssl/certs/2403402.pem --> /usr/share/ca-certificates/2403402.pem (1708 bytes)
I0729 12:42:41.235698 260472 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19341-233093/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0729 12:42:41.258918 260472 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0729 12:42:41.275147 260472 ssh_runner.go:195] Run: openssl version
I0729 12:42:41.281163 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0729 12:42:41.291624 260472 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0729 12:42:41.296196 260472 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jul 29 12:04 /usr/share/ca-certificates/minikubeCA.pem
I0729 12:42:41.296247 260472 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0729 12:42:41.301759 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0729 12:42:41.310741 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/240340.pem && ln -fs /usr/share/ca-certificates/240340.pem /etc/ssl/certs/240340.pem"
I0729 12:42:41.320986 260472 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/240340.pem
I0729 12:42:41.325289 260472 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jul 29 12:16 /usr/share/ca-certificates/240340.pem
I0729 12:42:41.325343 260472 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/240340.pem
I0729 12:42:41.331301 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/240340.pem /etc/ssl/certs/51391683.0"
I0729 12:42:41.341279 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2403402.pem && ln -fs /usr/share/ca-certificates/2403402.pem /etc/ssl/certs/2403402.pem"
I0729 12:42:41.351883 260472 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2403402.pem
I0729 12:42:41.355957 260472 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jul 29 12:16 /usr/share/ca-certificates/2403402.pem
I0729 12:42:41.356029 260472 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2403402.pem
I0729 12:42:41.361571 260472 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/2403402.pem /etc/ssl/certs/3ec20f2e.0"
I0729 12:42:41.370434 260472 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0729 12:42:41.374797 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0729 12:42:41.380122 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0729 12:42:41.385653 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0729 12:42:41.391013 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0729 12:42:41.396652 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0729 12:42:41.402042 260472 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0729 12:42:41.407437 260472 kubeadm.go:392] StartCluster: {Name:ha-767488 KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19319/minikube-v1.33.1-1721690939-19319-amd64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:2200 CPUs:2 DiskSize:20000 Driver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.3 Clust
erName:ha-767488 Namespace:default APIServerHAVIP:192.168.39.254 APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.39.217 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m02 IP:192.168.39.45 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m03 IP:192.168.39.210 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:true Worker:true} {Name:m04 IP:192.168.39.181 Port:0 KubernetesVersion:v1.30.3 ContainerRuntime:crio ControlPlane:false Worker:true}] Addons:map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:false default-storageclass:false efk:false freshp
od:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:false nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:false storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:true ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountG
ID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0729 12:42:41.407562 260472 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0729 12:42:41.407600 260472 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0729 12:42:41.448602 260472 cri.go:89] found id: "c2ecec373e7fc9b2df14a7ee038d73c5c0f8ef3e75270e347eb200f6abfb5000"
I0729 12:42:41.448629 260472 cri.go:89] found id: "6f541b63f34e8eeb46f9636fcd9f0442b732b33fe15a4bb1e996edfc3adf2fe8"
I0729 12:42:41.448633 260472 cri.go:89] found id: "18d760360355700e8a42faaba690a4a57ef58f437e0e7dbefd872256cc796a7a"
I0729 12:42:41.448637 260472 cri.go:89] found id: "66eeaa3de5dde1e9f2a918cb9fd9bc19adf3f3ed253d852c708c0ef05e28f69b"
I0729 12:42:41.448639 260472 cri.go:89] found id: "149dfcffe55a708779d440706d95050121ad76560bbaa46641838c344b217e7f"
I0729 12:42:41.448643 260472 cri.go:89] found id: "7ffae0e726786e23c9ba39593a6166909b333b1b7c99e601300f323d7ae2af85"
I0729 12:42:41.448645 260472 cri.go:89] found id: "d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b"
I0729 12:42:41.448647 260472 cri.go:89] found id: "88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770"
I0729 12:42:41.448650 260472 cri.go:89] found id: "45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722"
I0729 12:42:41.448655 260472 cri.go:89] found id: "76b855b3ad75bb209ec720b68db7bee9bfb69f8d1091919982820aab20439c00"
I0729 12:42:41.448657 260472 cri.go:89] found id: "a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad"
I0729 12:42:41.448660 260472 cri.go:89] found id: "5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887"
I0729 12:42:41.448662 260472 cri.go:89] found id: "5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf"
I0729 12:42:41.448665 260472 cri.go:89] found id: "c263b16acab21928710d48f8da4b2c33c4cf21fb27bd91c19afc83702ddea09d"
I0729 12:42:41.448671 260472 cri.go:89] found id: "ed92faf8d1c93020229e184b95ee1a802427dc8ac7d3670cd6503553d63e1ea0"
I0729 12:42:41.448673 260472 cri.go:89] found id: "e2114078a73c1867e40ef6bec0d5cdb7dae271d0467d72c5e1533e5f89ce4316"
I0729 12:42:41.448676 260472 cri.go:89] found id: "a99c50ffbfb2857e177c7f353942753fa63496d40986552ea6d4dd738053f5b1"
I0729 12:42:41.448680 260472 cri.go:89] found id: "f1ea8fbc1b3ff5827e936b241bd850fd33d0b66b1dc0fd42770eabd2b309b5bb"
I0729 12:42:41.448682 260472 cri.go:89] found id: "dab08a0e0f3c131c614ed8ec288a8bf50cd6b85a4433f41884c95cadbd27473a"
I0729 12:42:41.448685 260472 cri.go:89] found id: ""
I0729 12:42:41.448727 260472 ssh_runner.go:195] Run: sudo runc list -f json
==> CRI-O <==
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.307009799Z" level=debug msg="Response: &ImageFsInfoResponse{ImageFilesystems:[]*FilesystemUsage{&FilesystemUsage{Timestamp:1722257312306981756,FsId:&FilesystemIdentifier{Mountpoint:/var/lib/containers/storage/overlay-images,},UsedBytes:&UInt64Value{Value:154769,},InodesUsed:&UInt64Value{Value:72,},},},ContainerFilesystems:[]*FilesystemUsage{},}" file="otel-collector/interceptors.go:74" id=36c26d81-f99c-417b-8d7e-4956ddd70a4a name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.307627035Z" level=debug msg="Request: &ListContainersRequest{Filter:&ContainerFilter{Id:,State:nil,PodSandboxId:,LabelSelector:map[string]string{},},}" file="otel-collector/interceptors.go:62" id=81f084b2-780b-400e-b70d-9fa387f84af4 name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.307712475Z" level=debug msg="No filters were applied, returning full container list" file="server/container_list.go:60" id=81f084b2-780b-400e-b70d-9fa387f84af4 name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.308293845Z" level=debug msg="Response: &ListContainersResponse{Containers:[]*Container{&Container{Id:f525ef9d817220a42df6d260e402031bc0f40e9d8eccfa433f89cb8d789509a0,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:9,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_RUNNING,CreatedAt:1722257205679398693,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 9,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:e93850281207e87f03fc437f6794b568715decdd6504954ba8367283e92bcf15,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:4,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_RUNNING,CreatedAt:1722257157679385960,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 4,io.kubernetes.container.term
inationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:8,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_EXITED,CreatedAt:1722257106688290314,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 8,io.kubernetes.container
.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cf19aeac6987974197a0c4fc5ef354cf20bc2cfc5683873ace6945ca2a3d83c4,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:5,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_RUNNING,CreatedAt:1722257088688538659,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 5,io.kubernetes.container.termination
MessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:29fa3f76ed7ccda60c4c6e231a6887aa5229c2b25e6696966a476b9886d038b5,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:4,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_EXITED,CreatedAt:1722257007681963333,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 4,io.kubernetes.container.terminationMessagePath:
/dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:25b0c852c73cd36ba713615d693f2704d45118139c5677cd88d083912617daef,PodSandboxId:10b4f76c89c4ae4310d83c4946c797e37a19ad6264f2b8f3eae7f8c1f2d7c26e,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722257002105739336,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.c
ontainer.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:8e5267677fe3d34fad80faf3c7949935560d4b8794349dc0b074dc70c3429867,PodSandboxId:a2229e9d163bb77c610841ca6590b80077298eb172e4716a1dfd1479e0c25aa7,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722256998333917463,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: F
ile,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:3,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_EXITED,CreatedAt:1722256969229544253,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 3,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.
terminationGracePeriod: 30,},},&Container{Id:9d1de005960b44c240c3297855fd44976860ae0bb1f241b69ed4ebd6ad75874b,PodSandboxId:00984bd8001feb5ed9f77571c167e0803f7c15281c84a0804035a5d1d1ba10fd,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256969184377799,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]string{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol
\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b384618e5ae14c67777ad9c0294426c93963be6110cbbda099994dc515fd3e10,PodSandboxId:2c2519cb3cb91c93edfcbe22fc0f10c0060f2fedf0e9bbc4dee120d59ac19946,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:2,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_RUNNING,CreatedAt:1722256964992696502,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kubernetes.container.re
startCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cc3c94fe6246ab5e4e5ebc15e5df7177339e6e1803a5de62af9f05fcf468e4c0,PodSandboxId:33794e355298339573727419eb819362a6fbba43533c443606689368631c6ed7,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:2,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_RUNNING,CreatedAt:1722256964944547268,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a8364975298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 2,io.kubernetes.conta
iner.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b50ae6e8e38f62d2e29778940201dac4ae9487c76d83d6f2fc51b50ba9c57577,PodSandboxId:aab24bd3a9edf83f4494105a97d68b87240d7a362b49eeec83ef9e03042f16c9,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:2,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_RUNNING,CreatedAt:1722256964544785454,Labels:map[string]string{io.kubernetes.container.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/
termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2d5168de1ca60a1804768ba74dbfa0354d0345a9df24f0f768226a2be1bba584,PodSandboxId:b2a043288f89fcd77c823cdc1864f0ccad12b3e58b9073209c077a133e7d3c39,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256964678578114,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"contain
erPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:aa1dfc42a005d85caf3f6c775efc543ba30e572f1a2aef81da961336b90306ba,PodSandboxId:6be19c0f23e95c03a878e75b7eb44f4fef3b8e9aad64ae174f0a2abb9a5f4b9b,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:2,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_RUNNING,CreatedAt:1722256964588272188,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.kubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c
032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a311cff0c8eccdd5a09eaeacd7969122d9ddde39d014e13fa635c80a8c881828,PodSandboxId:b2a875cf8cfc199f2d4e139e07c304ad62106f9ac4b4993fa936cff56fd7c8f5,Metadata:&ContainerMetadata{Name:etcd,Attempt:2,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_RUNNING,CreatedAt:1722256964416223066,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubern
etes.container.hash: f2243a41,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:18d760360355700e8a42faaba690a4a57ef58f437e0e7dbefd872256cc796a7a,PodSandboxId:4ac1d50b066bb2eb7a34bb7d199b90348d91669b762f37cdc03526a6d0df2951,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:1,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_EXITED,CreatedAt:1722256691386408722,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kuberne
tes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:3f1e978a01d07fe5560351a9000228fcd6513bf1523fc69c552cfb9348a48738,PodSandboxId:6ff1b7f6ad7317cd752d49d26da2bdd58472e1bcf1b6e6326622f05f66bba0c2,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256251616439055,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 1,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cbbea78e99e7243a706c470f06d9c35c916c801b42b3d4292eaebc32b040bd88,PodSandboxId:a7dc5254878c7e9d75a9ddca7dba85c5cf5cdd5d4f268437ed042f869f7056a4,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256242073157926,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePa
th: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b,PodSandboxId:464e80f1474daa7b0db4ff0092f58e13b090fc858e0ff28f7a078ff04cdea981,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256219945374530,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\
"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770,PodSandboxId:4e921577c4923d7b6e0c696432d1abd949520609041ab98de7c322d2ccda0520,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:1,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_EXITED,CreatedAt:1722256216950640136,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a836497
5298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722,PodSandboxId:6fd6fea36e81ffa4290fe1a361be26054cf988c1ac45e7156ba5e6fbec0beb5f,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256216944029521,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]st
ring{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad,PodSandboxId:ebff2bebd5529e4a88cb12b43ccd259c745394eec3146bd27184e0b6b2ce8483,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:1,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_EXITED,CreatedAt:1722256208279122761,Labels:map[string]string{io.kubernetes.con
tainer.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887,PodSandboxId:4d030101f0f826244ce88da9b0694fb5a9203b3a51c102e6ee59de3ad816bb65,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:1,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_EXITED,CreatedAt:1722256208057598157,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.k
ubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf,PodSandboxId:c38a2d43be1535384115e15374717abe3880434bd6dbb72a6dcf2ae1a42119cd,Metadata:&ContainerMetadata{Name:etcd,Attempt:1,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_EXITED,CreatedAt:1722256207875875673,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kube
rnetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubernetes.container.hash: f2243a41,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},},}" file="otel-collector/interceptors.go:74" id=81f084b2-780b-400e-b70d-9fa387f84af4 name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.356327151Z" level=debug msg="Request: &VersionRequest{Version:,}" file="otel-collector/interceptors.go:62" id=734c4f67-d739-4ab6-adbd-f23cdf4a4311 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.356414996Z" level=debug msg="Response: &VersionResponse{Version:0.1.0,RuntimeName:cri-o,RuntimeVersion:1.29.1,RuntimeApiVersion:v1,}" file="otel-collector/interceptors.go:74" id=734c4f67-d739-4ab6-adbd-f23cdf4a4311 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.357621319Z" level=debug msg="Request: &ImageFsInfoRequest{}" file="otel-collector/interceptors.go:62" id=e3d3d6be-513c-42f3-b8b1-644fdae591de name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.358337433Z" level=debug msg="Response: &ImageFsInfoResponse{ImageFilesystems:[]*FilesystemUsage{&FilesystemUsage{Timestamp:1722257312358311598,FsId:&FilesystemIdentifier{Mountpoint:/var/lib/containers/storage/overlay-images,},UsedBytes:&UInt64Value{Value:154769,},InodesUsed:&UInt64Value{Value:72,},},},ContainerFilesystems:[]*FilesystemUsage{},}" file="otel-collector/interceptors.go:74" id=e3d3d6be-513c-42f3-b8b1-644fdae591de name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.359073265Z" level=debug msg="Request: &ListContainersRequest{Filter:&ContainerFilter{Id:,State:nil,PodSandboxId:,LabelSelector:map[string]string{},},}" file="otel-collector/interceptors.go:62" id=7d4d25b9-0618-463a-82d8-5400a05eecab name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.359146056Z" level=debug msg="No filters were applied, returning full container list" file="server/container_list.go:60" id=7d4d25b9-0618-463a-82d8-5400a05eecab name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.359597011Z" level=debug msg="Response: &ListContainersResponse{Containers:[]*Container{&Container{Id:f525ef9d817220a42df6d260e402031bc0f40e9d8eccfa433f89cb8d789509a0,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:9,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_RUNNING,CreatedAt:1722257205679398693,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 9,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:e93850281207e87f03fc437f6794b568715decdd6504954ba8367283e92bcf15,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:4,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_RUNNING,CreatedAt:1722257157679385960,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 4,io.kubernetes.container.term
inationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:8,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_EXITED,CreatedAt:1722257106688290314,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 8,io.kubernetes.container
.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cf19aeac6987974197a0c4fc5ef354cf20bc2cfc5683873ace6945ca2a3d83c4,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:5,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_RUNNING,CreatedAt:1722257088688538659,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 5,io.kubernetes.container.termination
MessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:29fa3f76ed7ccda60c4c6e231a6887aa5229c2b25e6696966a476b9886d038b5,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:4,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_EXITED,CreatedAt:1722257007681963333,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 4,io.kubernetes.container.terminationMessagePath:
/dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:25b0c852c73cd36ba713615d693f2704d45118139c5677cd88d083912617daef,PodSandboxId:10b4f76c89c4ae4310d83c4946c797e37a19ad6264f2b8f3eae7f8c1f2d7c26e,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722257002105739336,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.c
ontainer.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:8e5267677fe3d34fad80faf3c7949935560d4b8794349dc0b074dc70c3429867,PodSandboxId:a2229e9d163bb77c610841ca6590b80077298eb172e4716a1dfd1479e0c25aa7,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722256998333917463,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: F
ile,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:3,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_EXITED,CreatedAt:1722256969229544253,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 3,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.
terminationGracePeriod: 30,},},&Container{Id:9d1de005960b44c240c3297855fd44976860ae0bb1f241b69ed4ebd6ad75874b,PodSandboxId:00984bd8001feb5ed9f77571c167e0803f7c15281c84a0804035a5d1d1ba10fd,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256969184377799,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]string{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol
\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b384618e5ae14c67777ad9c0294426c93963be6110cbbda099994dc515fd3e10,PodSandboxId:2c2519cb3cb91c93edfcbe22fc0f10c0060f2fedf0e9bbc4dee120d59ac19946,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:2,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_RUNNING,CreatedAt:1722256964992696502,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kubernetes.container.re
startCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cc3c94fe6246ab5e4e5ebc15e5df7177339e6e1803a5de62af9f05fcf468e4c0,PodSandboxId:33794e355298339573727419eb819362a6fbba43533c443606689368631c6ed7,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:2,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_RUNNING,CreatedAt:1722256964944547268,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a8364975298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 2,io.kubernetes.conta
iner.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b50ae6e8e38f62d2e29778940201dac4ae9487c76d83d6f2fc51b50ba9c57577,PodSandboxId:aab24bd3a9edf83f4494105a97d68b87240d7a362b49eeec83ef9e03042f16c9,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:2,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_RUNNING,CreatedAt:1722256964544785454,Labels:map[string]string{io.kubernetes.container.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/
termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2d5168de1ca60a1804768ba74dbfa0354d0345a9df24f0f768226a2be1bba584,PodSandboxId:b2a043288f89fcd77c823cdc1864f0ccad12b3e58b9073209c077a133e7d3c39,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256964678578114,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"contain
erPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:aa1dfc42a005d85caf3f6c775efc543ba30e572f1a2aef81da961336b90306ba,PodSandboxId:6be19c0f23e95c03a878e75b7eb44f4fef3b8e9aad64ae174f0a2abb9a5f4b9b,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:2,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_RUNNING,CreatedAt:1722256964588272188,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.kubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c
032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a311cff0c8eccdd5a09eaeacd7969122d9ddde39d014e13fa635c80a8c881828,PodSandboxId:b2a875cf8cfc199f2d4e139e07c304ad62106f9ac4b4993fa936cff56fd7c8f5,Metadata:&ContainerMetadata{Name:etcd,Attempt:2,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_RUNNING,CreatedAt:1722256964416223066,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubern
etes.container.hash: f2243a41,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:18d760360355700e8a42faaba690a4a57ef58f437e0e7dbefd872256cc796a7a,PodSandboxId:4ac1d50b066bb2eb7a34bb7d199b90348d91669b762f37cdc03526a6d0df2951,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:1,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_EXITED,CreatedAt:1722256691386408722,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kuberne
tes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:3f1e978a01d07fe5560351a9000228fcd6513bf1523fc69c552cfb9348a48738,PodSandboxId:6ff1b7f6ad7317cd752d49d26da2bdd58472e1bcf1b6e6326622f05f66bba0c2,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256251616439055,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 1,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cbbea78e99e7243a706c470f06d9c35c916c801b42b3d4292eaebc32b040bd88,PodSandboxId:a7dc5254878c7e9d75a9ddca7dba85c5cf5cdd5d4f268437ed042f869f7056a4,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256242073157926,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePa
th: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b,PodSandboxId:464e80f1474daa7b0db4ff0092f58e13b090fc858e0ff28f7a078ff04cdea981,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256219945374530,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\
"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770,PodSandboxId:4e921577c4923d7b6e0c696432d1abd949520609041ab98de7c322d2ccda0520,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:1,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_EXITED,CreatedAt:1722256216950640136,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a836497
5298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722,PodSandboxId:6fd6fea36e81ffa4290fe1a361be26054cf988c1ac45e7156ba5e6fbec0beb5f,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256216944029521,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]st
ring{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad,PodSandboxId:ebff2bebd5529e4a88cb12b43ccd259c745394eec3146bd27184e0b6b2ce8483,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:1,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_EXITED,CreatedAt:1722256208279122761,Labels:map[string]string{io.kubernetes.con
tainer.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887,PodSandboxId:4d030101f0f826244ce88da9b0694fb5a9203b3a51c102e6ee59de3ad816bb65,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:1,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_EXITED,CreatedAt:1722256208057598157,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.k
ubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf,PodSandboxId:c38a2d43be1535384115e15374717abe3880434bd6dbb72a6dcf2ae1a42119cd,Metadata:&ContainerMetadata{Name:etcd,Attempt:1,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_EXITED,CreatedAt:1722256207875875673,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kube
rnetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubernetes.container.hash: f2243a41,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},},}" file="otel-collector/interceptors.go:74" id=7d4d25b9-0618-463a-82d8-5400a05eecab name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.411474084Z" level=debug msg="Request: &VersionRequest{Version:,}" file="otel-collector/interceptors.go:62" id=02bde2fc-f782-4a8e-8890-c1ed9a7748d7 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.411573825Z" level=debug msg="Response: &VersionResponse{Version:0.1.0,RuntimeName:cri-o,RuntimeVersion:1.29.1,RuntimeApiVersion:v1,}" file="otel-collector/interceptors.go:74" id=02bde2fc-f782-4a8e-8890-c1ed9a7748d7 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.412644473Z" level=debug msg="Request: &ImageFsInfoRequest{}" file="otel-collector/interceptors.go:62" id=592432de-5f66-418c-9ab0-f17034809ba1 name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.413459861Z" level=debug msg="Response: &ImageFsInfoResponse{ImageFilesystems:[]*FilesystemUsage{&FilesystemUsage{Timestamp:1722257312413434220,FsId:&FilesystemIdentifier{Mountpoint:/var/lib/containers/storage/overlay-images,},UsedBytes:&UInt64Value{Value:154769,},InodesUsed:&UInt64Value{Value:72,},},},ContainerFilesystems:[]*FilesystemUsage{},}" file="otel-collector/interceptors.go:74" id=592432de-5f66-418c-9ab0-f17034809ba1 name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.414002918Z" level=debug msg="Request: &ListContainersRequest{Filter:&ContainerFilter{Id:,State:nil,PodSandboxId:,LabelSelector:map[string]string{},},}" file="otel-collector/interceptors.go:62" id=ca54ea74-d3e1-44ef-a636-aa500cf80d5a name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.414076481Z" level=debug msg="No filters were applied, returning full container list" file="server/container_list.go:60" id=ca54ea74-d3e1-44ef-a636-aa500cf80d5a name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.414745573Z" level=debug msg="Response: &ListContainersResponse{Containers:[]*Container{&Container{Id:f525ef9d817220a42df6d260e402031bc0f40e9d8eccfa433f89cb8d789509a0,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:9,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_RUNNING,CreatedAt:1722257205679398693,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 9,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:e93850281207e87f03fc437f6794b568715decdd6504954ba8367283e92bcf15,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:4,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_RUNNING,CreatedAt:1722257157679385960,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 4,io.kubernetes.container.term
inationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:8,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_EXITED,CreatedAt:1722257106688290314,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 8,io.kubernetes.container
.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cf19aeac6987974197a0c4fc5ef354cf20bc2cfc5683873ace6945ca2a3d83c4,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:5,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_RUNNING,CreatedAt:1722257088688538659,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 5,io.kubernetes.container.termination
MessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:29fa3f76ed7ccda60c4c6e231a6887aa5229c2b25e6696966a476b9886d038b5,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:4,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_EXITED,CreatedAt:1722257007681963333,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 4,io.kubernetes.container.terminationMessagePath:
/dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:25b0c852c73cd36ba713615d693f2704d45118139c5677cd88d083912617daef,PodSandboxId:10b4f76c89c4ae4310d83c4946c797e37a19ad6264f2b8f3eae7f8c1f2d7c26e,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722257002105739336,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.c
ontainer.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:8e5267677fe3d34fad80faf3c7949935560d4b8794349dc0b074dc70c3429867,PodSandboxId:a2229e9d163bb77c610841ca6590b80077298eb172e4716a1dfd1479e0c25aa7,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722256998333917463,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: F
ile,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:3,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_EXITED,CreatedAt:1722256969229544253,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 3,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.
terminationGracePeriod: 30,},},&Container{Id:9d1de005960b44c240c3297855fd44976860ae0bb1f241b69ed4ebd6ad75874b,PodSandboxId:00984bd8001feb5ed9f77571c167e0803f7c15281c84a0804035a5d1d1ba10fd,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256969184377799,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]string{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol
\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b384618e5ae14c67777ad9c0294426c93963be6110cbbda099994dc515fd3e10,PodSandboxId:2c2519cb3cb91c93edfcbe22fc0f10c0060f2fedf0e9bbc4dee120d59ac19946,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:2,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_RUNNING,CreatedAt:1722256964992696502,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kubernetes.container.re
startCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cc3c94fe6246ab5e4e5ebc15e5df7177339e6e1803a5de62af9f05fcf468e4c0,PodSandboxId:33794e355298339573727419eb819362a6fbba43533c443606689368631c6ed7,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:2,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_RUNNING,CreatedAt:1722256964944547268,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a8364975298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 2,io.kubernetes.conta
iner.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b50ae6e8e38f62d2e29778940201dac4ae9487c76d83d6f2fc51b50ba9c57577,PodSandboxId:aab24bd3a9edf83f4494105a97d68b87240d7a362b49eeec83ef9e03042f16c9,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:2,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_RUNNING,CreatedAt:1722256964544785454,Labels:map[string]string{io.kubernetes.container.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/
termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2d5168de1ca60a1804768ba74dbfa0354d0345a9df24f0f768226a2be1bba584,PodSandboxId:b2a043288f89fcd77c823cdc1864f0ccad12b3e58b9073209c077a133e7d3c39,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256964678578114,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"contain
erPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:aa1dfc42a005d85caf3f6c775efc543ba30e572f1a2aef81da961336b90306ba,PodSandboxId:6be19c0f23e95c03a878e75b7eb44f4fef3b8e9aad64ae174f0a2abb9a5f4b9b,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:2,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_RUNNING,CreatedAt:1722256964588272188,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.kubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c
032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a311cff0c8eccdd5a09eaeacd7969122d9ddde39d014e13fa635c80a8c881828,PodSandboxId:b2a875cf8cfc199f2d4e139e07c304ad62106f9ac4b4993fa936cff56fd7c8f5,Metadata:&ContainerMetadata{Name:etcd,Attempt:2,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_RUNNING,CreatedAt:1722256964416223066,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubern
etes.container.hash: f2243a41,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:18d760360355700e8a42faaba690a4a57ef58f437e0e7dbefd872256cc796a7a,PodSandboxId:4ac1d50b066bb2eb7a34bb7d199b90348d91669b762f37cdc03526a6d0df2951,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:1,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_EXITED,CreatedAt:1722256691386408722,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kuberne
tes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:3f1e978a01d07fe5560351a9000228fcd6513bf1523fc69c552cfb9348a48738,PodSandboxId:6ff1b7f6ad7317cd752d49d26da2bdd58472e1bcf1b6e6326622f05f66bba0c2,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256251616439055,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 1,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cbbea78e99e7243a706c470f06d9c35c916c801b42b3d4292eaebc32b040bd88,PodSandboxId:a7dc5254878c7e9d75a9ddca7dba85c5cf5cdd5d4f268437ed042f869f7056a4,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256242073157926,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePa
th: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b,PodSandboxId:464e80f1474daa7b0db4ff0092f58e13b090fc858e0ff28f7a078ff04cdea981,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256219945374530,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\
"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770,PodSandboxId:4e921577c4923d7b6e0c696432d1abd949520609041ab98de7c322d2ccda0520,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:1,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_EXITED,CreatedAt:1722256216950640136,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a836497
5298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722,PodSandboxId:6fd6fea36e81ffa4290fe1a361be26054cf988c1ac45e7156ba5e6fbec0beb5f,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256216944029521,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]st
ring{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad,PodSandboxId:ebff2bebd5529e4a88cb12b43ccd259c745394eec3146bd27184e0b6b2ce8483,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:1,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_EXITED,CreatedAt:1722256208279122761,Labels:map[string]string{io.kubernetes.con
tainer.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887,PodSandboxId:4d030101f0f826244ce88da9b0694fb5a9203b3a51c102e6ee59de3ad816bb65,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:1,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_EXITED,CreatedAt:1722256208057598157,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.k
ubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf,PodSandboxId:c38a2d43be1535384115e15374717abe3880434bd6dbb72a6dcf2ae1a42119cd,Metadata:&ContainerMetadata{Name:etcd,Attempt:1,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_EXITED,CreatedAt:1722256207875875673,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kube
rnetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubernetes.container.hash: f2243a41,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},},}" file="otel-collector/interceptors.go:74" id=ca54ea74-d3e1-44ef-a636-aa500cf80d5a name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.461965975Z" level=debug msg="Request: &VersionRequest{Version:,}" file="otel-collector/interceptors.go:62" id=8b63b5e8-9b6b-43fa-bc2b-62f0a5aaa522 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.462059587Z" level=debug msg="Response: &VersionResponse{Version:0.1.0,RuntimeName:cri-o,RuntimeVersion:1.29.1,RuntimeApiVersion:v1,}" file="otel-collector/interceptors.go:74" id=8b63b5e8-9b6b-43fa-bc2b-62f0a5aaa522 name=/runtime.v1.RuntimeService/Version
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.463135764Z" level=debug msg="Request: &ImageFsInfoRequest{}" file="otel-collector/interceptors.go:62" id=56fe5fa6-b857-440d-8c71-d2122df265b3 name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.463628954Z" level=debug msg="Response: &ImageFsInfoResponse{ImageFilesystems:[]*FilesystemUsage{&FilesystemUsage{Timestamp:1722257312463600343,FsId:&FilesystemIdentifier{Mountpoint:/var/lib/containers/storage/overlay-images,},UsedBytes:&UInt64Value{Value:154769,},InodesUsed:&UInt64Value{Value:72,},},},ContainerFilesystems:[]*FilesystemUsage{},}" file="otel-collector/interceptors.go:74" id=56fe5fa6-b857-440d-8c71-d2122df265b3 name=/runtime.v1.ImageService/ImageFsInfo
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.464198763Z" level=debug msg="Request: &ListContainersRequest{Filter:&ContainerFilter{Id:,State:nil,PodSandboxId:,LabelSelector:map[string]string{},},}" file="otel-collector/interceptors.go:62" id=53f388d7-3fe8-4ba0-9fb5-59748d6deefd name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.464251032Z" level=debug msg="No filters were applied, returning full container list" file="server/container_list.go:60" id=53f388d7-3fe8-4ba0-9fb5-59748d6deefd name=/runtime.v1.RuntimeService/ListContainers
Jul 29 12:48:32 ha-767488 crio[6770]: time="2024-07-29 12:48:32.467026229Z" level=debug msg="Response: &ListContainersResponse{Containers:[]*Container{&Container{Id:f525ef9d817220a42df6d260e402031bc0f40e9d8eccfa433f89cb8d789509a0,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:9,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_RUNNING,CreatedAt:1722257205679398693,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 9,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:e93850281207e87f03fc437f6794b568715decdd6504954ba8367283e92bcf15,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:4,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_RUNNING,CreatedAt:1722257157679385960,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 4,io.kubernetes.container.term
inationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b,PodSandboxId:309c197fc5d3026fe5887ac8f85b4a3208f67502563a5aa976e8c6ad402a3454,Metadata:&ContainerMetadata{Name:kube-controller-manager,Attempt:8,},Image:&ImageSpec{Image:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e,State:CONTAINER_EXITED,CreatedAt:1722257106688290314,Labels:map[string]string{io.kubernetes.container.name: kube-controller-manager,io.kubernetes.pod.name: kube-controller-manager-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 6429dcd204de47eb64e9eb4c7981c7df,},Annotations:map[string]string{io.kubernetes.container.hash: bcb4918f,io.kubernetes.container.restartCount: 8,io.kubernetes.container
.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cf19aeac6987974197a0c4fc5ef354cf20bc2cfc5683873ace6945ca2a3d83c4,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:5,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_RUNNING,CreatedAt:1722257088688538659,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 5,io.kubernetes.container.termination
MessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:29fa3f76ed7ccda60c4c6e231a6887aa5229c2b25e6696966a476b9886d038b5,PodSandboxId:69a46f7b3f55baa72fdf5b1ae2cb64f450b9216cbcd0a3edd6803b0c48b41e2f,Metadata:&ContainerMetadata{Name:storage-provisioner,Attempt:4,},Image:&ImageSpec{Image:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562,State:CONTAINER_EXITED,CreatedAt:1722257007681963333,Labels:map[string]string{io.kubernetes.container.name: storage-provisioner,io.kubernetes.pod.name: storage-provisioner,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: baafb1c5-8785-44de-ba07-d858ba337fce,},Annotations:map[string]string{io.kubernetes.container.hash: 652d618a,io.kubernetes.container.restartCount: 4,io.kubernetes.container.terminationMessagePath:
/dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:25b0c852c73cd36ba713615d693f2704d45118139c5677cd88d083912617daef,PodSandboxId:10b4f76c89c4ae4310d83c4946c797e37a19ad6264f2b8f3eae7f8c1f2d7c26e,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722257002105739336,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.c
ontainer.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:8e5267677fe3d34fad80faf3c7949935560d4b8794349dc0b074dc70c3429867,PodSandboxId:a2229e9d163bb77c610841ca6590b80077298eb172e4716a1dfd1479e0c25aa7,Metadata:&ContainerMetadata{Name:busybox,Attempt:2,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_RUNNING,CreatedAt:1722256998333917463,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: F
ile,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf,PodSandboxId:8ed4d1a8b9e4974abdc5251351fb8c09c047e38bb1b36fb5216fc18bdbc8d157,Metadata:&ContainerMetadata{Name:kube-apiserver,Attempt:3,},Image:&ImageSpec{Image:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d,State:CONTAINER_EXITED,CreatedAt:1722256969229544253,Labels:map[string]string{io.kubernetes.container.name: kube-apiserver,io.kubernetes.pod.name: kube-apiserver-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: b1d029e38f53e06a3c7b5c185fd64a06,},Annotations:map[string]string{io.kubernetes.container.hash: 3e08f0ef,io.kubernetes.container.restartCount: 3,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.
terminationGracePeriod: 30,},},&Container{Id:9d1de005960b44c240c3297855fd44976860ae0bb1f241b69ed4ebd6ad75874b,PodSandboxId:00984bd8001feb5ed9f77571c167e0803f7c15281c84a0804035a5d1d1ba10fd,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256969184377799,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]string{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol
\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b384618e5ae14c67777ad9c0294426c93963be6110cbbda099994dc515fd3e10,PodSandboxId:2c2519cb3cb91c93edfcbe22fc0f10c0060f2fedf0e9bbc4dee120d59ac19946,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:2,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_RUNNING,CreatedAt:1722256964992696502,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kubernetes.container.re
startCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cc3c94fe6246ab5e4e5ebc15e5df7177339e6e1803a5de62af9f05fcf468e4c0,PodSandboxId:33794e355298339573727419eb819362a6fbba43533c443606689368631c6ed7,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:2,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_RUNNING,CreatedAt:1722256964944547268,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a8364975298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 2,io.kubernetes.conta
iner.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:b50ae6e8e38f62d2e29778940201dac4ae9487c76d83d6f2fc51b50ba9c57577,PodSandboxId:aab24bd3a9edf83f4494105a97d68b87240d7a362b49eeec83ef9e03042f16c9,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:2,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_RUNNING,CreatedAt:1722256964544785454,Labels:map[string]string{io.kubernetes.container.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/
termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:2d5168de1ca60a1804768ba74dbfa0354d0345a9df24f0f768226a2be1bba584,PodSandboxId:b2a043288f89fcd77c823cdc1864f0ccad12b3e58b9073209c077a133e7d3c39,Metadata:&ContainerMetadata{Name:coredns,Attempt:2,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_RUNNING,CreatedAt:1722256964678578114,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"contain
erPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:aa1dfc42a005d85caf3f6c775efc543ba30e572f1a2aef81da961336b90306ba,PodSandboxId:6be19c0f23e95c03a878e75b7eb44f4fef3b8e9aad64ae174f0a2abb9a5f4b9b,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:2,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_RUNNING,CreatedAt:1722256964588272188,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.kubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c
032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a311cff0c8eccdd5a09eaeacd7969122d9ddde39d014e13fa635c80a8c881828,PodSandboxId:b2a875cf8cfc199f2d4e139e07c304ad62106f9ac4b4993fa936cff56fd7c8f5,Metadata:&ContainerMetadata{Name:etcd,Attempt:2,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_RUNNING,CreatedAt:1722256964416223066,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubern
etes.container.hash: f2243a41,io.kubernetes.container.restartCount: 2,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:18d760360355700e8a42faaba690a4a57ef58f437e0e7dbefd872256cc796a7a,PodSandboxId:4ac1d50b066bb2eb7a34bb7d199b90348d91669b762f37cdc03526a6d0df2951,Metadata:&ContainerMetadata{Name:kube-vip,Attempt:1,},Image:&ImageSpec{Image:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12,State:CONTAINER_EXITED,CreatedAt:1722256691386408722,Labels:map[string]string{io.kubernetes.container.name: kube-vip,io.kubernetes.pod.name: kube-vip-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: db6837dfa9a0fa8b28ce8897488c95e3,},Annotations:map[string]string{io.kubernetes.container.hash: 3d55e6dc,io.kuberne
tes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:3f1e978a01d07fe5560351a9000228fcd6513bf1523fc69c552cfb9348a48738,PodSandboxId:6ff1b7f6ad7317cd752d49d26da2bdd58472e1bcf1b6e6326622f05f66bba0c2,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256251616439055,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-4ppv4,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: be02c819-68d0-4158-94c8-f6211d18670a,},Annotations:map[string]string{io.kubernetes.container.hash: ff023d68,io.kubernetes.container.restartCount: 1,io.kube
rnetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:cbbea78e99e7243a706c470f06d9c35c916c801b42b3d4292eaebc32b040bd88,PodSandboxId:a7dc5254878c7e9d75a9ddca7dba85c5cf5cdd5d4f268437ed042f869f7056a4,Metadata:&ContainerMetadata{Name:busybox,Attempt:1,},Image:&ImageSpec{Image:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a,State:CONTAINER_EXITED,CreatedAt:1722256242073157926,Labels:map[string]string{io.kubernetes.container.name: busybox,io.kubernetes.pod.name: busybox-fc5497c4f-trgfp,io.kubernetes.pod.namespace: default,io.kubernetes.pod.uid: 7969b6b0-a51a-4242-9ecd-1c2f60c5904b,},Annotations:map[string]string{io.kubernetes.container.hash: e7252d35,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePa
th: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b,PodSandboxId:464e80f1474daa7b0db4ff0092f58e13b090fc858e0ff28f7a078ff04cdea981,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256219945374530,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-k6r5l,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: f3b8cd5e-1836-4118-847f-888cd3aa6dd7,},Annotations:map[string]string{io.kubernetes.container.hash: 7b83be43,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\
"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770,PodSandboxId:4e921577c4923d7b6e0c696432d1abd949520609041ab98de7c322d2ccda0520,Metadata:&ContainerMetadata{Name:kube-proxy,Attempt:1,},Image:&ImageSpec{Image:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1,State:CONTAINER_EXITED,CreatedAt:1722256216950640136,Labels:map[string]string{io.kubernetes.container.name: kube-proxy,io.kubernetes.pod.name: kube-proxy-sqk96,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 0730198e-117f-40fe-8b70-8a836497
5298,},Annotations:map[string]string{io.kubernetes.container.hash: e92dcc64,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722,PodSandboxId:6fd6fea36e81ffa4290fe1a361be26054cf988c1ac45e7156ba5e6fbec0beb5f,Metadata:&ContainerMetadata{Name:coredns,Attempt:1,},Image:&ImageSpec{Image:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4,State:CONTAINER_EXITED,CreatedAt:1722256216944029521,Labels:map[string]string{io.kubernetes.container.name: coredns,io.kubernetes.pod.name: coredns-7db6d8ff4d-qqt5t,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 21c0e561-16ad-4e7c-9d2b-9fd551fb2d73,},Annotations:map[string]st
ring{io.kubernetes.container.hash: 6e12a500,io.kubernetes.container.ports: [{\"name\":\"dns\",\"containerPort\":53,\"protocol\":\"UDP\"},{\"name\":\"dns-tcp\",\"containerPort\":53,\"protocol\":\"TCP\"},{\"name\":\"metrics\",\"containerPort\":9153,\"protocol\":\"TCP\"}],io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad,PodSandboxId:ebff2bebd5529e4a88cb12b43ccd259c745394eec3146bd27184e0b6b2ce8483,Metadata:&ContainerMetadata{Name:kindnet-cni,Attempt:1,},Image:&ImageSpec{Image:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46,State:CONTAINER_EXITED,CreatedAt:1722256208279122761,Labels:map[string]string{io.kubernetes.con
tainer.name: kindnet-cni,io.kubernetes.pod.name: kindnet-6x56p,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 71b2a0c2-a003-42d6-b606-54d777bc10ee,},Annotations:map[string]string{io.kubernetes.container.hash: fe443934,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887,PodSandboxId:4d030101f0f826244ce88da9b0694fb5a9203b3a51c102e6ee59de3ad816bb65,Metadata:&ContainerMetadata{Name:kube-scheduler,Attempt:1,},Image:&ImageSpec{Image:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2,State:CONTAINER_EXITED,CreatedAt:1722256208057598157,Labels:map[string]string{io.kubernetes.container.name: kube-scheduler,io.k
ubernetes.pod.name: kube-scheduler-ha-767488,io.kubernetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 10b79fa17a00d66843eca1c032b6c3a0,},Annotations:map[string]string{io.kubernetes.container.hash: 7337c8d9,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},&Container{Id:5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf,PodSandboxId:c38a2d43be1535384115e15374717abe3880434bd6dbb72a6dcf2ae1a42119cd,Metadata:&ContainerMetadata{Name:etcd,Attempt:1,},Image:&ImageSpec{Image:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,Annotations:map[string]string{},UserSpecifiedImage:,RuntimeHandler:,},ImageRef:3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899,State:CONTAINER_EXITED,CreatedAt:1722256207875875673,Labels:map[string]string{io.kubernetes.container.name: etcd,io.kubernetes.pod.name: etcd-ha-767488,io.kube
rnetes.pod.namespace: kube-system,io.kubernetes.pod.uid: 98b1224b4c3f88a8bf50c36863b9c250,},Annotations:map[string]string{io.kubernetes.container.hash: f2243a41,io.kubernetes.container.restartCount: 1,io.kubernetes.container.terminationMessagePath: /dev/termination-log,io.kubernetes.container.terminationMessagePolicy: File,io.kubernetes.pod.terminationGracePeriod: 30,},},},}" file="otel-collector/interceptors.go:74" id=53f388d7-3fe8-4ba0-9fb5-59748d6deefd name=/runtime.v1.RuntimeService/ListContainers
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
f525ef9d81722 76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e About a minute ago Running kube-controller-manager 9 309c197fc5d30 kube-controller-manager-ha-767488
e93850281207e 1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d 2 minutes ago Running kube-apiserver 4 8ed4d1a8b9e49 kube-apiserver-ha-767488
2276a6710daab 76932a3b37d7eb138c8f47c9a2b4218f0466dd273badf856f2ce2f0277e15b5e 3 minutes ago Exited kube-controller-manager 8 309c197fc5d30 kube-controller-manager-ha-767488
cf19aeac69879 6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562 3 minutes ago Running storage-provisioner 5 69a46f7b3f55b storage-provisioner
29fa3f76ed7cc 6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562 5 minutes ago Exited storage-provisioner 4 69a46f7b3f55b storage-provisioner
25b0c852c73cd 8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a 5 minutes ago Running busybox 2 10b4f76c89c4a busybox-fc5497c4f-trgfp
8e5267677fe3d 8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a 5 minutes ago Running busybox 2 a2229e9d163bb busybox-fc5497c4f-4ppv4
76489ee06a477 1f6d574d502f3b61c851b1bbd4ef2a964ce4c70071dd8da556f2d490d36b095d 5 minutes ago Exited kube-apiserver 3 8ed4d1a8b9e49 kube-apiserver-ha-767488
9d1de005960b4 cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4 5 minutes ago Running coredns 2 00984bd8001fe coredns-7db6d8ff4d-qqt5t
b384618e5ae14 38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12 5 minutes ago Running kube-vip 2 2c2519cb3cb91 kube-vip-ha-767488
cc3c94fe6246a 55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1 5 minutes ago Running kube-proxy 2 33794e3552983 kube-proxy-sqk96
2d5168de1ca60 cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4 5 minutes ago Running coredns 2 b2a043288f89f coredns-7db6d8ff4d-k6r5l
aa1dfc42a005d 3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2 5 minutes ago Running kube-scheduler 2 6be19c0f23e95 kube-scheduler-ha-767488
b50ae6e8e38f6 6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46 5 minutes ago Running kindnet-cni 2 aab24bd3a9edf kindnet-6x56p
a311cff0c8ecc 3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899 5 minutes ago Running etcd 2 b2a875cf8cfc1 etcd-ha-767488
18d7603603557 38af8ddebf499adc4631fe68b0ee224ffd6d7dd6b4aeeb393aff3d33cb94eb12 10 minutes ago Exited kube-vip 1 4ac1d50b066bb kube-vip-ha-767488
3f1e978a01d07 8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a 17 minutes ago Exited busybox 1 6ff1b7f6ad731 busybox-fc5497c4f-4ppv4
cbbea78e99e72 8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a 17 minutes ago Exited busybox 1 a7dc5254878c7 busybox-fc5497c4f-trgfp
d899a73918641 cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4 18 minutes ago Exited coredns 1 464e80f1474da coredns-7db6d8ff4d-k6r5l
88ec5aa0ed7ec 55bb025d2cfa592b9381d01e122e72a1ed4b29ca32f86b7d289d99da794784d1 18 minutes ago Exited kube-proxy 1 4e921577c4923 kube-proxy-sqk96
45379775c471b cbb01a7bd410dc08ba382018ab909a674fb0e48687f0c00797ed5bc34fcc6bb4 18 minutes ago Exited coredns 1 6fd6fea36e81f coredns-7db6d8ff4d-qqt5t
a327747c60c54 6f1d07c71fa0f426df75b802ec3f53ac5c1ae4110f67ab6d9a760083cc1d0f46 18 minutes ago Exited kindnet-cni 1 ebff2bebd5529 kindnet-6x56p
5e886bb5a4a2e 3edc18e7b76722eb2eb37a0858c09caacbd422d6e0cae4c2e5ce67bc9a9795e2 18 minutes ago Exited kube-scheduler 1 4d030101f0f82 kube-scheduler-ha-767488
5c8cded716df9 3861cfcd7c04ccac1f062788eca39487248527ef0c0cfd477a83d7691a75a899 18 minutes ago Exited etcd 1 c38a2d43be153 etcd-ha-767488
==> coredns [2d5168de1ca60a1804768ba74dbfa0354d0345a9df24f0f768226a2be1bba584] <==
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": net/http: TLS handshake timeout
[INFO] plugin/kubernetes: Trace[283503875]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:42:53.754) (total time: 10001ms):
Trace[283503875]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": net/http: TLS handshake timeout 10001ms (12:43:03.755)
Trace[283503875]: [10.001379228s] [10.001379228s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": net/http: TLS handshake timeout
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> coredns [45379775c471bd4a9ef1739d974345e80f77081cba400e4da29837d8833d8722] <==
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Unauthorized
[INFO] plugin/kubernetes: Trace[841416442]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:15.890) (total time: 11819ms):
Trace[841416442]: ---"Objects listed" error:Unauthorized 11819ms (12:40:27.709)
Trace[841416442]: [11.819152896s] [11.819152896s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Unauthorized
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Unauthorized
[INFO] plugin/kubernetes: Trace[2022085669]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:15.047) (total time: 12661ms):
Trace[2022085669]: ---"Objects listed" error:Unauthorized 12661ms (12:40:27.709)
Trace[2022085669]: [12.66151731s] [12.66151731s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Unauthorized
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?resourceVersion=3010": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: Trace[1130676405]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:32.086) (total time: 10721ms):
Trace[1130676405]: ---"Objects listed" error:Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?resourceVersion=3010": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF 10720ms (12:40:42.807)
Trace[1130676405]: [10.721021558s] [10.721021558s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?resourceVersion=3010": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
==> coredns [9d1de005960b44c240c3297855fd44976860ae0bb1f241b69ed4ebd6ad75874b] <==
Trace[394769481]: [10.001110422s] [10.001110422s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": net/http: TLS handshake timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53396->10.96.0.1:443: read: connection reset by peer
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53396->10.96.0.1:443: read: connection reset by peer
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53384->10.96.0.1:443: read: connection reset by peer
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53384->10.96.0.1:443: read: connection reset by peer
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53380->10.96.0.1:443: read: connection reset by peer
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: read tcp 10.244.0.10:53380->10.96.0.1:443: read: connection reset by peer
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> coredns [d899a73918641072da6b9eb108383a2b7685b434530cc0a8f7e8a2e87f11493b] <==
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Unauthorized
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Unauthorized
[INFO] plugin/kubernetes: Trace[1030282606]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:15.356) (total time: 12346ms):
Trace[1030282606]: ---"Objects listed" error:Unauthorized 12346ms (12:40:27.702)
Trace[1030282606]: [12.346347085s] [12.346347085s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Unauthorized
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Unauthorized
[INFO] plugin/kubernetes: Trace[418228940]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:15.563) (total time: 12139ms):
Trace[418228940]: ---"Objects listed" error:Unauthorized 12138ms (12:40:27.702)
Trace[418228940]: [12.139191986s] [12.139191986s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Unauthorized
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3039": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: Trace[2011977158]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:31.350) (total time: 11455ms):
Trace[2011977158]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3039": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF 11455ms (12:40:42.805)
Trace[2011977158]: [11.45543795s] [11.45543795s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?resourceVersion=3039": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?resourceVersion=3048": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://10.96.0.1:443/apis/discovery.k8s.io/v1/endpointslices?resourceVersion=3048": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] plugin/kubernetes: Trace[856661345]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231 (29-Jul-2024 12:40:31.528) (total time: 11278ms):
Trace[856661345]: ---"Objects listed" error:Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF 11278ms (12:40:42.807)
Trace[856661345]: [11.278535864s] [11.278535864s] END
[ERROR] plugin/kubernetes: pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?resourceVersion=3042": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
==> describe nodes <==
Name: ha-767488
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-767488
kubernetes.io/os=linux
minikube.k8s.io/commit=b867516af467da0393bcbe7e6497c888199628ff
minikube.k8s.io/name=ha-767488
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_07_29T12_21_07_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 29 Jul 2024 12:21:03 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-767488
AcquireTime: <unset>
RenewTime: Mon, 29 Jul 2024 12:48:25 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 29 Jul 2024 12:43:36 +0000 Mon, 29 Jul 2024 12:21:03 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 29 Jul 2024 12:43:36 +0000 Mon, 29 Jul 2024 12:21:03 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 29 Jul 2024 12:43:36 +0000 Mon, 29 Jul 2024 12:21:03 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 29 Jul 2024 12:43:36 +0000 Mon, 29 Jul 2024 12:21:32 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.217
Hostname: ha-767488
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
System Info:
Machine ID: 4910accb98434efca56ff8b39068800c
System UUID: 4910accb-9843-4efc-a56f-f8b39068800c
Boot ID: f538ab8c-89b7-40ce-b82e-7644a867ee15
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: cri-o://1.29.1
Kubelet Version: v1.30.3
Kube-Proxy Version: v1.30.3
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-fc5497c4f-4ppv4 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
default busybox-fc5497c4f-trgfp 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system coredns-7db6d8ff4d-k6r5l 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (3%!)(MISSING) 170Mi (8%!)(MISSING) 27m
kube-system coredns-7db6d8ff4d-qqt5t 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (3%!)(MISSING) 170Mi (8%!)(MISSING) 27m
kube-system etcd-ha-767488 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (4%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kindnet-6x56p 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 27m
kube-system kube-apiserver-ha-767488 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kube-controller-manager-ha-767488 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kube-proxy-sqk96 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kube-scheduler-ha-767488 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
kube-system kube-vip-ha-767488 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 15m
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 27m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 290Mi (13%!)(MISSING) 390Mi (18%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 5m6s kube-proxy
Normal Starting 17m kube-proxy
Normal Starting 27m kube-proxy
Normal Starting 27m kubelet Starting kubelet.
Normal NodeAllocatableEnforced 27m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientPID 27m (x7 over 27m) kubelet Node ha-767488 status is now: NodeHasSufficientPID
Normal NodeHasNoDiskPressure 27m (x8 over 27m) kubelet Node ha-767488 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientMemory 27m (x8 over 27m) kubelet Node ha-767488 status is now: NodeHasSufficientMemory
Normal NodeHasSufficientMemory 27m kubelet Node ha-767488 status is now: NodeHasSufficientMemory
Normal NodeHasSufficientPID 27m kubelet Node ha-767488 status is now: NodeHasSufficientPID
Normal Starting 27m kubelet Starting kubelet.
Normal NodeAllocatableEnforced 27m kubelet Updated Node Allocatable limit across pods
Normal NodeHasNoDiskPressure 27m kubelet Node ha-767488 status is now: NodeHasNoDiskPressure
Normal RegisteredNode 27m node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal NodeReady 27m kubelet Node ha-767488 status is now: NodeReady
Normal RegisteredNode 25m node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal RegisteredNode 24m node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal RegisteredNode 22m node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal RegisteredNode 16m node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Warning ContainerGCFailed 6m26s (x4 over 19m) kubelet rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /var/run/crio/crio.sock: connect: no such file or directory"
Normal RegisteredNode 4m55s node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal RegisteredNode 95s node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Normal RegisteredNode 31s node-controller Node ha-767488 event: Registered Node ha-767488 in Controller
Name: ha-767488-m02
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-767488-m02
kubernetes.io/os=linux
minikube.k8s.io/commit=b867516af467da0393bcbe7e6497c888199628ff
minikube.k8s.io/name=ha-767488
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_07_29T12_22_28_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 29 Jul 2024 12:22:26 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-767488-m02
AcquireTime: <unset>
RenewTime: Mon, 29 Jul 2024 12:48:29 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 29 Jul 2024 12:44:13 +0000 Mon, 29 Jul 2024 12:22:26 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 29 Jul 2024 12:44:13 +0000 Mon, 29 Jul 2024 12:22:26 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 29 Jul 2024 12:44:13 +0000 Mon, 29 Jul 2024 12:22:26 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 29 Jul 2024 12:44:13 +0000 Mon, 29 Jul 2024 12:22:49 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.45
Hostname: ha-767488-m02
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
System Info:
Machine ID: d9a3fe2d6456464f8574d4c1d95e4f21
System UUID: d9a3fe2d-6456-464f-8574-d4c1d95e4f21
Boot ID: 5cef9760-b094-4a5a-943c-bf1eb8a249d0
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: cri-o://1.29.1
Kubelet Version: v1.30.3
Kube-Proxy Version: v1.30.3
PodCIDR: 10.244.1.0/24
PodCIDRs: 10.244.1.0/24
Non-terminated Pods: (8 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox-fc5497c4f-jjx77 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 11m
kube-system etcd-ha-767488-m02 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (4%!)(MISSING) 0 (0%!)(MISSING) 26m
kube-system kindnet-l7jpd 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 26m
kube-system kube-apiserver-ha-767488-m02 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 26m
kube-system kube-controller-manager-ha-767488-m02 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 26m
kube-system kube-proxy-d9lg8 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 26m
kube-system kube-scheduler-ha-767488-m02 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 26m
kube-system kube-vip-ha-767488-m02 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 25m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 750m (37%!)(MISSING) 100m (5%!)(MISSING)
memory 150Mi (7%!)(MISSING) 50Mi (2%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 4m38s kube-proxy
Normal Starting 16m kube-proxy
Normal Starting 22m kube-proxy
Normal Starting 26m kube-proxy
Normal NodeAllocatableEnforced 26m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 26m (x8 over 26m) kubelet Node ha-767488-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 26m (x8 over 26m) kubelet Node ha-767488-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 26m (x7 over 26m) kubelet Node ha-767488-m02 status is now: NodeHasSufficientPID
Normal RegisteredNode 26m node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal RegisteredNode 25m node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal RegisteredNode 24m node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal NodeAllocatableEnforced 22m kubelet Updated Node Allocatable limit across pods
Warning Rebooted 22m kubelet Node ha-767488-m02 has been rebooted, boot id: 9ab58707-555a-4bb6-83c9-2399f8c434d4
Normal Starting 22m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 22m kubelet Node ha-767488-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 22m kubelet Node ha-767488-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 22m kubelet Node ha-767488-m02 status is now: NodeHasSufficientPID
Normal RegisteredNode 22m node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Warning ContainerGCFailed 17m kubelet rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /var/run/crio/crio.sock: connect: no such file or directory"
Normal RegisteredNode 16m node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal Starting 5m27s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m27s (x8 over 5m27s) kubelet Node ha-767488-m02 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m27s (x8 over 5m27s) kubelet Node ha-767488-m02 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m27s (x7 over 5m27s) kubelet Node ha-767488-m02 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m27s kubelet Updated Node Allocatable limit across pods
Normal RegisteredNode 4m55s node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal RegisteredNode 95s node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Normal RegisteredNode 31s node-controller Node ha-767488-m02 event: Registered Node ha-767488-m02 in Controller
Name: ha-767488-m03
Roles: control-plane
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-767488-m03
kubernetes.io/os=linux
minikube.k8s.io/commit=b867516af467da0393bcbe7e6497c888199628ff
minikube.k8s.io/name=ha-767488
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_07_29T12_23_45_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 29 Jul 2024 12:23:42 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-767488-m03
AcquireTime: <unset>
RenewTime: Mon, 29 Jul 2024 12:48:29 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 29 Jul 2024 12:47:58 +0000 Mon, 29 Jul 2024 12:47:28 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 29 Jul 2024 12:47:58 +0000 Mon, 29 Jul 2024 12:47:28 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 29 Jul 2024 12:47:58 +0000 Mon, 29 Jul 2024 12:47:28 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 29 Jul 2024 12:47:58 +0000 Mon, 29 Jul 2024 12:47:28 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.210
Hostname: ha-767488-m03
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
System Info:
Machine ID: 5ca168b2de41451a82ff59b787c535ad
System UUID: 5ca168b2-de41-451a-82ff-59b787c535ad
Boot ID: f8572197-e522-4d4c-92d1-3c0e30179060
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: cri-o://1.29.1
Kubelet Version: v1.30.3
Kube-Proxy Version: v1.30.3
PodCIDR: 10.244.2.0/24
PodCIDRs: 10.244.2.0/24
Non-terminated Pods: (7 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system etcd-ha-767488-m03 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (4%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system kindnet-bz9pp 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 24m
kube-system kube-apiserver-ha-767488-m03 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system kube-controller-manager-ha-767488-m03 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system kube-proxy-tzj27 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system kube-scheduler-ha-767488-m03 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
kube-system kube-vip-ha-767488-m03 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 24m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 750m (37%!)(MISSING) 100m (5%!)(MISSING)
memory 150Mi (7%!)(MISSING) 50Mi (2%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 46s kube-proxy
Normal Starting 24m kube-proxy
Normal NodeHasSufficientPID 24m (x7 over 24m) kubelet Node ha-767488-m03 status is now: NodeHasSufficientPID
Normal RegisteredNode 24m node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal NodeAllocatableEnforced 24m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 24m (x8 over 24m) kubelet Node ha-767488-m03 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 24m (x8 over 24m) kubelet Node ha-767488-m03 status is now: NodeHasNoDiskPressure
Normal RegisteredNode 24m node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal RegisteredNode 24m node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal RegisteredNode 22m node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal NodeNotReady 21m node-controller Node ha-767488-m03 status is now: NodeNotReady
Normal RegisteredNode 16m node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal RegisteredNode 4m56s node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal RegisteredNode 96s node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Normal Starting 66s kubelet Starting kubelet.
Normal NodeAllocatableEnforced 66s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 65s (x2 over 66s) kubelet Node ha-767488-m03 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 65s (x2 over 66s) kubelet Node ha-767488-m03 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 65s (x2 over 66s) kubelet Node ha-767488-m03 status is now: NodeHasSufficientPID
Warning Rebooted 65s kubelet Node ha-767488-m03 has been rebooted, boot id: f8572197-e522-4d4c-92d1-3c0e30179060
Normal NodeReady 65s kubelet Node ha-767488-m03 status is now: NodeReady
Normal RegisteredNode 32s node-controller Node ha-767488-m03 event: Registered Node ha-767488-m03 in Controller
Name: ha-767488-m04
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ha-767488-m04
kubernetes.io/os=linux
minikube.k8s.io/commit=b867516af467da0393bcbe7e6497c888199628ff
minikube.k8s.io/name=ha-767488
minikube.k8s.io/primary=false
minikube.k8s.io/updated_at=2024_07_29T12_24_51_0700
minikube.k8s.io/version=v1.33.1
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 29 Jul 2024 12:24:50 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: ha-767488-m04
AcquireTime: <unset>
RenewTime: Mon, 29 Jul 2024 12:48:30 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 29 Jul 2024 12:48:20 +0000 Mon, 29 Jul 2024 12:48:20 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 29 Jul 2024 12:48:20 +0000 Mon, 29 Jul 2024 12:48:20 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 29 Jul 2024 12:48:20 +0000 Mon, 29 Jul 2024 12:48:20 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 29 Jul 2024 12:48:20 +0000 Mon, 29 Jul 2024 12:48:20 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.39.181
Hostname: ha-767488-m04
Capacity:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 17734596Ki
hugepages-2Mi: 0
memory: 2164184Ki
pods: 110
System Info:
Machine ID: 326a5fb51aae42b7b8056fc3c9e53faf
System UUID: 326a5fb5-1aae-42b7-b805-6fc3c9e53faf
Boot ID: 5525fcaf-d53c-41e4-a857-9519defa86cc
Kernel Version: 5.10.207
OS Image: Buildroot 2023.02.9
Operating System: linux
Architecture: amd64
Container Runtime Version: cri-o://1.29.1
Kubelet Version: v1.30.3
Kube-Proxy Version: v1.30.3
PodCIDR: 10.244.3.0/24
PodCIDRs: 10.244.3.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kindnet-bgb2n 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING) 23m
kube-system kube-proxy-2m5gr 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 23m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (5%!)(MISSING) 100m (5%!)(MISSING)
memory 50Mi (2%!)(MISSING) 50Mi (2%!)(MISSING)
ephemeral-storage 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 9s kube-proxy
Normal Starting 23m kube-proxy
Normal NodeAllocatableEnforced 23m kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 23m (x2 over 23m) kubelet Node ha-767488-m04 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 23m (x2 over 23m) kubelet Node ha-767488-m04 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 23m (x2 over 23m) kubelet Node ha-767488-m04 status is now: NodeHasSufficientPID
Normal RegisteredNode 23m node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal RegisteredNode 23m node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal RegisteredNode 23m node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal NodeReady 23m kubelet Node ha-767488-m04 status is now: NodeReady
Normal RegisteredNode 22m node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal NodeNotReady 21m node-controller Node ha-767488-m04 status is now: NodeNotReady
Normal RegisteredNode 16m node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal RegisteredNode 4m56s node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal RegisteredNode 96s node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal RegisteredNode 32s node-controller Node ha-767488-m04 event: Registered Node ha-767488-m04 in Controller
Normal Starting 14s kubelet Starting kubelet.
Normal NodeAllocatableEnforced 14s kubelet Updated Node Allocatable limit across pods
Normal NodeHasSufficientMemory 13s (x2 over 13s) kubelet Node ha-767488-m04 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 13s (x2 over 13s) kubelet Node ha-767488-m04 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 13s (x2 over 13s) kubelet Node ha-767488-m04 status is now: NodeHasSufficientPID
Warning Rebooted 13s kubelet Node ha-767488-m04 has been rebooted, boot id: 5525fcaf-d53c-41e4-a857-9519defa86cc
Normal NodeReady 13s kubelet Node ha-767488-m04 status is now: NodeReady
==> dmesg <==
[ +10.417395] kauditd_printk_skb: 26 callbacks suppressed
[ +5.584590] kauditd_printk_skb: 34 callbacks suppressed
[Jul29 12:22] kauditd_printk_skb: 26 callbacks suppressed
[Jul29 12:25] kauditd_printk_skb: 10 callbacks suppressed
[Jul29 12:30] systemd-fstab-generator[3284]: Ignoring "noauto" option for root device
[ +0.152481] systemd-fstab-generator[3296]: Ignoring "noauto" option for root device
[ +0.201233] systemd-fstab-generator[3310]: Ignoring "noauto" option for root device
[ +0.141805] systemd-fstab-generator[3322]: Ignoring "noauto" option for root device
[ +0.319718] systemd-fstab-generator[3350]: Ignoring "noauto" option for root device
[ +4.887179] systemd-fstab-generator[3463]: Ignoring "noauto" option for root device
[ +0.088800] kauditd_printk_skb: 100 callbacks suppressed
[ +9.359831] kauditd_printk_skb: 76 callbacks suppressed
[ +5.040803] kauditd_printk_skb: 30 callbacks suppressed
[ +16.902160] kauditd_printk_skb: 1 callbacks suppressed
[ +12.806405] kauditd_printk_skb: 5 callbacks suppressed
[Jul29 12:42] systemd-fstab-generator[6690]: Ignoring "noauto" option for root device
[ +0.156941] systemd-fstab-generator[6701]: Ignoring "noauto" option for root device
[ +0.179346] systemd-fstab-generator[6715]: Ignoring "noauto" option for root device
[ +0.150490] systemd-fstab-generator[6727]: Ignoring "noauto" option for root device
[ +0.290091] systemd-fstab-generator[6755]: Ignoring "noauto" option for root device
[ +8.442471] kauditd_printk_skb: 100 callbacks suppressed
[ +0.232725] systemd-fstab-generator[6987]: Ignoring "noauto" option for root device
[ +4.882020] kauditd_printk_skb: 101 callbacks suppressed
[Jul29 12:43] kauditd_printk_skb: 11 callbacks suppressed
[ +20.903422] kauditd_printk_skb: 10 callbacks suppressed
==> etcd [5c8cded716df9fd9e6e43d630eaed9773e3d3f4a99b24b1e2662048f450fddbf] <==
{"level":"info","ts":"2024-07-29T12:40:51.117051Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"a09c9983ac28f1fd [logterm: 7, index: 3480] sent MsgPreVote request to d9000071a51f92ea at term 7"}
{"level":"info","ts":"2024-07-29T12:40:52.278962Z","caller":"osutil/interrupt_unix.go:64","msg":"received signal; shutting down","signal":"terminated"}
{"level":"info","ts":"2024-07-29T12:40:52.279017Z","caller":"embed/etcd.go:375","msg":"closing etcd server","name":"ha-767488","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.39.217:2380"],"advertise-client-urls":["https://192.168.39.217:2379"]}
{"level":"warn","ts":"2024-07-29T12:40:52.279122Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}
{"level":"warn","ts":"2024-07-29T12:40:52.279145Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 127.0.0.1:2379: use of closed network connection"}
{"level":"warn","ts":"2024-07-29T12:40:52.290948Z","caller":"embed/serve.go:212","msg":"stopping secure grpc server due to error","error":"accept tcp 192.168.39.217:2379: use of closed network connection"}
{"level":"warn","ts":"2024-07-29T12:40:52.291007Z","caller":"embed/serve.go:214","msg":"stopped secure grpc server due to error","error":"accept tcp 192.168.39.217:2379: use of closed network connection"}
{"level":"info","ts":"2024-07-29T12:40:52.291066Z","caller":"etcdserver/server.go:1462","msg":"skipped leadership transfer; local server is not leader","local-member-id":"a09c9983ac28f1fd","current-leader-member-id":"0"}
{"level":"info","ts":"2024-07-29T12:40:52.291289Z","caller":"rafthttp/peer.go:330","msg":"stopping remote peer","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291329Z","caller":"rafthttp/stream.go:294","msg":"stopped TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291362Z","caller":"rafthttp/stream.go:294","msg":"stopped TCP streaming connection with remote peer","stream-writer-type":"stream Message","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291459Z","caller":"rafthttp/pipeline.go:85","msg":"stopped HTTP pipelining with remote peer","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291525Z","caller":"rafthttp/stream.go:442","msg":"stopped stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291589Z","caller":"rafthttp/stream.go:442","msg":"stopped stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291602Z","caller":"rafthttp/peer.go:335","msg":"stopped remote peer","remote-peer-id":"30f76e47e42605a5"}
{"level":"info","ts":"2024-07-29T12:40:52.291608Z","caller":"rafthttp/peer.go:330","msg":"stopping remote peer","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.291616Z","caller":"rafthttp/stream.go:294","msg":"stopped TCP streaming connection with remote peer","stream-writer-type":"unknown stream","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.291637Z","caller":"rafthttp/stream.go:294","msg":"stopped TCP streaming connection with remote peer","stream-writer-type":"unknown stream","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.29172Z","caller":"rafthttp/pipeline.go:85","msg":"stopped HTTP pipelining with remote peer","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.291751Z","caller":"rafthttp/stream.go:442","msg":"stopped stream reader with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.291782Z","caller":"rafthttp/stream.go:442","msg":"stopped stream reader with remote peer","stream-reader-type":"stream Message","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.291865Z","caller":"rafthttp/peer.go:335","msg":"stopped remote peer","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:40:52.304523Z","caller":"embed/etcd.go:579","msg":"stopping serving peer traffic","address":"192.168.39.217:2380"}
{"level":"info","ts":"2024-07-29T12:40:52.30477Z","caller":"embed/etcd.go:584","msg":"stopped serving peer traffic","address":"192.168.39.217:2380"}
{"level":"info","ts":"2024-07-29T12:40:52.304858Z","caller":"embed/etcd.go:377","msg":"closed etcd server","name":"ha-767488","data-dir":"/var/lib/minikube/etcd","advertise-peer-urls":["https://192.168.39.217:2380"],"advertise-client-urls":["https://192.168.39.217:2379"]}
==> etcd [a311cff0c8eccdd5a09eaeacd7969122d9ddde39d014e13fa635c80a8c881828] <==
{"level":"warn","ts":"2024-07-29T12:47:10.268021Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: no route to host"}
{"level":"warn","ts":"2024-07-29T12:47:15.268948Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: i/o timeout"}
{"level":"warn","ts":"2024-07-29T12:47:15.269045Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: i/o timeout"}
{"level":"warn","ts":"2024-07-29T12:47:20.26945Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: no route to host"}
{"level":"warn","ts":"2024-07-29T12:47:20.269572Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: no route to host"}
{"level":"warn","ts":"2024-07-29T12:47:25.269545Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:25.26973Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:30.269867Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:30.270071Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:35.270164Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:35.270231Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:40.271157Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"warn","ts":"2024-07-29T12:47:40.271332Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"d9000071a51f92ea","rtt":"0s","error":"dial tcp 192.168.39.210:2380: connect: connection refused"}
{"level":"info","ts":"2024-07-29T12:47:44.279992Z","caller":"rafthttp/peer_status.go:53","msg":"peer became active","peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:47:44.280139Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream Message","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:47:44.280224Z","caller":"rafthttp/stream.go:412","msg":"established TCP streaming connection with remote peer","stream-reader-type":"stream MsgApp v2","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:47:44.287376Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"a09c9983ac28f1fd","to":"d9000071a51f92ea","stream-type":"stream MsgApp v2"}
{"level":"info","ts":"2024-07-29T12:47:44.287506Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream MsgApp v2","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"info","ts":"2024-07-29T12:47:44.291104Z","caller":"rafthttp/stream.go:249","msg":"set message encoder","from":"a09c9983ac28f1fd","to":"d9000071a51f92ea","stream-type":"stream Message"}
{"level":"info","ts":"2024-07-29T12:47:44.291154Z","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"a09c9983ac28f1fd","remote-peer-id":"d9000071a51f92ea"}
{"level":"warn","ts":"2024-07-29T12:47:44.965528Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"106.230668ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/etcd-ha-767488-m03\" ","response":"range_response_count:1 size:5802"}
{"level":"info","ts":"2024-07-29T12:47:44.966066Z","caller":"traceutil/trace.go:171","msg":"trace[453902694] range","detail":"{range_begin:/registry/pods/kube-system/etcd-ha-767488-m03; range_end:; response_count:1; response_revision:3782; }","duration":"106.799466ms","start":"2024-07-29T12:47:44.859168Z","end":"2024-07-29T12:47:44.965968Z","steps":["trace[453902694] 'range keys from in-memory index tree' (duration: 103.837804ms)"],"step_count":1}
{"level":"info","ts":"2024-07-29T12:47:50.971028Z","caller":"traceutil/trace.go:171","msg":"trace[1827493490] linearizableReadLoop","detail":"{readStateIndex:4330; appliedIndex:4330; }","duration":"111.574421ms","start":"2024-07-29T12:47:50.859424Z","end":"2024-07-29T12:47:50.970999Z","steps":["trace[1827493490] 'read index received' (duration: 111.56917ms)","trace[1827493490] 'applied index is now lower than readState.Index' (duration: 3.955µs)"],"step_count":2}
{"level":"warn","ts":"2024-07-29T12:47:50.971255Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"111.808091ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/etcd-ha-767488-m03\" ","response":"range_response_count:1 size:5802"}
{"level":"info","ts":"2024-07-29T12:47:50.971313Z","caller":"traceutil/trace.go:171","msg":"trace[1184293185] range","detail":"{range_begin:/registry/pods/kube-system/etcd-ha-767488-m03; range_end:; response_count:1; response_revision:3807; }","duration":"111.898697ms","start":"2024-07-29T12:47:50.8594Z","end":"2024-07-29T12:47:50.971299Z","steps":["trace[1184293185] 'agreement among raft nodes before linearized reading' (duration: 111.702182ms)"],"step_count":1}
==> kernel <==
12:48:33 up 28 min, 0 users, load average: 0.66, 0.36, 0.31
Linux ha-767488 5.10.207 #1 SMP Tue Jul 23 04:25:44 UTC 2024 x86_64 GNU/Linux
PRETTY_NAME="Buildroot 2023.02.9"
==> kindnet [a327747c60c54b449c2a93db3bd91d7cabe5a45c6e98b63844422773b88816ad] <==
I0729 12:40:29.352753 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:40:29.352910 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:40:29.352935 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
I0729 12:40:29.353062 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:40:29.353084 1 main.go:299] handling current node
I0729 12:40:39.356408 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:40:39.356467 1 main.go:299] handling current node
I0729 12:40:39.356485 1 main.go:295] Handling node with IPs: map[192.168.39.45:{}]
I0729 12:40:39.356493 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:40:39.356693 1 main.go:295] Handling node with IPs: map[192.168.39.210:{}]
I0729 12:40:39.356728 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:40:39.356862 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:40:39.356895 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
W0729 12:40:42.805541 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.2/tools/cache/reflector.go:232: failed to list *v1.Node: Get "https://10.96.0.1:443/api/v1/nodes?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
E0729 12:40:42.805601 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.2/tools/cache/reflector.go:232: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://10.96.0.1:443/api/v1/nodes?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused - error from a previous attempt: unexpected EOF
I0729 12:40:49.352084 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:40:49.352122 1 main.go:299] handling current node
I0729 12:40:49.352136 1 main.go:295] Handling node with IPs: map[192.168.39.45:{}]
I0729 12:40:49.352140 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:40:49.352268 1 main.go:295] Handling node with IPs: map[192.168.39.210:{}]
I0729 12:40:49.352274 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:40:49.352317 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:40:49.352321 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
W0729 12:40:50.573724 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.2/tools/cache/reflector.go:232: failed to list *v1.Node: Get "https://10.96.0.1:443/api/v1/nodes?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused
E0729 12:40:50.573775 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.2/tools/cache/reflector.go:232: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://10.96.0.1:443/api/v1/nodes?resourceVersion=3049": dial tcp 10.96.0.1:443: connect: connection refused
==> kindnet [b50ae6e8e38f62d2e29778940201dac4ae9487c76d83d6f2fc51b50ba9c57577] <==
I0729 12:47:55.907962 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:48:05.899403 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:48:05.899505 1 main.go:299] handling current node
I0729 12:48:05.899541 1 main.go:295] Handling node with IPs: map[192.168.39.45:{}]
I0729 12:48:05.899572 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:48:05.899747 1 main.go:295] Handling node with IPs: map[192.168.39.210:{}]
I0729 12:48:05.899783 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:48:05.900023 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:48:05.900055 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
I0729 12:48:15.907553 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:48:15.907654 1 main.go:299] handling current node
I0729 12:48:15.907674 1 main.go:295] Handling node with IPs: map[192.168.39.45:{}]
I0729 12:48:15.907683 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:48:15.907923 1 main.go:295] Handling node with IPs: map[192.168.39.210:{}]
I0729 12:48:15.907962 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:48:15.908062 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:48:15.908093 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
I0729 12:48:25.900373 1 main.go:295] Handling node with IPs: map[192.168.39.217:{}]
I0729 12:48:25.900491 1 main.go:299] handling current node
I0729 12:48:25.900524 1 main.go:295] Handling node with IPs: map[192.168.39.45:{}]
I0729 12:48:25.900544 1 main.go:322] Node ha-767488-m02 has CIDR [10.244.1.0/24]
I0729 12:48:25.900716 1 main.go:295] Handling node with IPs: map[192.168.39.210:{}]
I0729 12:48:25.900741 1 main.go:322] Node ha-767488-m03 has CIDR [10.244.2.0/24]
I0729 12:48:25.900893 1 main.go:295] Handling node with IPs: map[192.168.39.181:{}]
I0729 12:48:25.900921 1 main.go:322] Node ha-767488-m04 has CIDR [10.244.3.0/24]
==> kube-apiserver [76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf] <==
I0729 12:42:49.430000 1 options.go:221] external host was not specified, using 192.168.39.217
I0729 12:42:49.431238 1 server.go:148] Version: v1.30.3
I0729 12:42:49.431325 1 server.go:150] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0729 12:42:49.866306 1 shared_informer.go:313] Waiting for caches to sync for node_authorizer
I0729 12:42:49.878045 1 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
I0729 12:42:49.881570 1 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0729 12:42:49.881667 1 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
I0729 12:42:49.881920 1 instance.go:299] Using reconciler: lease
W0729 12:43:09.865456 1 logging.go:59] [core] [Channel #1 SubChannel #4] grpc: addrConn.createTransport failed to connect to {Addr: "127.0.0.1:2379", ServerName: "127.0.0.1:2379", }. Err: connection error: desc = "transport: authentication handshake failed: context canceled"
W0729 12:43:09.865695 1 logging.go:59] [core] [Channel #2 SubChannel #3] grpc: addrConn.createTransport failed to connect to {Addr: "127.0.0.1:2379", ServerName: "127.0.0.1:2379", }. Err: connection error: desc = "transport: authentication handshake failed: context canceled"
F0729 12:43:09.882923 1 instance.go:292] Error creating leases: error creating storage factory: context deadline exceeded
==> kube-apiserver [e93850281207e87f03fc437f6794b568715decdd6504954ba8367283e92bcf15] <==
I0729 12:45:59.628077 1 crd_finalizer.go:266] Starting CRDFinalizer
I0729 12:45:59.628961 1 aggregator.go:163] waiting for initial CRD sync...
I0729 12:45:59.629024 1 gc_controller.go:78] Starting apiserver lease garbage collector
I0729 12:45:59.629373 1 available_controller.go:423] Starting AvailableConditionController
I0729 12:45:59.638565 1 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I0729 12:45:59.676095 1 shared_informer.go:320] Caches are synced for node_authorizer
I0729 12:45:59.693227 1 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
I0729 12:45:59.693266 1 policy_source.go:224] refreshing policies
I0729 12:45:59.696534 1 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
I0729 12:45:59.726284 1 cache.go:39] Caches are synced for APIServiceRegistrationController controller
I0729 12:45:59.731995 1 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
I0729 12:45:59.732057 1 handler_discovery.go:447] Starting ResourceDiscoveryManager
I0729 12:45:59.732134 1 shared_informer.go:320] Caches are synced for configmaps
I0729 12:45:59.732180 1 apf_controller.go:379] Running API Priority and Fairness config worker
I0729 12:45:59.732200 1 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
I0729 12:45:59.732874 1 shared_informer.go:320] Caches are synced for crd-autoregister
I0729 12:45:59.733570 1 aggregator.go:165] initial CRD sync complete...
I0729 12:45:59.733620 1 autoregister_controller.go:141] Starting autoregister controller
I0729 12:45:59.733627 1 cache.go:32] Waiting for caches to sync for autoregister controller
I0729 12:45:59.733632 1 cache.go:39] Caches are synced for autoregister controller
I0729 12:45:59.738603 1 cache.go:39] Caches are synced for AvailableConditionController controller
I0729 12:46:00.638385 1 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
W0729 12:46:01.060195 1 lease.go:265] Resetting endpoints for master service "kubernetes" to [192.168.39.217 192.168.39.45]
I0729 12:46:01.061970 1 controller.go:615] quota admission added evaluator for: endpoints
I0729 12:46:01.070860 1 controller.go:615] quota admission added evaluator for: endpointslices.discovery.k8s.io
==> kube-controller-manager [2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b] <==
I0729 12:45:07.191616 1 serving.go:380] Generated self-signed cert in-memory
I0729 12:45:07.718594 1 controllermanager.go:189] "Starting" version="v1.30.3"
I0729 12:45:07.718688 1 controllermanager.go:191] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0729 12:45:07.721330 1 secure_serving.go:213] Serving securely on 127.0.0.1:10257
I0729 12:45:07.723008 1 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/minikube/certs/front-proxy-ca.crt"
I0729 12:45:07.723288 1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0729 12:45:07.723400 1 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/minikube/certs/ca.crt"
E0729 12:45:17.724994 1 controllermanager.go:234] "Error building controller context" err="failed to wait for apiserver being healthy: timed out waiting for the condition: failed to get apiserver /healthz status: Get \"https://192.168.39.217:8443/healthz\": dial tcp 192.168.39.217:8443: connect: connection refused"
==> kube-controller-manager [f525ef9d817220a42df6d260e402031bc0f40e9d8eccfa433f89cb8d789509a0] <==
I0729 12:46:57.268379 1 shared_informer.go:320] Caches are synced for TTL after finished
I0729 12:46:57.274850 1 shared_informer.go:320] Caches are synced for service account
I0729 12:46:57.279272 1 shared_informer.go:320] Caches are synced for job
I0729 12:46:57.281941 1 shared_informer.go:320] Caches are synced for HPA
I0729 12:46:57.302717 1 node_lifecycle_controller.go:879] "Missing timestamp for Node. Assuming now as a timestamp" logger="node-lifecycle-controller" node="ha-767488-m04"
I0729 12:46:57.302855 1 node_lifecycle_controller.go:879] "Missing timestamp for Node. Assuming now as a timestamp" logger="node-lifecycle-controller" node="ha-767488"
I0729 12:46:57.302885 1 node_lifecycle_controller.go:879] "Missing timestamp for Node. Assuming now as a timestamp" logger="node-lifecycle-controller" node="ha-767488-m02"
I0729 12:46:57.302912 1 node_lifecycle_controller.go:879] "Missing timestamp for Node. Assuming now as a timestamp" logger="node-lifecycle-controller" node="ha-767488-m03"
I0729 12:46:57.303103 1 node_lifecycle_controller.go:1073] "Controller detected that zone is now in new state" logger="node-lifecycle-controller" zone="" newState="Normal"
I0729 12:46:57.306059 1 shared_informer.go:320] Caches are synced for ReplicationController
I0729 12:46:57.316545 1 shared_informer.go:320] Caches are synced for disruption
I0729 12:46:57.355364 1 shared_informer.go:320] Caches are synced for cronjob
I0729 12:46:57.366896 1 shared_informer.go:320] Caches are synced for ClusterRoleAggregator
I0729 12:46:57.410323 1 shared_informer.go:320] Caches are synced for endpoint_slice_mirroring
I0729 12:46:57.414574 1 shared_informer.go:320] Caches are synced for resource quota
I0729 12:46:57.464419 1 shared_informer.go:320] Caches are synced for resource quota
I0729 12:46:57.471777 1 shared_informer.go:320] Caches are synced for endpoint_slice
I0729 12:46:57.890551 1 shared_informer.go:320] Caches are synced for garbage collector
I0729 12:46:57.958671 1 shared_informer.go:320] Caches are synced for garbage collector
I0729 12:46:57.958714 1 garbagecollector.go:157] "All resource monitors have synced. Proceeding to collect garbage" logger="garbage-collector-controller"
I0729 12:47:28.903601 1 replica_set.go:676] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-fc5497c4f" duration="81.335µs"
I0729 12:47:29.046385 1 replica_set.go:676] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-fc5497c4f" duration="103.149µs"
I0729 12:47:29.064238 1 replica_set.go:676] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-fc5497c4f" duration="61.904µs"
I0729 12:47:29.065729 1 replica_set.go:676] "Finished syncing" logger="replicaset-controller" kind="ReplicaSet" key="default/busybox-fc5497c4f" duration="34.519µs"
I0729 12:48:20.069830 1 topologycache.go:237] "Can't get CPU or zone information for node" logger="endpointslice-controller" node="ha-767488-m04"
==> kube-proxy [88ec5aa0ed7ecc3dc5a8c5528be305d387f6052f87d62b602f7249bf07514770] <==
E0729 12:38:58.686582 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:04.830714 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:04.830972 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:04.831090 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:04.831124 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:04.831182 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:04.831211 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:14.047354 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:14.048013 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:14.047870 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:14.048111 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:17.119592 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:17.119666 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:35.551618 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:35.551700 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:35.551992 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:35.552162 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:39:41.695764 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:39:41.696014 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:40:06.272423 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:40:06.272868 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&resourceVersion=2975": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:40:21.631308 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:40:21.631558 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=3000": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:40:24.703209 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:40:24.703408 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&resourceVersion=2945": dial tcp 192.168.39.254:8443: connect: no route to host
==> kube-proxy [cc3c94fe6246ab5e4e5ebc15e5df7177339e6e1803a5de62af9f05fcf468e4c0] <==
I0729 12:43:26.005579 1 proxier.go:243] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses"
I0729 12:43:26.005921 1 server.go:872] "Version info" version="v1.30.3"
I0729 12:43:26.005961 1 server.go:874] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0729 12:43:26.008082 1 config.go:192] "Starting service config controller"
I0729 12:43:26.008129 1 shared_informer.go:313] Waiting for caches to sync for service config
I0729 12:43:26.008154 1 config.go:101] "Starting endpoint slice config controller"
I0729 12:43:26.008158 1 shared_informer.go:313] Waiting for caches to sync for endpoint slice config
I0729 12:43:26.009228 1 config.go:319] "Starting node config controller"
I0729 12:43:26.009261 1 shared_informer.go:313] Waiting for caches to sync for node config
E0729 12:43:29.022916 1 event_broadcaster.go:279] "Unable to write event (may retry after sleeping)" err="Post \"https://control-plane.minikube.internal:8443/apis/events.k8s.io/v1/namespaces/default/events\": dial tcp 192.168.39.254:8443: connect: no route to host"
W0729 12:43:29.023565 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:29.023756 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:43:29.023657 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:43:29.023892 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:29.023968 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:29.024040 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:43:32.094641 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:32.094724 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: Get "https://control-plane.minikube.internal:8443/apis/discovery.k8s.io/v1/endpointslices?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:43:32.094863 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:32.094895 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://control-plane.minikube.internal:8443/api/v1/nodes?fieldSelector=metadata.name%!D(MISSING)ha-767488&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
W0729 12:43:32.094978 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
E0729 12:43:32.095150 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://control-plane.minikube.internal:8443/api/v1/services?labelSelector=%!s(MISSING)ervice.kubernetes.io%!F(MISSING)headless%!C(MISSING)%!s(MISSING)ervice.kubernetes.io%!F(MISSING)service-proxy-name&limit=500&resourceVersion=0": dial tcp 192.168.39.254:8443: connect: no route to host
I0729 12:43:33.908457 1 shared_informer.go:320] Caches are synced for service config
I0729 12:43:34.210378 1 shared_informer.go:320] Caches are synced for node config
I0729 12:43:34.908288 1 shared_informer.go:320] Caches are synced for endpoint slice config
==> kube-scheduler [5e886bb5a4a2efa60f1c13486b5b6179b7fb96f992a1ec20d9263cb7bd3ab887] <==
W0729 12:40:22.243325 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0729 12:40:22.243419 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
W0729 12:40:23.541590 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0729 12:40:23.541652 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
W0729 12:40:24.030114 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
E0729 12:40:24.030218 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
W0729 12:40:24.827144 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope
E0729 12:40:24.827194 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIStorageCapacity: failed to list *v1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope
W0729 12:40:25.963020 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope
E0729 12:40:25.963127 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:kube-scheduler" cannot list resource "namespaces" in API group "" at the cluster scope
W0729 12:40:27.525553 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0729 12:40:27.525717 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
W0729 12:40:31.457216 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0729 12:40:31.457249 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
W0729 12:40:31.946204 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0729 12:40:31.946255 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
W0729 12:40:31.987696 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0729 12:40:31.987742 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
W0729 12:40:32.539286 1 reflector.go:547] runtime/asm_amd64.s:1695: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0729 12:40:32.539318 1 reflector.go:150] runtime/asm_amd64.s:1695: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
W0729 12:40:33.993576 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0729 12:40:33.993629 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
W0729 12:40:34.509160 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0729 12:40:34.509295 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0729 12:40:52.283637 1 run.go:74] "command failed" err="finished without leader elect"
==> kube-scheduler [aa1dfc42a005d85caf3f6c775efc543ba30e572f1a2aef81da961336b90306ba] <==
W0729 12:45:23.326174 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicationController: Get "https://192.168.39.217:8443/api/v1/replicationcontrollers?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:23.326305 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: Get "https://192.168.39.217:8443/api/v1/replicationcontrollers?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:24.458610 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PodDisruptionBudget: Get "https://192.168.39.217:8443/apis/policy/v1/poddisruptionbudgets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:24.458749 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: Get "https://192.168.39.217:8443/apis/policy/v1/poddisruptionbudgets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:31.373849 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:31.374004 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:32.914014 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://192.168.39.217:8443/api/v1/services?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:32.914137 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get "https://192.168.39.217:8443/api/v1/services?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:33.370926 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StorageClass: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:33.371053 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:38.194294 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Pod: Get "https://192.168.39.217:8443/api/v1/pods?fieldSelector=status.phase%3DSucceeded%!C(MISSING)status.phase%3DFailed&limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:38.194364 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Pod: failed to list *v1.Pod: Get "https://192.168.39.217:8443/api/v1/pods?fieldSelector=status.phase%3DSucceeded%!C(MISSING)status.phase%3DFailed&limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:39.021894 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.StatefulSet: Get "https://192.168.39.217:8443/apis/apps/v1/statefulsets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:39.022019 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: Get "https://192.168.39.217:8443/apis/apps/v1/statefulsets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:40.035456 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.ReplicaSet: Get "https://192.168.39.217:8443/apis/apps/v1/replicasets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:40.035546 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: Get "https://192.168.39.217:8443/apis/apps/v1/replicasets?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:41.325471 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSINode: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/csinodes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:41.325533 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSINode: failed to list *v1.CSINode: Get "https://192.168.39.217:8443/apis/storage.k8s.io/v1/csinodes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:53.830647 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://192.168.39.217:8443/api/v1/nodes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:53.830888 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get "https://192.168.39.217:8443/api/v1/nodes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:54.268363 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Namespace: Get "https://192.168.39.217:8443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:54.268503 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Namespace: failed to list *v1.Namespace: Get "https://192.168.39.217:8443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
W0729 12:45:54.603189 1 reflector.go:547] k8s.io/client-go/informers/factory.go:160: failed to list *v1.PersistentVolume: Get "https://192.168.39.217:8443/api/v1/persistentvolumes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
E0729 12:45:54.603346 1 reflector.go:150] k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: Get "https://192.168.39.217:8443/api/v1/persistentvolumes?limit=500&resourceVersion=0": dial tcp 192.168.39.217:8443: connect: connection refused
I0729 12:46:02.189750 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Jul 29 12:45:51 ha-767488 kubelet[1381]: I0729 12:45:51.667355 1381 scope.go:117] "RemoveContainer" containerID="2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b"
Jul 29 12:45:51 ha-767488 kubelet[1381]: E0729 12:45:51.667697 1381 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=kube-controller-manager pod=kube-controller-manager-ha-767488_kube-system(6429dcd204de47eb64e9eb4c7981c7df)\"" pod="kube-system/kube-controller-manager-ha-767488" podUID="6429dcd204de47eb64e9eb4c7981c7df"
Jul 29 12:45:57 ha-767488 kubelet[1381]: I0729 12:45:57.666883 1381 scope.go:117] "RemoveContainer" containerID="76489ee06a4777248ed9c7fd562daf9d74eb130de043482ad38e6ad5a1844cdf"
Jul 29 12:46:04 ha-767488 kubelet[1381]: I0729 12:46:04.667382 1381 scope.go:117] "RemoveContainer" containerID="2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b"
Jul 29 12:46:04 ha-767488 kubelet[1381]: E0729 12:46:04.667691 1381 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=kube-controller-manager pod=kube-controller-manager-ha-767488_kube-system(6429dcd204de47eb64e9eb4c7981c7df)\"" pod="kube-system/kube-controller-manager-ha-767488" podUID="6429dcd204de47eb64e9eb4c7981c7df"
Jul 29 12:46:06 ha-767488 kubelet[1381]: E0729 12:46:06.693101 1381 iptables.go:577] "Could not set up iptables canary" err=<
Jul 29 12:46:06 ha-767488 kubelet[1381]: error creating chain "KUBE-KUBELET-CANARY": exit status 3: Ignoring deprecated --wait-interval option.
Jul 29 12:46:06 ha-767488 kubelet[1381]: ip6tables v1.8.9 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Jul 29 12:46:06 ha-767488 kubelet[1381]: Perhaps ip6tables or your kernel needs to be upgraded.
Jul 29 12:46:06 ha-767488 kubelet[1381]: > table="nat" chain="KUBE-KUBELET-CANARY"
Jul 29 12:46:19 ha-767488 kubelet[1381]: I0729 12:46:19.667641 1381 scope.go:117] "RemoveContainer" containerID="2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b"
Jul 29 12:46:19 ha-767488 kubelet[1381]: E0729 12:46:19.668493 1381 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=kube-controller-manager pod=kube-controller-manager-ha-767488_kube-system(6429dcd204de47eb64e9eb4c7981c7df)\"" pod="kube-system/kube-controller-manager-ha-767488" podUID="6429dcd204de47eb64e9eb4c7981c7df"
Jul 29 12:46:32 ha-767488 kubelet[1381]: I0729 12:46:32.667323 1381 scope.go:117] "RemoveContainer" containerID="2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b"
Jul 29 12:46:32 ha-767488 kubelet[1381]: E0729 12:46:32.669010 1381 pod_workers.go:1298] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-controller-manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=kube-controller-manager pod=kube-controller-manager-ha-767488_kube-system(6429dcd204de47eb64e9eb4c7981c7df)\"" pod="kube-system/kube-controller-manager-ha-767488" podUID="6429dcd204de47eb64e9eb4c7981c7df"
Jul 29 12:46:45 ha-767488 kubelet[1381]: I0729 12:46:45.667906 1381 scope.go:117] "RemoveContainer" containerID="2276a6710daab233cb8757ef722b8e0329b2cf2fbd551a34d51bf17ad366e88b"
Jul 29 12:47:06 ha-767488 kubelet[1381]: E0729 12:47:06.688205 1381 iptables.go:577] "Could not set up iptables canary" err=<
Jul 29 12:47:06 ha-767488 kubelet[1381]: error creating chain "KUBE-KUBELET-CANARY": exit status 3: Ignoring deprecated --wait-interval option.
Jul 29 12:47:06 ha-767488 kubelet[1381]: ip6tables v1.8.9 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Jul 29 12:47:06 ha-767488 kubelet[1381]: Perhaps ip6tables or your kernel needs to be upgraded.
Jul 29 12:47:06 ha-767488 kubelet[1381]: > table="nat" chain="KUBE-KUBELET-CANARY"
Jul 29 12:48:06 ha-767488 kubelet[1381]: E0729 12:48:06.688745 1381 iptables.go:577] "Could not set up iptables canary" err=<
Jul 29 12:48:06 ha-767488 kubelet[1381]: error creating chain "KUBE-KUBELET-CANARY": exit status 3: Ignoring deprecated --wait-interval option.
Jul 29 12:48:06 ha-767488 kubelet[1381]: ip6tables v1.8.9 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Jul 29 12:48:06 ha-767488 kubelet[1381]: Perhaps ip6tables or your kernel needs to be upgraded.
Jul 29 12:48:06 ha-767488 kubelet[1381]: > table="nat" chain="KUBE-KUBELET-CANARY"
-- /stdout --
** stderr **
E0729 12:48:31.979343 262797 logs.go:258] failed to output last start logs: failed to read file /home/jenkins/minikube-integration/19341-233093/.minikube/logs/lastStart.txt: bufio.Scanner: token too long
** /stderr **
helpers_test.go:254: (dbg) Run: out/minikube-linux-amd64 status --format={{.APIServer}} -p ha-767488 -n ha-767488
helpers_test.go:261: (dbg) Run: kubectl --context ha-767488 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:285: <<< TestMultiControlPlane/serial/DegradedAfterClusterRestart FAILED: end of post-mortem logs <<<
helpers_test.go:286: ---------------------/post-mortem---------------------------------
--- FAIL: TestMultiControlPlane/serial/DegradedAfterClusterRestart (3.05s)