=== RUN TestKubernetesUpgrade
=== PAUSE TestKubernetesUpgrade
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:229: (dbg) Run: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220921215522-10174 --memory=2200 --kubernetes-version=v1.16.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:229: (dbg) Done: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220921215522-10174 --memory=2200 --kubernetes-version=v1.16.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: (45.93352882s)
version_upgrade_test.go:234: (dbg) Run: out/minikube-linux-amd64 stop -p kubernetes-upgrade-20220921215522-10174
version_upgrade_test.go:234: (dbg) Done: out/minikube-linux-amd64 stop -p kubernetes-upgrade-20220921215522-10174: (1.451142512s)
version_upgrade_test.go:239: (dbg) Run: out/minikube-linux-amd64 -p kubernetes-upgrade-20220921215522-10174 status --format={{.Host}}
version_upgrade_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 -p kubernetes-upgrade-20220921215522-10174 status --format={{.Host}}: exit status 7 (116.677508ms)
-- stdout --
Stopped
-- /stdout --
version_upgrade_test.go:241: status error: exit status 7 (may be ok)
version_upgrade_test.go:250: (dbg) Run: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220921215522-10174 --memory=2200 --kubernetes-version=v1.25.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:250: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220921215522-10174 --memory=2200 --kubernetes-version=v1.25.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: exit status 109 (8m25.341134077s)
-- stdout --
* [kubernetes-upgrade-20220921215522-10174] minikube v1.27.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=14995
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
* Using the docker driver based on existing profile
* Starting control plane node kubernetes-upgrade-20220921215522-10174 in cluster kubernetes-upgrade-20220921215522-10174
* Pulling base image ...
* Restarting existing docker container for "kubernetes-upgrade-20220921215522-10174" ...
* Preparing Kubernetes v1.25.2 on containerd 1.6.8 ...
- kubelet.cni-conf-dir=/etc/cni/net.mk
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
X Problems detected in kubelet:
Sep 21 22:03:45 kubernetes-upgrade-20220921215522-10174 kubelet[12183]: E0921 22:03:45.374524 12183 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12194]: E0921 22:03:46.123389 12194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12205]: E0921 22:03:46.873433 12205 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
-- /stdout --
** stderr **
I0921 21:56:10.331340 163433 out.go:296] Setting OutFile to fd 1 ...
I0921 21:56:10.331470 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:56:10.331485 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:56:10.331492 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:56:10.331667 163433 root.go:333] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/bin
I0921 21:56:10.332402 163433 out.go:303] Setting JSON to false
I0921 21:56:10.334048 163433 start.go:115] hostinfo: {"hostname":"ubuntu-20-agent-6","uptime":2321,"bootTime":1663795049,"procs":549,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1017-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0921 21:56:10.334125 163433 start.go:125] virtualization: kvm guest
I0921 21:56:10.335968 163433 out.go:177] * [kubernetes-upgrade-20220921215522-10174] minikube v1.27.0 on Ubuntu 20.04 (kvm/amd64)
I0921 21:56:10.337813 163433 out.go:177] - MINIKUBE_LOCATION=14995
I0921 21:56:10.337743 163433 notify.go:214] Checking for updates...
I0921 21:56:10.340552 163433 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0921 21:56:10.341844 163433 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig
I0921 21:56:10.343195 163433 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube
I0921 21:56:10.344671 163433 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0921 21:56:10.346603 163433 config.go:180] Loaded profile config "kubernetes-upgrade-20220921215522-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.16.0
I0921 21:56:10.347270 163433 driver.go:365] Setting default libvirt URI to qemu:///system
I0921 21:56:10.377224 163433 docker.go:137] docker version: linux-20.10.18
I0921 21:56:10.377364 163433 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0921 21:56:10.487660 163433 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:3 ContainersPaused:0 ContainersStopped:1 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:81 OomKillDisable:true NGoroutines:78 SystemTime:2022-09-21 21:56:10.404570965 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1017-gcp OperatingSystem:Ubuntu 20.04.5 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33660669952 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:20.10.18 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 Expected:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.10.2] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0921 21:56:10.487806 163433 docker.go:254] overlay module found
I0921 21:56:10.490036 163433 out.go:177] * Using the docker driver based on existing profile
I0921 21:56:10.491262 163433 start.go:284] selected driver: docker
I0921 21:56:10.491291 163433 start.go:808] validating driver "docker" against &{Name:kubernetes-upgrade-20220921215522-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:kubernetes-upgrade-20220921215522-10174 Namespace:default
APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Moun
tPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet}
I0921 21:56:10.491429 163433 start.go:819] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0921 21:56:10.492670 163433 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0921 21:56:10.594538 163433 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:3 ContainersPaused:0 ContainersStopped:1 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:81 OomKillDisable:true NGoroutines:78 SystemTime:2022-09-21 21:56:10.521340444 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1017-gcp OperatingSystem:Ubuntu 20.04.5 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33660669952 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:20.10.18 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 Expected:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.10.2] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0921 21:56:10.594910 163433 cni.go:95] Creating CNI manager for ""
I0921 21:56:10.594926 163433 cni.go:162] "docker" driver + containerd runtime found, recommending kindnet
I0921 21:56:10.594985 163433 start_flags.go:316] config:
{Name:kubernetes-upgrade-20220921215522-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.25.2 ClusterName:kubernetes-upgrade-20220921215522-10174 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.loc
al ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false Di
sableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet}
I0921 21:56:10.597705 163433 out.go:177] * Starting control plane node kubernetes-upgrade-20220921215522-10174 in cluster kubernetes-upgrade-20220921215522-10174
I0921 21:56:10.599045 163433 cache.go:120] Beginning downloading kic base image for docker with containerd
I0921 21:56:10.600331 163433 out.go:177] * Pulling base image ...
I0921 21:56:10.601558 163433 preload.go:132] Checking if preload exists for k8s version v1.25.2 and runtime containerd
I0921 21:56:10.601605 163433 preload.go:148] Found local preload: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4
I0921 21:56:10.601617 163433 cache.go:57] Caching tarball of preloaded images
I0921 21:56:10.601663 163433 image.go:75] Checking for gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c in local docker daemon
I0921 21:56:10.601879 163433 preload.go:174] Found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4 in cache, skipping download
I0921 21:56:10.601898 163433 cache.go:60] Finished verifying existence of preloaded tar for v1.25.2 on containerd
I0921 21:56:10.602026 163433 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/config.json ...
I0921 21:56:10.630294 163433 image.go:79] Found gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c in local docker daemon, skipping pull
I0921 21:56:10.630320 163433 cache.go:142] gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c exists in daemon, skipping load
I0921 21:56:10.630329 163433 cache.go:208] Successfully downloaded all kic artifacts
I0921 21:56:10.630367 163433 start.go:364] acquiring machines lock for kubernetes-upgrade-20220921215522-10174: {Name:mk20cd57c93d40829efc7af906ff33505be4f28e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0921 21:56:10.630503 163433 start.go:368] acquired machines lock for "kubernetes-upgrade-20220921215522-10174" in 79.587µs
I0921 21:56:10.630528 163433 start.go:96] Skipping create...Using existing machine configuration
I0921 21:56:10.630533 163433 fix.go:55] fixHost starting:
I0921 21:56:10.630793 163433 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-20220921215522-10174 --format={{.State.Status}}
I0921 21:56:10.652513 163433 fix.go:103] recreateIfNeeded on kubernetes-upgrade-20220921215522-10174: state=Stopped err=<nil>
W0921 21:56:10.652557 163433 fix.go:129] unexpected machine state, will restart: <nil>
I0921 21:56:10.654643 163433 out.go:177] * Restarting existing docker container for "kubernetes-upgrade-20220921215522-10174" ...
I0921 21:56:10.655913 163433 cli_runner.go:164] Run: docker start kubernetes-upgrade-20220921215522-10174
I0921 21:56:11.033379 163433 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-20220921215522-10174 --format={{.State.Status}}
I0921 21:56:11.061699 163433 kic.go:415] container "kubernetes-upgrade-20220921215522-10174" state is running.
I0921 21:56:11.062108 163433 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220921215522-10174
I0921 21:56:11.089465 163433 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/config.json ...
I0921 21:56:11.089701 163433 machine.go:88] provisioning docker machine ...
I0921 21:56:11.089740 163433 ubuntu.go:169] provisioning hostname "kubernetes-upgrade-20220921215522-10174"
I0921 21:56:11.089791 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:11.118858 163433 main.go:134] libmachine: Using SSH client type: native
I0921 21:56:11.119079 163433 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7ec520] 0x7ef6a0 <nil> [] 0s} 127.0.0.1 49344 <nil> <nil>}
I0921 21:56:11.119101 163433 main.go:134] libmachine: About to run SSH command:
sudo hostname kubernetes-upgrade-20220921215522-10174 && echo "kubernetes-upgrade-20220921215522-10174" | sudo tee /etc/hostname
I0921 21:56:11.119698 163433 main.go:134] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:59362->127.0.0.1:49344: read: connection reset by peer
I0921 21:56:14.259786 163433 main.go:134] libmachine: SSH cmd err, output: <nil>: kubernetes-upgrade-20220921215522-10174
I0921 21:56:14.259886 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:14.282895 163433 main.go:134] libmachine: Using SSH client type: native
I0921 21:56:14.283036 163433 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7ec520] 0x7ef6a0 <nil> [] 0s} 127.0.0.1 49344 <nil> <nil>}
I0921 21:56:14.283059 163433 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\skubernetes-upgrade-20220921215522-10174' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 kubernetes-upgrade-20220921215522-10174/g' /etc/hosts;
else
echo '127.0.1.1 kubernetes-upgrade-20220921215522-10174' | sudo tee -a /etc/hosts;
fi
fi
I0921 21:56:14.415904 163433 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0921 21:56:14.415930 163433 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/c
erts/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube}
I0921 21:56:14.415973 163433 ubuntu.go:177] setting up certificates
I0921 21:56:14.415983 163433 provision.go:83] configureAuth start
I0921 21:56:14.416036 163433 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220921215522-10174
I0921 21:56:14.446504 163433 provision.go:138] copyHostCerts
I0921 21:56:14.446571 163433 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem, removing ...
I0921 21:56:14.446594 163433 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem
I0921 21:56:14.446661 163433 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem (1675 bytes)
I0921 21:56:14.446784 163433 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem, removing ...
I0921 21:56:14.446805 163433 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem
I0921 21:56:14.446843 163433 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem (1078 bytes)
I0921 21:56:14.446930 163433 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem, removing ...
I0921 21:56:14.446943 163433 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem
I0921 21:56:14.446966 163433 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem (1123 bytes)
I0921 21:56:14.447045 163433 provision.go:112] generating server cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem org=jenkins.kubernetes-upgrade-20220921215522-10174 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube kubernetes-upgrade-20220921215522-10174]
I0921 21:56:14.758339 163433 provision.go:172] copyRemoteCerts
I0921 21:56:14.758470 163433 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0921 21:56:14.758518 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:14.784838 163433 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/kubernetes-upgrade-20220921215522-10174/id_rsa Username:docker}
I0921 21:56:14.886688 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0921 21:56:14.907362 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem --> /etc/docker/server.pem (1289 bytes)
I0921 21:56:14.926571 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0921 21:56:14.943628 163433 provision.go:86] duration metric: configureAuth took 527.622441ms
I0921 21:56:14.943658 163433 ubuntu.go:193] setting minikube options for container-runtime
I0921 21:56:14.943854 163433 config.go:180] Loaded profile config "kubernetes-upgrade-20220921215522-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 21:56:14.943870 163433 machine.go:91] provisioned docker machine in 3.854154196s
I0921 21:56:14.943876 163433 start.go:300] post-start starting for "kubernetes-upgrade-20220921215522-10174" (driver="docker")
I0921 21:56:14.943883 163433 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0921 21:56:14.943919 163433 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0921 21:56:14.943951 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:14.969386 163433 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/kubernetes-upgrade-20220921215522-10174/id_rsa Username:docker}
I0921 21:56:15.062984 163433 ssh_runner.go:195] Run: cat /etc/os-release
I0921 21:56:15.065773 163433 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0921 21:56:15.065796 163433 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0921 21:56:15.065804 163433 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0921 21:56:15.065810 163433 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0921 21:56:15.065822 163433 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/addons for local assets ...
I0921 21:56:15.065871 163433 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files for local assets ...
I0921 21:56:15.065935 163433 filesync.go:149] local asset: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem -> 101742.pem in /etc/ssl/certs
I0921 21:56:15.066011 163433 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0921 21:56:15.072296 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem --> /etc/ssl/certs/101742.pem (1708 bytes)
I0921 21:56:15.088672 163433 start.go:303] post-start completed in 144.782833ms
I0921 21:56:15.088749 163433 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0921 21:56:15.088797 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:15.114151 163433 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/kubernetes-upgrade-20220921215522-10174/id_rsa Username:docker}
I0921 21:56:15.204163 163433 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0921 21:56:15.208064 163433 fix.go:57] fixHost completed within 4.577524765s
I0921 21:56:15.208089 163433 start.go:83] releasing machines lock for "kubernetes-upgrade-20220921215522-10174", held for 4.577567952s
I0921 21:56:15.208196 163433 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220921215522-10174
I0921 21:56:15.231039 163433 ssh_runner.go:195] Run: systemctl --version
I0921 21:56:15.231096 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:15.231129 163433 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0921 21:56:15.231189 163433 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220921215522-10174
I0921 21:56:15.258580 163433 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/kubernetes-upgrade-20220921215522-10174/id_rsa Username:docker}
I0921 21:56:15.260147 163433 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/kubernetes-upgrade-20220921215522-10174/id_rsa Username:docker}
I0921 21:56:15.379852 163433 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0921 21:56:15.392076 163433 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0921 21:56:15.401772 163433 docker.go:188] disabling docker service ...
I0921 21:56:15.401815 163433 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0921 21:56:15.412274 163433 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0921 21:56:15.421438 163433 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0921 21:56:15.492628 163433 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0921 21:56:15.580773 163433 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0921 21:56:15.590130 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0921 21:56:15.602646 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*sandbox_image = .*$|sandbox_image = "registry.k8s.io/pause:3.8"|' -i /etc/containerd/config.toml"
I0921 21:56:15.610097 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*restrict_oom_score_adj = .*$|restrict_oom_score_adj = false|' -i /etc/containerd/config.toml"
I0921 21:56:15.618079 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*SystemdCgroup = .*$|SystemdCgroup = false|' -i /etc/containerd/config.toml"
I0921 21:56:15.627516 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*conf_dir = .*$|conf_dir = "/etc/cni/net.mk"|' -i /etc/containerd/config.toml"
I0921 21:56:15.635486 163433 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0921 21:56:15.642158 163433 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0921 21:56:15.649321 163433 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0921 21:56:15.735660 163433 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0921 21:56:15.845934 163433 start.go:450] Will wait 60s for socket path /run/containerd/containerd.sock
I0921 21:56:15.846008 163433 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0921 21:56:15.849816 163433 start.go:471] Will wait 60s for crictl version
I0921 21:56:15.849880 163433 ssh_runner.go:195] Run: sudo crictl version
I0921 21:56:15.892953 163433 start.go:480] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.8
RuntimeApiVersion: v1alpha2
I0921 21:56:15.893038 163433 ssh_runner.go:195] Run: containerd --version
I0921 21:56:15.925993 163433 ssh_runner.go:195] Run: containerd --version
I0921 21:56:15.961048 163433 out.go:177] * Preparing Kubernetes v1.25.2 on containerd 1.6.8 ...
I0921 21:56:15.962324 163433 cli_runner.go:164] Run: docker network inspect kubernetes-upgrade-20220921215522-10174 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0921 21:56:15.985812 163433 ssh_runner.go:195] Run: grep 192.168.67.1 host.minikube.internal$ /etc/hosts
I0921 21:56:15.989027 163433 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.67.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0921 21:56:16.000091 163433 out.go:177] - kubelet.cni-conf-dir=/etc/cni/net.mk
I0921 21:56:16.001787 163433 preload.go:132] Checking if preload exists for k8s version v1.25.2 and runtime containerd
I0921 21:56:16.001873 163433 ssh_runner.go:195] Run: sudo crictl images --output json
I0921 21:56:16.028632 163433 containerd.go:549] couldn't find preloaded image for "registry.k8s.io/kube-apiserver:v1.25.2". assuming images are not preloaded.
I0921 21:56:16.028687 163433 ssh_runner.go:195] Run: which lz4
I0921 21:56:16.032046 163433 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0921 21:56:16.035143 163433 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/preloaded.tar.lz4': No such file or directory
I0921 21:56:16.035176 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (426272118 bytes)
I0921 21:56:17.067812 163433 containerd.go:496] Took 1.035797 seconds to copy over tarball
I0921 21:56:17.067906 163433 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0921 21:56:19.516702 163433 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.448764257s)
I0921 21:56:19.516735 163433 containerd.go:503] Took 2.448890 seconds t extract the tarball
I0921 21:56:19.516748 163433 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0921 21:56:20.318669 163433 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0921 21:56:20.392284 163433 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0921 21:56:20.605112 163433 ssh_runner.go:195] Run: sudo crictl images --output json
I0921 21:56:20.630679 163433 cache_images.go:88] LoadImages start: [registry.k8s.io/kube-apiserver:v1.25.2 registry.k8s.io/kube-controller-manager:v1.25.2 registry.k8s.io/kube-scheduler:v1.25.2 registry.k8s.io/kube-proxy:v1.25.2 registry.k8s.io/pause:3.8 registry.k8s.io/etcd:3.5.4-0 registry.k8s.io/coredns/coredns:v1.9.3 gcr.io/k8s-minikube/storage-provisioner:v5]
I0921 21:56:20.630734 163433 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0921 21:56:20.630773 163433 image.go:134] retrieving image: registry.k8s.io/kube-proxy:v1.25.2
I0921 21:56:20.630808 163433 image.go:134] retrieving image: registry.k8s.io/pause:3.8
I0921 21:56:20.630818 163433 image.go:134] retrieving image: registry.k8s.io/kube-controller-manager:v1.25.2
I0921 21:56:20.630837 163433 image.go:134] retrieving image: registry.k8s.io/kube-apiserver:v1.25.2
I0921 21:56:20.630920 163433 image.go:134] retrieving image: registry.k8s.io/coredns/coredns:v1.9.3
I0921 21:56:20.630934 163433 image.go:134] retrieving image: registry.k8s.io/kube-scheduler:v1.25.2
I0921 21:56:20.630783 163433 image.go:134] retrieving image: registry.k8s.io/etcd:3.5.4-0
I0921 21:56:20.631918 163433 image.go:177] daemon lookup for registry.k8s.io/kube-proxy:v1.25.2: Error: No such image: registry.k8s.io/kube-proxy:v1.25.2
I0921 21:56:20.631944 163433 image.go:177] daemon lookup for registry.k8s.io/kube-apiserver:v1.25.2: Error: No such image: registry.k8s.io/kube-apiserver:v1.25.2
I0921 21:56:20.631963 163433 image.go:177] daemon lookup for registry.k8s.io/kube-scheduler:v1.25.2: Error: No such image: registry.k8s.io/kube-scheduler:v1.25.2
I0921 21:56:20.631983 163433 image.go:177] daemon lookup for registry.k8s.io/etcd:3.5.4-0: Error: No such image: registry.k8s.io/etcd:3.5.4-0
I0921 21:56:20.631963 163433 image.go:177] daemon lookup for registry.k8s.io/pause:3.8: Error: No such image: registry.k8s.io/pause:3.8
I0921 21:56:20.632025 163433 image.go:177] daemon lookup for registry.k8s.io/kube-controller-manager:v1.25.2: Error: No such image: registry.k8s.io/kube-controller-manager:v1.25.2
I0921 21:56:20.632069 163433 image.go:177] daemon lookup for registry.k8s.io/coredns/coredns:v1.9.3: Error: No such image: registry.k8s.io/coredns/coredns:v1.9.3
I0921 21:56:20.632141 163433 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0921 21:56:21.204916 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/etcd:3.5.4-0"
I0921 21:56:21.208786 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/kube-scheduler:v1.25.2"
I0921 21:56:21.208993 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/pause:3.8"
I0921 21:56:21.211741 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/coredns/coredns:v1.9.3"
I0921 21:56:21.220098 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/kube-apiserver:v1.25.2"
I0921 21:56:21.229445 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/kube-proxy:v1.25.2"
I0921 21:56:21.263606 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep registry.k8s.io/kube-controller-manager:v1.25.2"
I0921 21:56:21.481454 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep gcr.io/k8s-minikube/storage-provisioner:v5"
I0921 21:56:22.085332 163433 cache_images.go:116] "registry.k8s.io/etcd:3.5.4-0" needs transfer: "registry.k8s.io/etcd:3.5.4-0" does not exist at hash "a8a176a5d5d698f9409dc246f81fa69d37d4a2f4132ba5e62e72a78476b27f66" in container runtime
I0921 21:56:22.085388 163433 cri.go:216] Removing image: registry.k8s.io/etcd:3.5.4-0
I0921 21:56:22.085446 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.085458 163433 cache_images.go:116] "registry.k8s.io/kube-scheduler:v1.25.2" needs transfer: "registry.k8s.io/kube-scheduler:v1.25.2" does not exist at hash "ca0ea1ee3cfd3d1ced15a8e6f4a236a436c5733b20a0b2dbbfbfd59977e12959" in container runtime
I0921 21:56:22.085498 163433 cri.go:216] Removing image: registry.k8s.io/kube-scheduler:v1.25.2
I0921 21:56:22.085542 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.088947 163433 cache_images.go:116] "registry.k8s.io/pause:3.8" needs transfer: "registry.k8s.io/pause:3.8" does not exist at hash "4873874c08efc72e9729683a83ffbb7502ee729e9a5ac097723806ea7fa13517" in container runtime
I0921 21:56:22.088988 163433 cri.go:216] Removing image: registry.k8s.io/pause:3.8
I0921 21:56:22.089022 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.089129 163433 cache_images.go:116] "registry.k8s.io/coredns/coredns:v1.9.3" needs transfer: "registry.k8s.io/coredns/coredns:v1.9.3" does not exist at hash "5185b96f0becf59032b8e3646e99f84d9655dff3ac9e2605e0dc77f9c441ae4a" in container runtime
I0921 21:56:22.089157 163433 cri.go:216] Removing image: registry.k8s.io/coredns/coredns:v1.9.3
I0921 21:56:22.089184 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.105533 163433 cache_images.go:116] "registry.k8s.io/kube-apiserver:v1.25.2" needs transfer: "registry.k8s.io/kube-apiserver:v1.25.2" does not exist at hash "97801f83949087fbdcc09b1c84ddda0ed5d01f4aabd17787a7714eb2796082b3" in container runtime
I0921 21:56:22.105578 163433 cri.go:216] Removing image: registry.k8s.io/kube-apiserver:v1.25.2
I0921 21:56:22.105628 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.176346 163433 cache_images.go:116] "registry.k8s.io/kube-proxy:v1.25.2" needs transfer: "registry.k8s.io/kube-proxy:v1.25.2" does not exist at hash "1c7d8c51823b5eb08189d553d911097ec8a6a40fea40bb5bdea91842f30d2e86" in container runtime
I0921 21:56:22.176404 163433 cri.go:216] Removing image: registry.k8s.io/kube-proxy:v1.25.2
I0921 21:56:22.176458 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.190226 163433 cache_images.go:116] "registry.k8s.io/kube-controller-manager:v1.25.2" needs transfer: "registry.k8s.io/kube-controller-manager:v1.25.2" does not exist at hash "dbfceb93c69b6d85661fe46c3e50de9e927e4895ebba2892a1db116e69c81890" in container runtime
I0921 21:56:22.190283 163433 cri.go:216] Removing image: registry.k8s.io/kube-controller-manager:v1.25.2
I0921 21:56:22.190325 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.212989 163433 cache_images.go:116] "gcr.io/k8s-minikube/storage-provisioner:v5" needs transfer: "gcr.io/k8s-minikube/storage-provisioner:v5" does not exist at hash "6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562" in container runtime
I0921 21:56:22.213034 163433 cri.go:216] Removing image: gcr.io/k8s-minikube/storage-provisioner:v5
I0921 21:56:22.213075 163433 ssh_runner.go:195] Run: which crictl
I0921 21:56:22.213125 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.5.4-0
I0921 21:56:22.213204 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.25.2
I0921 21:56:22.213240 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.8
I0921 21:56:22.213287 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/coredns/coredns:v1.9.3
I0921 21:56:22.213299 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.25.2
I0921 21:56:22.213370 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.25.2
I0921 21:56:22.213419 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.25.2
I0921 21:56:24.570622 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/etcd:3.5.4-0: (2.357457258s)
I0921 21:56:24.570659 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.5.4-0
I0921 21:56:24.570715 163433 ssh_runner.go:235] Completed: which crictl: (2.357624166s)
I0921 21:56:24.570739 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/etcd_3.5.4-0
I0921 21:56:24.570753 163433 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi gcr.io/k8s-minikube/storage-provisioner:v5
I0921 21:56:24.574767 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/kube-apiserver:v1.25.2: (2.361444016s)
I0921 21:56:24.574799 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.25.2
I0921 21:56:24.574830 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/kube-scheduler:v1.25.2: (2.361542546s)
I0921 21:56:24.574853 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/kube-proxy:v1.25.2: (2.361455016s)
I0921 21:56:24.574868 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.25.2
I0921 21:56:24.574875 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.25.2
I0921 21:56:24.574889 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-apiserver_v1.25.2
I0921 21:56:24.574893 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/pause:3.8: (2.361630108s)
I0921 21:56:24.574902 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/pause_3.8
I0921 21:56:24.574944 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-scheduler_v1.25.2
I0921 21:56:24.574945 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-proxy_v1.25.2
I0921 21:56:24.574957 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/pause_3.8
I0921 21:56:24.574961 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/coredns/coredns:v1.9.3: (2.361653306s)
I0921 21:56:24.574972 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/coredns/coredns_v1.9.3
I0921 21:56:24.574996 163433 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi registry.k8s.io/kube-controller-manager:v1.25.2: (2.361558149s)
I0921 21:56:24.575004 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.25.2
I0921 21:56:24.575016 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/coredns_v1.9.3
I0921 21:56:24.575048 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-controller-manager_v1.25.2
I0921 21:56:24.611381 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/etcd_3.5.4-0: stat -c "%s %y" /var/lib/minikube/images/etcd_3.5.4-0: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/etcd_3.5.4-0': No such file or directory
I0921 21:56:24.611407 163433 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5
I0921 21:56:24.611424 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.5.4-0 --> /var/lib/minikube/images/etcd_3.5.4-0 (102160384 bytes)
I0921 21:56:24.611447 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-scheduler_v1.25.2: stat -c "%s %y" /var/lib/minikube/images/kube-scheduler_v1.25.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-scheduler_v1.25.2': No such file or directory
I0921 21:56:24.611466 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.25.2 --> /var/lib/minikube/images/kube-scheduler_v1.25.2 (15798784 bytes)
I0921 21:56:24.611506 163433 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/storage-provisioner_v5
I0921 21:56:24.611530 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/coredns_v1.9.3: stat -c "%s %y" /var/lib/minikube/images/coredns_v1.9.3: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/coredns_v1.9.3': No such file or directory
I0921 21:56:24.611564 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/coredns/coredns_v1.9.3 --> /var/lib/minikube/images/coredns_v1.9.3 (14839296 bytes)
I0921 21:56:24.611579 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/pause_3.8: stat -c "%s %y" /var/lib/minikube/images/pause_3.8: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/pause_3.8': No such file or directory
I0921 21:56:24.611607 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/pause_3.8 --> /var/lib/minikube/images/pause_3.8 (311296 bytes)
I0921 21:56:24.611669 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-proxy_v1.25.2: stat -c "%s %y" /var/lib/minikube/images/kube-proxy_v1.25.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-proxy_v1.25.2': No such file or directory
I0921 21:56:24.611695 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.25.2 --> /var/lib/minikube/images/kube-proxy_v1.25.2 (20265472 bytes)
I0921 21:56:24.611708 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-apiserver_v1.25.2: stat -c "%s %y" /var/lib/minikube/images/kube-apiserver_v1.25.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-apiserver_v1.25.2': No such file or directory
I0921 21:56:24.611765 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.25.2 --> /var/lib/minikube/images/kube-apiserver_v1.25.2 (34238464 bytes)
I0921 21:56:24.611773 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-controller-manager_v1.25.2: stat -c "%s %y" /var/lib/minikube/images/kube-controller-manager_v1.25.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-controller-manager_v1.25.2': No such file or directory
I0921 21:56:24.611793 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.25.2 --> /var/lib/minikube/images/kube-controller-manager_v1.25.2 (31264256 bytes)
I0921 21:56:24.622202 163433 ssh_runner.go:352] existence check for /var/lib/minikube/images/storage-provisioner_v5: stat -c "%s %y" /var/lib/minikube/images/storage-provisioner_v5: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/storage-provisioner_v5': No such file or directory
I0921 21:56:24.622244 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 --> /var/lib/minikube/images/storage-provisioner_v5 (9060352 bytes)
I0921 21:56:24.682710 163433 containerd.go:233] Loading image: /var/lib/minikube/images/pause_3.8
I0921 21:56:24.682788 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/pause_3.8
I0921 21:56:24.937271 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/pause_3.8 from cache
I0921 21:56:24.937314 163433 containerd.go:233] Loading image: /var/lib/minikube/images/storage-provisioner_v5
I0921 21:56:24.937357 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/storage-provisioner_v5
I0921 21:56:25.588194 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 from cache
I0921 21:56:25.588255 163433 containerd.go:233] Loading image: /var/lib/minikube/images/kube-scheduler_v1.25.2
I0921 21:56:25.588306 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-scheduler_v1.25.2
I0921 21:56:26.465593 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-scheduler_v1.25.2 from cache
I0921 21:56:26.465646 163433 containerd.go:233] Loading image: /var/lib/minikube/images/coredns_v1.9.3
I0921 21:56:26.465706 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/coredns_v1.9.3
I0921 21:56:27.136655 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/coredns/coredns_v1.9.3 from cache
I0921 21:56:27.136701 163433 containerd.go:233] Loading image: /var/lib/minikube/images/kube-proxy_v1.25.2
I0921 21:56:27.136748 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-proxy_v1.25.2
I0921 21:56:27.776645 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-proxy_v1.25.2 from cache
I0921 21:56:27.776692 163433 containerd.go:233] Loading image: /var/lib/minikube/images/kube-controller-manager_v1.25.2
I0921 21:56:27.776740 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-controller-manager_v1.25.2
I0921 21:56:29.653312 163433 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-controller-manager_v1.25.2: (1.876522782s)
I0921 21:56:29.653352 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-controller-manager_v1.25.2 from cache
I0921 21:56:29.653387 163433 containerd.go:233] Loading image: /var/lib/minikube/images/kube-apiserver_v1.25.2
I0921 21:56:29.653443 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-apiserver_v1.25.2
I0921 21:56:32.228396 163433 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-apiserver_v1.25.2: (2.574920147s)
I0921 21:56:32.228432 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/kube-apiserver_v1.25.2 from cache
I0921 21:56:32.228463 163433 containerd.go:233] Loading image: /var/lib/minikube/images/etcd_3.5.4-0
I0921 21:56:32.228507 163433 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/etcd_3.5.4-0
I0921 21:56:37.285785 163433 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/etcd_3.5.4-0: (5.057244447s)
I0921 21:56:37.285819 163433 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/images/amd64/registry.k8s.io/etcd_3.5.4-0 from cache
I0921 21:56:37.285864 163433 cache_images.go:123] Successfully loaded all cached images
I0921 21:56:37.285878 163433 cache_images.go:92] LoadImages completed in 16.655174825s
I0921 21:56:37.285931 163433 ssh_runner.go:195] Run: sudo crictl info
I0921 21:56:37.309301 163433 cni.go:95] Creating CNI manager for ""
I0921 21:56:37.309329 163433 cni.go:162] "docker" driver + containerd runtime found, recommending kindnet
I0921 21:56:37.309344 163433 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0921 21:56:37.309360 163433 kubeadm.go:156] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.25.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:kubernetes-upgrade-20220921215522-10174 NodeName:kubernetes-upgrade-20220921215522-10174 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.67.2 CgroupDriver:cgroupfs ClientCAFi
le:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false}
I0921 21:56:37.309517 163433 kubeadm.go:161] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "kubernetes-upgrade-20220921215522-10174"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.25.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0921 21:56:37.309616 163433 kubeadm.go:962] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.25.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cni-conf-dir=/etc/cni/net.mk --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=kubernetes-upgrade-20220921215522-10174 --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2 --runtime-request-timeout=15m
[Install]
config:
{KubernetesVersion:v1.25.2 ClusterName:kubernetes-upgrade-20220921215522-10174 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0921 21:56:37.309675 163433 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.25.2
I0921 21:56:37.316733 163433 binaries.go:44] Found k8s binaries, skipping transfer
I0921 21:56:37.316807 163433 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0921 21:56:37.323261 163433 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (563 bytes)
I0921 21:56:37.335521 163433 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0921 21:56:37.348505 163433 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2068 bytes)
I0921 21:56:37.361258 163433 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0921 21:56:37.364268 163433 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0921 21:56:37.431032 163433 certs.go:54] Setting up /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174 for IP: 192.168.67.2
I0921 21:56:37.431171 163433 certs.go:182] skipping minikubeCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.key
I0921 21:56:37.431227 163433 certs.go:182] skipping proxyClientCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.key
I0921 21:56:37.431327 163433 certs.go:298] skipping minikube-user signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/client.key
I0921 21:56:37.431423 163433 certs.go:298] skipping minikube signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/apiserver.key.c7fa3a9e
I0921 21:56:37.431484 163433 certs.go:298] skipping aggregator signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/proxy-client.key
I0921 21:56:37.431625 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174.pem (1338 bytes)
W0921 21:56:37.431679 163433 certs.go:384] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174_empty.pem, impossibly tiny 0 bytes
I0921 21:56:37.431692 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem (1679 bytes)
I0921 21:56:37.431738 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem (1078 bytes)
I0921 21:56:37.431770 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem (1123 bytes)
I0921 21:56:37.431797 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/key.pem (1675 bytes)
I0921 21:56:37.431839 163433 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem (1708 bytes)
I0921 21:56:37.432438 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0921 21:56:37.452586 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0921 21:56:37.470430 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0921 21:56:37.490494 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0921 21:56:37.508293 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0921 21:56:37.525537 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0921 21:56:37.542461 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0921 21:56:37.558844 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0921 21:56:37.575269 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174.pem --> /usr/share/ca-certificates/10174.pem (1338 bytes)
I0921 21:56:37.591971 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem --> /usr/share/ca-certificates/101742.pem (1708 bytes)
I0921 21:56:37.609047 163433 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0921 21:56:37.626087 163433 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0921 21:56:37.638310 163433 ssh_runner.go:195] Run: openssl version
I0921 21:56:37.642981 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/10174.pem && ln -fs /usr/share/ca-certificates/10174.pem /etc/ssl/certs/10174.pem"
I0921 21:56:37.649948 163433 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/10174.pem
I0921 21:56:37.652903 163433 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Sep 21 21:32 /usr/share/ca-certificates/10174.pem
I0921 21:56:37.652945 163433 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/10174.pem
I0921 21:56:37.657558 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/10174.pem /etc/ssl/certs/51391683.0"
I0921 21:56:37.663908 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/101742.pem && ln -fs /usr/share/ca-certificates/101742.pem /etc/ssl/certs/101742.pem"
I0921 21:56:37.670847 163433 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/101742.pem
I0921 21:56:37.673590 163433 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Sep 21 21:32 /usr/share/ca-certificates/101742.pem
I0921 21:56:37.673627 163433 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/101742.pem
I0921 21:56:37.678260 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/101742.pem /etc/ssl/certs/3ec20f2e.0"
I0921 21:56:37.684643 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0921 21:56:37.691591 163433 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0921 21:56:37.694518 163433 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Sep 21 21:28 /usr/share/ca-certificates/minikubeCA.pem
I0921 21:56:37.694560 163433 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0921 21:56:37.699047 163433 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0921 21:56:37.705399 163433 kubeadm.go:396] StartCluster: {Name:kubernetes-upgrade-20220921215522-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.25.2 ClusterName:kubernetes-upgrade-20220921215522-10174 Namespace:default APIServerName:miniku
beCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.25.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p
MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet}
I0921 21:56:37.705510 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0921 21:56:37.705541 163433 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0921 21:56:37.728701 163433 cri.go:87] found id: ""
I0921 21:56:37.728761 163433 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0921 21:56:37.735360 163433 kubeadm.go:411] found existing configuration files, will attempt cluster restart
I0921 21:56:37.735382 163433 kubeadm.go:627] restartCluster start
I0921 21:56:37.735414 163433 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0921 21:56:37.742060 163433 kubeadm.go:127] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0921 21:56:37.742793 163433 kubeconfig.go:116] verify returned: extract IP: "kubernetes-upgrade-20220921215522-10174" does not appear in /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig
I0921 21:56:37.743179 163433 kubeconfig.go:127] "kubernetes-upgrade-20220921215522-10174" context is missing from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig - will repair!
I0921 21:56:37.743831 163433 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig: {Name:mkdec5faf1bb182b50a3cd5458b352f38075dc15 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 21:56:37.773145 163433 kapi.go:59] client config for kubernetes-upgrade-20220921215522-10174: &rest.Config{Host:"https://192.168.67.2:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/kubernetes-upgrade-20220921215522-10174/client.crt", KeyFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikub
e/profiles/kubernetes-upgrade-20220921215522-10174/client.key", CAFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x177c400), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0921 21:56:37.773709 163433 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0921 21:56:37.781114 163433 kubeadm.go:594] needs reconfigure: configs differ:
-- stdout --
--- /var/tmp/minikube/kubeadm.yaml 2022-09-21 21:55:39.346078055 +0000
+++ /var/tmp/minikube/kubeadm.yaml.new 2022-09-21 21:56:37.357931182 +0000
@@ -1,4 +1,4 @@
-apiVersion: kubeadm.k8s.io/v1beta1
+apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
@@ -17,7 +17,7 @@
node-ip: 192.168.67.2
taints: []
---
-apiVersion: kubeadm.k8s.io/v1beta1
+apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
@@ -31,16 +31,14 @@
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
-clusterName: kubernetes-upgrade-20220921215522-10174
+clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
-dns:
- type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
- listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
-kubernetesVersion: v1.16.0
+ proxy-refresh-interval: "70000"
+kubernetesVersion: v1.25.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
-- /stdout --
I0921 21:56:37.781133 163433 kubeadm.go:1114] stopping kube-system containers ...
I0921 21:56:37.781144 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name: Namespaces:[kube-system]}
I0921 21:56:37.781189 163433 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0921 21:56:37.806824 163433 cri.go:87] found id: ""
I0921 21:56:37.806889 163433 ssh_runner.go:195] Run: sudo systemctl stop kubelet
I0921 21:56:37.816568 163433 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0921 21:56:37.823171 163433 kubeadm.go:155] found existing configuration files:
-rw------- 1 root root 5763 Sep 21 21:55 /etc/kubernetes/admin.conf
-rw------- 1 root root 5795 Sep 21 21:55 /etc/kubernetes/controller-manager.conf
-rw------- 1 root root 5959 Sep 21 21:55 /etc/kubernetes/kubelet.conf
-rw------- 1 root root 5743 Sep 21 21:55 /etc/kubernetes/scheduler.conf
I0921 21:56:37.823215 163433 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0921 21:56:37.829655 163433 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0921 21:56:37.836313 163433 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0921 21:56:37.843147 163433 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0921 21:56:37.849644 163433 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0921 21:56:37.868031 163433 kubeadm.go:704] reconfiguring cluster from /var/tmp/minikube/kubeadm.yaml
I0921 21:56:37.868055 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"
I0921 21:56:37.914056 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init phase kubeconfig all --config /var/tmp/minikube/kubeadm.yaml"
I0921 21:56:38.509391 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init phase kubelet-start --config /var/tmp/minikube/kubeadm.yaml"
I0921 21:56:38.720801 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init phase control-plane all --config /var/tmp/minikube/kubeadm.yaml"
I0921 21:56:38.776684 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init phase etcd local --config /var/tmp/minikube/kubeadm.yaml"
I0921 21:56:38.821473 163433 api_server.go:51] waiting for apiserver process to appear ...
I0921 21:56:38.821541 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:39.363418 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:39.863350 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:40.362776 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:40.862989 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:41.363110 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:41.864070 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:42.363613 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:42.863166 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:43.363538 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:43.863271 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:44.363689 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:44.863710 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:45.363450 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:45.863207 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:46.363393 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:46.863017 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:47.362827 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:47.862843 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:48.363012 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:48.862984 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:49.363214 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:49.862940 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:50.363406 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:50.863307 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:51.363226 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:51.863752 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:52.363563 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:52.863767 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:53.362794 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:53.863045 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:54.363693 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:54.863784 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:55.363193 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:55.863562 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:56.363072 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:56.863673 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:57.363301 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:57.862802 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:58.363661 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:58.862859 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:59.363013 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:56:59.863552 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:00.363736 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:00.863689 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:01.362901 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:01.863692 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:02.363764 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:02.863627 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:03.363666 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:03.863238 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:04.363699 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:04.863144 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:05.363655 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:05.862795 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:06.363580 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:06.863697 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:07.363056 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:07.863381 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:08.362969 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:08.863597 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:09.362817 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:09.863768 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:10.363112 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:10.863677 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:11.363630 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:11.862948 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:12.363055 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:12.863116 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:13.363194 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:13.863812 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:14.363772 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:14.863066 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:15.363594 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:15.862833 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:16.363057 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:16.863476 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:17.363457 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:17.863598 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:18.363200 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:18.863662 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:19.363783 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:19.863606 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:20.363513 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:20.863177 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:21.363552 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:21.863605 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:22.362969 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:22.863219 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:23.362975 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:23.862827 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:24.363759 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:24.862887 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:25.363082 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:25.862946 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:26.362806 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:26.862848 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:27.362854 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:27.862860 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:28.363238 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:28.863462 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:29.363410 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:29.862868 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:30.362806 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:30.863747 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:31.362855 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:31.862795 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:32.362839 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:32.863197 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:33.363200 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:33.863202 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:34.363417 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:34.863330 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:35.363564 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:35.863799 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:36.363495 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:36.862795 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:37.363575 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:37.863132 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:38.363245 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:38.863005 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:57:38.863086 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:57:38.894188 163433 cri.go:87] found id: ""
I0921 21:57:38.894218 163433 logs.go:274] 0 containers: []
W0921 21:57:38.894227 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:57:38.894235 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:57:38.894283 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:57:38.926492 163433 cri.go:87] found id: ""
I0921 21:57:38.926522 163433 logs.go:274] 0 containers: []
W0921 21:57:38.926530 163433 logs.go:276] No container was found matching "etcd"
I0921 21:57:38.926537 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:57:38.926597 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:57:38.952934 163433 cri.go:87] found id: ""
I0921 21:57:38.952959 163433 logs.go:274] 0 containers: []
W0921 21:57:38.952965 163433 logs.go:276] No container was found matching "coredns"
I0921 21:57:38.952971 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:57:38.953018 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:57:38.974768 163433 cri.go:87] found id: ""
I0921 21:57:38.974796 163433 logs.go:274] 0 containers: []
W0921 21:57:38.974803 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:57:38.974811 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:57:38.974864 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:57:39.014918 163433 cri.go:87] found id: ""
I0921 21:57:39.014948 163433 logs.go:274] 0 containers: []
W0921 21:57:39.014957 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:57:39.014964 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:57:39.015013 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:57:39.044495 163433 cri.go:87] found id: ""
I0921 21:57:39.044521 163433 logs.go:274] 0 containers: []
W0921 21:57:39.044531 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:57:39.044539 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:57:39.044600 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:57:39.070959 163433 cri.go:87] found id: ""
I0921 21:57:39.070983 163433 logs.go:274] 0 containers: []
W0921 21:57:39.070989 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:57:39.070995 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:57:39.071036 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:57:39.104125 163433 cri.go:87] found id: ""
I0921 21:57:39.104156 163433 logs.go:274] 0 containers: []
W0921 21:57:39.104165 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:57:39.104177 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:57:39.104190 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:57:39.130272 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:49 kubernetes-upgrade-20220921215522-10174 kubelet[1391]: E0921 21:56:49.371828 1391 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.130914 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:50 kubernetes-upgrade-20220921215522-10174 kubelet[1406]: E0921 21:56:50.120343 1406 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.131517 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:50 kubernetes-upgrade-20220921215522-10174 kubelet[1419]: E0921 21:56:50.892521 1419 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.132096 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:51 kubernetes-upgrade-20220921215522-10174 kubelet[1433]: E0921 21:56:51.655491 1433 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.132480 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:52 kubernetes-upgrade-20220921215522-10174 kubelet[1445]: E0921 21:56:52.410609 1445 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.132859 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:53 kubernetes-upgrade-20220921215522-10174 kubelet[1458]: E0921 21:56:53.129607 1458 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.133258 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:53 kubernetes-upgrade-20220921215522-10174 kubelet[1470]: E0921 21:56:53.885429 1470 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.133655 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:54 kubernetes-upgrade-20220921215522-10174 kubelet[1485]: E0921 21:56:54.633153 1485 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.134077 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:55 kubernetes-upgrade-20220921215522-10174 kubelet[1498]: E0921 21:56:55.380845 1498 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.134490 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:56 kubernetes-upgrade-20220921215522-10174 kubelet[1512]: E0921 21:56:56.130685 1512 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.134899 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:56 kubernetes-upgrade-20220921215522-10174 kubelet[1525]: E0921 21:56:56.871709 1525 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.135301 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:57 kubernetes-upgrade-20220921215522-10174 kubelet[1540]: E0921 21:56:57.621586 1540 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.135706 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:58 kubernetes-upgrade-20220921215522-10174 kubelet[1554]: E0921 21:56:58.373184 1554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.136148 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:59 kubernetes-upgrade-20220921215522-10174 kubelet[1569]: E0921 21:56:59.122077 1569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.136557 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:59 kubernetes-upgrade-20220921215522-10174 kubelet[1582]: E0921 21:56:59.880949 1582 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.136971 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:00 kubernetes-upgrade-20220921215522-10174 kubelet[1596]: E0921 21:57:00.626960 1596 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.137401 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:01 kubernetes-upgrade-20220921215522-10174 kubelet[1609]: E0921 21:57:01.373032 1609 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.137779 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:02 kubernetes-upgrade-20220921215522-10174 kubelet[1623]: E0921 21:57:02.152349 1623 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.138155 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:02 kubernetes-upgrade-20220921215522-10174 kubelet[1636]: E0921 21:57:02.879019 1636 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.138535 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:03 kubernetes-upgrade-20220921215522-10174 kubelet[1651]: E0921 21:57:03.630895 1651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.138930 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:04 kubernetes-upgrade-20220921215522-10174 kubelet[1663]: E0921 21:57:04.384693 1663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.139319 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:05 kubernetes-upgrade-20220921215522-10174 kubelet[1677]: E0921 21:57:05.137440 1677 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.139710 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:05 kubernetes-upgrade-20220921215522-10174 kubelet[1690]: E0921 21:57:05.872384 1690 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.140128 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:06 kubernetes-upgrade-20220921215522-10174 kubelet[1704]: E0921 21:57:06.626578 1704 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.140516 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:07 kubernetes-upgrade-20220921215522-10174 kubelet[1716]: E0921 21:57:07.372786 1716 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.140897 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:08 kubernetes-upgrade-20220921215522-10174 kubelet[1731]: E0921 21:57:08.125209 1731 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.141280 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:08 kubernetes-upgrade-20220921215522-10174 kubelet[1743]: E0921 21:57:08.875144 1743 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.141682 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:09 kubernetes-upgrade-20220921215522-10174 kubelet[1758]: E0921 21:57:09.636568 1758 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.142068 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:10 kubernetes-upgrade-20220921215522-10174 kubelet[1770]: E0921 21:57:10.371114 1770 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.142441 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1785]: E0921 21:57:11.135143 1785 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.142819 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1797]: E0921 21:57:11.883503 1797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.143206 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:12 kubernetes-upgrade-20220921215522-10174 kubelet[1813]: E0921 21:57:12.638473 1813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.143589 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:13 kubernetes-upgrade-20220921215522-10174 kubelet[1825]: E0921 21:57:13.380895 1825 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.144020 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1840]: E0921 21:57:14.132900 1840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.144398 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1854]: E0921 21:57:14.886412 1854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.144876 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:15 kubernetes-upgrade-20220921215522-10174 kubelet[1869]: E0921 21:57:15.626883 1869 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.145291 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:16 kubernetes-upgrade-20220921215522-10174 kubelet[1883]: E0921 21:57:16.371277 1883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.145674 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1898]: E0921 21:57:17.133191 1898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.146058 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1911]: E0921 21:57:17.887173 1911 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.146436 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:18 kubernetes-upgrade-20220921215522-10174 kubelet[1926]: E0921 21:57:18.653902 1926 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.146835 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:19 kubernetes-upgrade-20220921215522-10174 kubelet[1938]: E0921 21:57:19.383818 1938 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.147210 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1952]: E0921 21:57:20.134629 1952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.147585 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1964]: E0921 21:57:20.874735 1964 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.147986 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:21 kubernetes-upgrade-20220921215522-10174 kubelet[1980]: E0921 21:57:21.642629 1980 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.148364 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:22 kubernetes-upgrade-20220921215522-10174 kubelet[1993]: E0921 21:57:22.383938 1993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.148761 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2007]: E0921 21:57:23.163080 2007 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.149153 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2019]: E0921 21:57:23.898283 2019 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.149530 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:24 kubernetes-upgrade-20220921215522-10174 kubelet[2034]: E0921 21:57:24.644635 2034 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.149914 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:25 kubernetes-upgrade-20220921215522-10174 kubelet[2048]: E0921 21:57:25.386236 2048 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.150298 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2062]: E0921 21:57:26.138653 2062 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.150674 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2074]: E0921 21:57:26.881606 2074 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.151060 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:27 kubernetes-upgrade-20220921215522-10174 kubelet[2089]: E0921 21:57:27.634384 2089 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.151448 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:28 kubernetes-upgrade-20220921215522-10174 kubelet[2101]: E0921 21:57:28.380595 2101 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.151849 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2115]: E0921 21:57:29.141993 2115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.152234 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2127]: E0921 21:57:29.881873 2127 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.152615 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:30 kubernetes-upgrade-20220921215522-10174 kubelet[2141]: E0921 21:57:30.643982 2141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.152990 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:31 kubernetes-upgrade-20220921215522-10174 kubelet[2153]: E0921 21:57:31.387353 2153 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.153364 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2167]: E0921 21:57:32.141004 2167 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.153744 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2180]: E0921 21:57:32.874796 2180 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.154127 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:33 kubernetes-upgrade-20220921215522-10174 kubelet[2195]: E0921 21:57:33.642006 2195 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.154507 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:34 kubernetes-upgrade-20220921215522-10174 kubelet[2208]: E0921 21:57:34.374657 2208 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.154901 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2222]: E0921 21:57:35.147067 2222 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.155278 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.155679 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.156091 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.156477 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.156890 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:57:39.157024 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:57:39.157039 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:57:39.173331 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:57:39.173366 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:57:39.248752 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:57:39.248782 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:57:39.248801 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:57:39.292441 163433 logs.go:123] Gathering logs for container status ...
I0921 21:57:39.292484 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:57:39.327194 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:57:39.327244 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:57:39.327410 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:57:39.327431 163433 out.go:239] Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.327439 163433 out.go:239] Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.327447 163433 out.go:239] Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.327461 163433 out.go:239] Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:39.327469 163433 out.go:239] Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:57:39.327476 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:57:39.327485 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:57:49.328705 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:49.362930 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:57:49.363009 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:57:49.393912 163433 cri.go:87] found id: ""
I0921 21:57:49.393948 163433 logs.go:274] 0 containers: []
W0921 21:57:49.393958 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:57:49.393966 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:57:49.394028 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:57:49.422659 163433 cri.go:87] found id: ""
I0921 21:57:49.422690 163433 logs.go:274] 0 containers: []
W0921 21:57:49.422699 163433 logs.go:276] No container was found matching "etcd"
I0921 21:57:49.422705 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:57:49.422768 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:57:49.447909 163433 cri.go:87] found id: ""
I0921 21:57:49.447936 163433 logs.go:274] 0 containers: []
W0921 21:57:49.447943 163433 logs.go:276] No container was found matching "coredns"
I0921 21:57:49.447949 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:57:49.448002 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:57:49.470954 163433 cri.go:87] found id: ""
I0921 21:57:49.470981 163433 logs.go:274] 0 containers: []
W0921 21:57:49.470988 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:57:49.470993 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:57:49.471035 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:57:49.495362 163433 cri.go:87] found id: ""
I0921 21:57:49.495390 163433 logs.go:274] 0 containers: []
W0921 21:57:49.495398 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:57:49.495407 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:57:49.495465 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:57:49.520329 163433 cri.go:87] found id: ""
I0921 21:57:49.520352 163433 logs.go:274] 0 containers: []
W0921 21:57:49.520360 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:57:49.520367 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:57:49.520406 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:57:49.544637 163433 cri.go:87] found id: ""
I0921 21:57:49.544663 163433 logs.go:274] 0 containers: []
W0921 21:57:49.544669 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:57:49.544677 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:57:49.544739 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:57:49.569543 163433 cri.go:87] found id: ""
I0921 21:57:49.569572 163433 logs.go:274] 0 containers: []
W0921 21:57:49.569581 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:57:49.569593 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:57:49.569607 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:57:49.586699 163433 logs.go:138] Found kubelet problem: Sep 21 21:56:59 kubernetes-upgrade-20220921215522-10174 kubelet[1582]: E0921 21:56:59.880949 1582 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.587358 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:00 kubernetes-upgrade-20220921215522-10174 kubelet[1596]: E0921 21:57:00.626960 1596 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.588060 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:01 kubernetes-upgrade-20220921215522-10174 kubelet[1609]: E0921 21:57:01.373032 1609 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.588754 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:02 kubernetes-upgrade-20220921215522-10174 kubelet[1623]: E0921 21:57:02.152349 1623 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.589453 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:02 kubernetes-upgrade-20220921215522-10174 kubelet[1636]: E0921 21:57:02.879019 1636 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.590100 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:03 kubernetes-upgrade-20220921215522-10174 kubelet[1651]: E0921 21:57:03.630895 1651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.590749 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:04 kubernetes-upgrade-20220921215522-10174 kubelet[1663]: E0921 21:57:04.384693 1663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.591388 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:05 kubernetes-upgrade-20220921215522-10174 kubelet[1677]: E0921 21:57:05.137440 1677 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.592055 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:05 kubernetes-upgrade-20220921215522-10174 kubelet[1690]: E0921 21:57:05.872384 1690 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.592474 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:06 kubernetes-upgrade-20220921215522-10174 kubelet[1704]: E0921 21:57:06.626578 1704 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.592911 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:07 kubernetes-upgrade-20220921215522-10174 kubelet[1716]: E0921 21:57:07.372786 1716 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.593360 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:08 kubernetes-upgrade-20220921215522-10174 kubelet[1731]: E0921 21:57:08.125209 1731 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.593848 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:08 kubernetes-upgrade-20220921215522-10174 kubelet[1743]: E0921 21:57:08.875144 1743 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.594315 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:09 kubernetes-upgrade-20220921215522-10174 kubelet[1758]: E0921 21:57:09.636568 1758 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.594762 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:10 kubernetes-upgrade-20220921215522-10174 kubelet[1770]: E0921 21:57:10.371114 1770 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.595201 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1785]: E0921 21:57:11.135143 1785 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.595734 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1797]: E0921 21:57:11.883503 1797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.596260 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:12 kubernetes-upgrade-20220921215522-10174 kubelet[1813]: E0921 21:57:12.638473 1813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.596772 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:13 kubernetes-upgrade-20220921215522-10174 kubelet[1825]: E0921 21:57:13.380895 1825 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.597473 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1840]: E0921 21:57:14.132900 1840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.598047 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1854]: E0921 21:57:14.886412 1854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.598503 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:15 kubernetes-upgrade-20220921215522-10174 kubelet[1869]: E0921 21:57:15.626883 1869 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.599095 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:16 kubernetes-upgrade-20220921215522-10174 kubelet[1883]: E0921 21:57:16.371277 1883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.599844 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1898]: E0921 21:57:17.133191 1898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.600489 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1911]: E0921 21:57:17.887173 1911 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.601097 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:18 kubernetes-upgrade-20220921215522-10174 kubelet[1926]: E0921 21:57:18.653902 1926 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.601709 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:19 kubernetes-upgrade-20220921215522-10174 kubelet[1938]: E0921 21:57:19.383818 1938 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.602172 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1952]: E0921 21:57:20.134629 1952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.602682 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1964]: E0921 21:57:20.874735 1964 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.603265 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:21 kubernetes-upgrade-20220921215522-10174 kubelet[1980]: E0921 21:57:21.642629 1980 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.603813 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:22 kubernetes-upgrade-20220921215522-10174 kubelet[1993]: E0921 21:57:22.383938 1993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.604222 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2007]: E0921 21:57:23.163080 2007 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.604649 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2019]: E0921 21:57:23.898283 2019 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.605060 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:24 kubernetes-upgrade-20220921215522-10174 kubelet[2034]: E0921 21:57:24.644635 2034 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.605477 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:25 kubernetes-upgrade-20220921215522-10174 kubelet[2048]: E0921 21:57:25.386236 2048 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.605899 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2062]: E0921 21:57:26.138653 2062 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.606310 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2074]: E0921 21:57:26.881606 2074 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.606745 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:27 kubernetes-upgrade-20220921215522-10174 kubelet[2089]: E0921 21:57:27.634384 2089 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.607190 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:28 kubernetes-upgrade-20220921215522-10174 kubelet[2101]: E0921 21:57:28.380595 2101 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.607610 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2115]: E0921 21:57:29.141993 2115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.608058 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2127]: E0921 21:57:29.881873 2127 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.608460 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:30 kubernetes-upgrade-20220921215522-10174 kubelet[2141]: E0921 21:57:30.643982 2141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.608865 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:31 kubernetes-upgrade-20220921215522-10174 kubelet[2153]: E0921 21:57:31.387353 2153 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.609284 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2167]: E0921 21:57:32.141004 2167 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.609705 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2180]: E0921 21:57:32.874796 2180 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.610129 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:33 kubernetes-upgrade-20220921215522-10174 kubelet[2195]: E0921 21:57:33.642006 2195 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.610697 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:34 kubernetes-upgrade-20220921215522-10174 kubelet[2208]: E0921 21:57:34.374657 2208 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.611151 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2222]: E0921 21:57:35.147067 2222 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.611780 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.612390 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.613069 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.613511 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.613927 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.614329 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:39 kubernetes-upgrade-20220921215522-10174 kubelet[2436]: E0921 21:57:39.640823 2436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.614732 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:40 kubernetes-upgrade-20220921215522-10174 kubelet[2448]: E0921 21:57:40.380285 2448 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.615137 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2460]: E0921 21:57:41.157225 2460 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.615525 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2471]: E0921 21:57:41.891401 2471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.615971 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:42 kubernetes-upgrade-20220921215522-10174 kubelet[2482]: E0921 21:57:42.649592 2482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.616375 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:43 kubernetes-upgrade-20220921215522-10174 kubelet[2492]: E0921 21:57:43.370917 2492 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.616776 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2503]: E0921 21:57:44.139479 2503 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.617183 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2514]: E0921 21:57:44.889947 2514 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.617594 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:45 kubernetes-upgrade-20220921215522-10174 kubelet[2524]: E0921 21:57:45.634469 2524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.618021 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.618424 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.618804 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.619211 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.619594 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:57:49.619736 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:57:49.619764 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:57:49.638052 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:57:49.638090 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:57:49.697023 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:57:49.697051 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:57:49.697062 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:57:49.733721 163433 logs.go:123] Gathering logs for container status ...
I0921 21:57:49.733750 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:57:49.760424 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:57:49.760459 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:57:49.760567 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:57:49.760584 163433 out.go:239] Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.760589 163433 out.go:239] Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.760594 163433 out.go:239] Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.760607 163433 out.go:239] Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:57:49.760615 163433 out.go:239] Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:57:49.760626 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:57:49.760633 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:57:59.761918 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:57:59.862750 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:57:59.862841 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:57:59.886652 163433 cri.go:87] found id: ""
I0921 21:57:59.886674 163433 logs.go:274] 0 containers: []
W0921 21:57:59.886681 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:57:59.886687 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:57:59.886739 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:57:59.910351 163433 cri.go:87] found id: ""
I0921 21:57:59.910378 163433 logs.go:274] 0 containers: []
W0921 21:57:59.910387 163433 logs.go:276] No container was found matching "etcd"
I0921 21:57:59.910395 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:57:59.910452 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:57:59.934483 163433 cri.go:87] found id: ""
I0921 21:57:59.934515 163433 logs.go:274] 0 containers: []
W0921 21:57:59.934525 163433 logs.go:276] No container was found matching "coredns"
I0921 21:57:59.934532 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:57:59.934608 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:57:59.958298 163433 cri.go:87] found id: ""
I0921 21:57:59.958329 163433 logs.go:274] 0 containers: []
W0921 21:57:59.958337 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:57:59.958342 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:57:59.958391 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:57:59.982031 163433 cri.go:87] found id: ""
I0921 21:57:59.982060 163433 logs.go:274] 0 containers: []
W0921 21:57:59.982068 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:57:59.982077 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:57:59.982127 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:00.006566 163433 cri.go:87] found id: ""
I0921 21:58:00.006589 163433 logs.go:274] 0 containers: []
W0921 21:58:00.006595 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:00.006601 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:00.006644 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:00.040965 163433 cri.go:87] found id: ""
I0921 21:58:00.040991 163433 logs.go:274] 0 containers: []
W0921 21:58:00.041000 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:00.041007 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:00.041128 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:00.064646 163433 cri.go:87] found id: ""
I0921 21:58:00.064670 163433 logs.go:274] 0 containers: []
W0921 21:58:00.064677 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:00.064686 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:00.064701 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:00.078915 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:00.078943 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:00.132975 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:00.132997 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:00.133007 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:00.170716 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:00.170752 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:00.197469 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:00.197500 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:00.216133 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:10 kubernetes-upgrade-20220921215522-10174 kubelet[1770]: E0921 21:57:10.371114 1770 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.216577 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1785]: E0921 21:57:11.135143 1785 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.217033 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:11 kubernetes-upgrade-20220921215522-10174 kubelet[1797]: E0921 21:57:11.883503 1797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.217523 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:12 kubernetes-upgrade-20220921215522-10174 kubelet[1813]: E0921 21:57:12.638473 1813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.218027 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:13 kubernetes-upgrade-20220921215522-10174 kubelet[1825]: E0921 21:57:13.380895 1825 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.218500 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1840]: E0921 21:57:14.132900 1840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.218934 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:14 kubernetes-upgrade-20220921215522-10174 kubelet[1854]: E0921 21:57:14.886412 1854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.219384 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:15 kubernetes-upgrade-20220921215522-10174 kubelet[1869]: E0921 21:57:15.626883 1869 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.220026 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:16 kubernetes-upgrade-20220921215522-10174 kubelet[1883]: E0921 21:57:16.371277 1883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.220675 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1898]: E0921 21:57:17.133191 1898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.221323 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:17 kubernetes-upgrade-20220921215522-10174 kubelet[1911]: E0921 21:57:17.887173 1911 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.221898 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:18 kubernetes-upgrade-20220921215522-10174 kubelet[1926]: E0921 21:57:18.653902 1926 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.222337 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:19 kubernetes-upgrade-20220921215522-10174 kubelet[1938]: E0921 21:57:19.383818 1938 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.222733 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1952]: E0921 21:57:20.134629 1952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.223174 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1964]: E0921 21:57:20.874735 1964 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.223604 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:21 kubernetes-upgrade-20220921215522-10174 kubelet[1980]: E0921 21:57:21.642629 1980 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.224078 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:22 kubernetes-upgrade-20220921215522-10174 kubelet[1993]: E0921 21:57:22.383938 1993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.224540 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2007]: E0921 21:57:23.163080 2007 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.224973 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2019]: E0921 21:57:23.898283 2019 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.225412 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:24 kubernetes-upgrade-20220921215522-10174 kubelet[2034]: E0921 21:57:24.644635 2034 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.225837 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:25 kubernetes-upgrade-20220921215522-10174 kubelet[2048]: E0921 21:57:25.386236 2048 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.226296 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2062]: E0921 21:57:26.138653 2062 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.226739 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2074]: E0921 21:57:26.881606 2074 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.227230 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:27 kubernetes-upgrade-20220921215522-10174 kubelet[2089]: E0921 21:57:27.634384 2089 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.227674 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:28 kubernetes-upgrade-20220921215522-10174 kubelet[2101]: E0921 21:57:28.380595 2101 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.228127 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2115]: E0921 21:57:29.141993 2115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.228552 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2127]: E0921 21:57:29.881873 2127 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.228964 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:30 kubernetes-upgrade-20220921215522-10174 kubelet[2141]: E0921 21:57:30.643982 2141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.229390 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:31 kubernetes-upgrade-20220921215522-10174 kubelet[2153]: E0921 21:57:31.387353 2153 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.229787 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2167]: E0921 21:57:32.141004 2167 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.230272 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2180]: E0921 21:57:32.874796 2180 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.230750 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:33 kubernetes-upgrade-20220921215522-10174 kubelet[2195]: E0921 21:57:33.642006 2195 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.231170 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:34 kubernetes-upgrade-20220921215522-10174 kubelet[2208]: E0921 21:57:34.374657 2208 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.231653 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2222]: E0921 21:57:35.147067 2222 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.232100 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.232511 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.232952 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.233360 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.233765 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.234168 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:39 kubernetes-upgrade-20220921215522-10174 kubelet[2436]: E0921 21:57:39.640823 2436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.234732 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:40 kubernetes-upgrade-20220921215522-10174 kubelet[2448]: E0921 21:57:40.380285 2448 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.235381 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2460]: E0921 21:57:41.157225 2460 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.236005 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2471]: E0921 21:57:41.891401 2471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.236638 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:42 kubernetes-upgrade-20220921215522-10174 kubelet[2482]: E0921 21:57:42.649592 2482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.237029 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:43 kubernetes-upgrade-20220921215522-10174 kubelet[2492]: E0921 21:57:43.370917 2492 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.237430 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2503]: E0921 21:57:44.139479 2503 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.237815 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2514]: E0921 21:57:44.889947 2514 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.238226 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:45 kubernetes-upgrade-20220921215522-10174 kubelet[2524]: E0921 21:57:45.634469 2524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.238621 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.239164 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.239665 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.240211 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.240602 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.241031 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2729]: E0921 21:57:50.127167 2729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.241448 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2739]: E0921 21:57:50.874508 2739 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.241850 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:51 kubernetes-upgrade-20220921215522-10174 kubelet[2750]: E0921 21:57:51.642311 2750 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.242286 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:52 kubernetes-upgrade-20220921215522-10174 kubelet[2760]: E0921 21:57:52.371286 2760 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.242723 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2772]: E0921 21:57:53.128949 2772 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.243184 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2782]: E0921 21:57:53.875177 2782 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.243632 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:54 kubernetes-upgrade-20220921215522-10174 kubelet[2792]: E0921 21:57:54.628094 2792 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.244173 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:55 kubernetes-upgrade-20220921215522-10174 kubelet[2803]: E0921 21:57:55.390864 2803 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.244743 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2813]: E0921 21:57:56.138948 2813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.245166 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.245573 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.245977 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.246388 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.246795 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:00.246949 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:00.246965 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:00.247072 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:00.247090 163433 out.go:239] Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.247100 163433 out.go:239] Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.247109 163433 out.go:239] Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.247125 163433 out.go:239] Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:00.247132 163433 out.go:239] Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:00.247138 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:00.247151 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:58:10.248903 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:58:10.363021 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:58:10.363102 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:58:10.394069 163433 cri.go:87] found id: ""
I0921 21:58:10.394099 163433 logs.go:274] 0 containers: []
W0921 21:58:10.394108 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:58:10.394115 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:58:10.394166 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:58:10.424049 163433 cri.go:87] found id: ""
I0921 21:58:10.424080 163433 logs.go:274] 0 containers: []
W0921 21:58:10.424089 163433 logs.go:276] No container was found matching "etcd"
I0921 21:58:10.424097 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:58:10.424148 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:58:10.447255 163433 cri.go:87] found id: ""
I0921 21:58:10.447278 163433 logs.go:274] 0 containers: []
W0921 21:58:10.447285 163433 logs.go:276] No container was found matching "coredns"
I0921 21:58:10.447290 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:58:10.447340 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:58:10.470403 163433 cri.go:87] found id: ""
I0921 21:58:10.470427 163433 logs.go:274] 0 containers: []
W0921 21:58:10.470433 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:58:10.470439 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:58:10.470490 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:58:10.496364 163433 cri.go:87] found id: ""
I0921 21:58:10.496393 163433 logs.go:274] 0 containers: []
W0921 21:58:10.496402 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:58:10.496409 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:58:10.496468 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:10.520943 163433 cri.go:87] found id: ""
I0921 21:58:10.520969 163433 logs.go:274] 0 containers: []
W0921 21:58:10.520979 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:10.520987 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:10.521056 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:10.543829 163433 cri.go:87] found id: ""
I0921 21:58:10.543852 163433 logs.go:274] 0 containers: []
W0921 21:58:10.543858 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:10.543863 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:10.543921 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:10.566214 163433 cri.go:87] found id: ""
I0921 21:58:10.566239 163433 logs.go:274] 0 containers: []
W0921 21:58:10.566245 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:10.566253 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:10.566266 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:10.622778 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:10.622815 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:10.650026 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:10.650052 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:10.665443 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:20 kubernetes-upgrade-20220921215522-10174 kubelet[1964]: E0921 21:57:20.874735 1964 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.665852 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:21 kubernetes-upgrade-20220921215522-10174 kubelet[1980]: E0921 21:57:21.642629 1980 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.666243 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:22 kubernetes-upgrade-20220921215522-10174 kubelet[1993]: E0921 21:57:22.383938 1993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.666643 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2007]: E0921 21:57:23.163080 2007 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.667035 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:23 kubernetes-upgrade-20220921215522-10174 kubelet[2019]: E0921 21:57:23.898283 2019 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.667426 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:24 kubernetes-upgrade-20220921215522-10174 kubelet[2034]: E0921 21:57:24.644635 2034 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.667884 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:25 kubernetes-upgrade-20220921215522-10174 kubelet[2048]: E0921 21:57:25.386236 2048 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.668282 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2062]: E0921 21:57:26.138653 2062 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.668674 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:26 kubernetes-upgrade-20220921215522-10174 kubelet[2074]: E0921 21:57:26.881606 2074 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.669060 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:27 kubernetes-upgrade-20220921215522-10174 kubelet[2089]: E0921 21:57:27.634384 2089 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.669471 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:28 kubernetes-upgrade-20220921215522-10174 kubelet[2101]: E0921 21:57:28.380595 2101 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.669870 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2115]: E0921 21:57:29.141993 2115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.670268 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:29 kubernetes-upgrade-20220921215522-10174 kubelet[2127]: E0921 21:57:29.881873 2127 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.670671 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:30 kubernetes-upgrade-20220921215522-10174 kubelet[2141]: E0921 21:57:30.643982 2141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.671089 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:31 kubernetes-upgrade-20220921215522-10174 kubelet[2153]: E0921 21:57:31.387353 2153 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.671487 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2167]: E0921 21:57:32.141004 2167 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.671934 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2180]: E0921 21:57:32.874796 2180 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.672327 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:33 kubernetes-upgrade-20220921215522-10174 kubelet[2195]: E0921 21:57:33.642006 2195 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.672720 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:34 kubernetes-upgrade-20220921215522-10174 kubelet[2208]: E0921 21:57:34.374657 2208 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.673110 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2222]: E0921 21:57:35.147067 2222 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.673504 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.673893 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.674282 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.674678 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.675065 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.675464 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:39 kubernetes-upgrade-20220921215522-10174 kubelet[2436]: E0921 21:57:39.640823 2436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.675898 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:40 kubernetes-upgrade-20220921215522-10174 kubelet[2448]: E0921 21:57:40.380285 2448 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.676285 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2460]: E0921 21:57:41.157225 2460 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.676708 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2471]: E0921 21:57:41.891401 2471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.677234 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:42 kubernetes-upgrade-20220921215522-10174 kubelet[2482]: E0921 21:57:42.649592 2482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.677705 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:43 kubernetes-upgrade-20220921215522-10174 kubelet[2492]: E0921 21:57:43.370917 2492 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.678149 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2503]: E0921 21:57:44.139479 2503 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.678587 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2514]: E0921 21:57:44.889947 2514 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.679004 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:45 kubernetes-upgrade-20220921215522-10174 kubelet[2524]: E0921 21:57:45.634469 2524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.679516 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.680040 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.680497 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.680942 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.681398 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.681829 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2729]: E0921 21:57:50.127167 2729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.682256 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2739]: E0921 21:57:50.874508 2739 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.682678 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:51 kubernetes-upgrade-20220921215522-10174 kubelet[2750]: E0921 21:57:51.642311 2750 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.683099 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:52 kubernetes-upgrade-20220921215522-10174 kubelet[2760]: E0921 21:57:52.371286 2760 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.683540 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2772]: E0921 21:57:53.128949 2772 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.684084 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2782]: E0921 21:57:53.875177 2782 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.684556 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:54 kubernetes-upgrade-20220921215522-10174 kubelet[2792]: E0921 21:57:54.628094 2792 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.684987 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:55 kubernetes-upgrade-20220921215522-10174 kubelet[2803]: E0921 21:57:55.390864 2803 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.685414 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2813]: E0921 21:57:56.138948 2813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.685850 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.686279 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.686696 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.687144 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.687615 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.688097 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:00 kubernetes-upgrade-20220921215522-10174 kubelet[3015]: E0921 21:58:00.621495 3015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.688558 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:01 kubernetes-upgrade-20220921215522-10174 kubelet[3026]: E0921 21:58:01.373196 3026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.689049 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3037]: E0921 21:58:02.136352 3037 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.689528 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3049]: E0921 21:58:02.882321 3049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.689993 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:03 kubernetes-upgrade-20220921215522-10174 kubelet[3060]: E0921 21:58:03.625330 3060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.690434 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:04 kubernetes-upgrade-20220921215522-10174 kubelet[3072]: E0921 21:58:04.375601 3072 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.690857 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3083]: E0921 21:58:05.130018 3083 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.691277 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3094]: E0921 21:58:05.880978 3094 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.691731 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:06 kubernetes-upgrade-20220921215522-10174 kubelet[3105]: E0921 21:58:06.623228 3105 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.692176 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.692601 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.693032 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.693498 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.693960 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:10.694155 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:10.694173 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:10.712179 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:10.712211 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:10.771703 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:10.771772 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:10.771801 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:10.771943 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:10.771966 163433 out.go:239] Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.771980 163433 out.go:239] Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.771992 163433 out.go:239] Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.772002 163433 out.go:239] Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:10.772014 163433 out.go:239] Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:10.772024 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:10.772031 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:58:20.773183 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:58:20.863377 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:58:20.863453 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:58:20.892562 163433 cri.go:87] found id: ""
I0921 21:58:20.892588 163433 logs.go:274] 0 containers: []
W0921 21:58:20.892596 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:58:20.892605 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:58:20.892662 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:58:20.921367 163433 cri.go:87] found id: ""
I0921 21:58:20.921397 163433 logs.go:274] 0 containers: []
W0921 21:58:20.921406 163433 logs.go:276] No container was found matching "etcd"
I0921 21:58:20.921414 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:58:20.921466 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:58:20.953836 163433 cri.go:87] found id: ""
I0921 21:58:20.953863 163433 logs.go:274] 0 containers: []
W0921 21:58:20.953869 163433 logs.go:276] No container was found matching "coredns"
I0921 21:58:20.953875 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:58:20.953921 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:58:21.048920 163433 cri.go:87] found id: ""
I0921 21:58:21.048955 163433 logs.go:274] 0 containers: []
W0921 21:58:21.048961 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:58:21.048967 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:58:21.049006 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:58:21.085190 163433 cri.go:87] found id: ""
I0921 21:58:21.085216 163433 logs.go:274] 0 containers: []
W0921 21:58:21.085224 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:58:21.085232 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:58:21.085286 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:21.125244 163433 cri.go:87] found id: ""
I0921 21:58:21.125266 163433 logs.go:274] 0 containers: []
W0921 21:58:21.125274 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:21.125282 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:21.125328 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:21.152179 163433 cri.go:87] found id: ""
I0921 21:58:21.152205 163433 logs.go:274] 0 containers: []
W0921 21:58:21.152214 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:21.152222 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:21.152274 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:21.237893 163433 cri.go:87] found id: ""
I0921 21:58:21.237922 163433 logs.go:274] 0 containers: []
W0921 21:58:21.237931 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:21.237942 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:21.237956 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:21.255938 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:31 kubernetes-upgrade-20220921215522-10174 kubelet[2153]: E0921 21:57:31.387353 2153 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.256376 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2167]: E0921 21:57:32.141004 2167 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.256772 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:32 kubernetes-upgrade-20220921215522-10174 kubelet[2180]: E0921 21:57:32.874796 2180 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.257173 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:33 kubernetes-upgrade-20220921215522-10174 kubelet[2195]: E0921 21:57:33.642006 2195 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.257568 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:34 kubernetes-upgrade-20220921215522-10174 kubelet[2208]: E0921 21:57:34.374657 2208 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.257966 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2222]: E0921 21:57:35.147067 2222 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.258397 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:35 kubernetes-upgrade-20220921215522-10174 kubelet[2235]: E0921 21:57:35.879490 2235 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.258892 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:36 kubernetes-upgrade-20220921215522-10174 kubelet[2250]: E0921 21:57:36.640238 2250 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.259328 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:37 kubernetes-upgrade-20220921215522-10174 kubelet[2263]: E0921 21:57:37.374557 2263 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.259761 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2278]: E0921 21:57:38.141807 2278 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.260160 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:38 kubernetes-upgrade-20220921215522-10174 kubelet[2290]: E0921 21:57:38.876371 2290 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.260597 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:39 kubernetes-upgrade-20220921215522-10174 kubelet[2436]: E0921 21:57:39.640823 2436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.260999 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:40 kubernetes-upgrade-20220921215522-10174 kubelet[2448]: E0921 21:57:40.380285 2448 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.261412 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2460]: E0921 21:57:41.157225 2460 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.261808 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:41 kubernetes-upgrade-20220921215522-10174 kubelet[2471]: E0921 21:57:41.891401 2471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.262191 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:42 kubernetes-upgrade-20220921215522-10174 kubelet[2482]: E0921 21:57:42.649592 2482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.262585 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:43 kubernetes-upgrade-20220921215522-10174 kubelet[2492]: E0921 21:57:43.370917 2492 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.262967 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2503]: E0921 21:57:44.139479 2503 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.263341 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2514]: E0921 21:57:44.889947 2514 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.263802 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:45 kubernetes-upgrade-20220921215522-10174 kubelet[2524]: E0921 21:57:45.634469 2524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.264194 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.264734 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.265220 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.265686 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.266116 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.266520 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2729]: E0921 21:57:50.127167 2729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.266944 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2739]: E0921 21:57:50.874508 2739 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.267349 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:51 kubernetes-upgrade-20220921215522-10174 kubelet[2750]: E0921 21:57:51.642311 2750 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.267761 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:52 kubernetes-upgrade-20220921215522-10174 kubelet[2760]: E0921 21:57:52.371286 2760 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.268160 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2772]: E0921 21:57:53.128949 2772 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.268563 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2782]: E0921 21:57:53.875177 2782 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.268961 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:54 kubernetes-upgrade-20220921215522-10174 kubelet[2792]: E0921 21:57:54.628094 2792 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.269375 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:55 kubernetes-upgrade-20220921215522-10174 kubelet[2803]: E0921 21:57:55.390864 2803 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.269771 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2813]: E0921 21:57:56.138948 2813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.270186 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.270603 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.271006 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.271516 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.272015 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.272451 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:00 kubernetes-upgrade-20220921215522-10174 kubelet[3015]: E0921 21:58:00.621495 3015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.272887 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:01 kubernetes-upgrade-20220921215522-10174 kubelet[3026]: E0921 21:58:01.373196 3026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.273291 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3037]: E0921 21:58:02.136352 3037 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.273737 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3049]: E0921 21:58:02.882321 3049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.274371 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:03 kubernetes-upgrade-20220921215522-10174 kubelet[3060]: E0921 21:58:03.625330 3060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.274979 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:04 kubernetes-upgrade-20220921215522-10174 kubelet[3072]: E0921 21:58:04.375601 3072 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.275439 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3083]: E0921 21:58:05.130018 3083 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.275905 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3094]: E0921 21:58:05.880978 3094 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.276359 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:06 kubernetes-upgrade-20220921215522-10174 kubelet[3105]: E0921 21:58:06.623228 3105 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.276764 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.277183 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.277592 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.278030 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.278446 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.278873 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3306]: E0921 21:58:11.134783 3306 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.279345 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3318]: E0921 21:58:11.875201 3318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.279747 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:12 kubernetes-upgrade-20220921215522-10174 kubelet[3329]: E0921 21:58:12.632869 3329 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.280134 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:13 kubernetes-upgrade-20220921215522-10174 kubelet[3341]: E0921 21:58:13.374875 3341 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.280521 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3353]: E0921 21:58:14.123030 3353 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.280910 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3364]: E0921 21:58:14.876732 3364 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.281307 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:15 kubernetes-upgrade-20220921215522-10174 kubelet[3374]: E0921 21:58:15.624588 3374 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.281773 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:16 kubernetes-upgrade-20220921215522-10174 kubelet[3385]: E0921 21:58:16.371393 3385 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.282157 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3396]: E0921 21:58:17.137955 3396 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.282572 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.282960 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.283345 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.283764 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.284178 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:21.284338 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:21.284357 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:21.299587 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:21.299614 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:21.360540 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:21.360565 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:21.360579 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:21.397491 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:21.397523 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:21.427754 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:21.427779 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:21.427901 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:21.427930 163433 out.go:239] Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.427938 163433 out.go:239] Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.427945 163433 out.go:239] Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.427953 163433 out.go:239] Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:21.427963 163433 out.go:239] Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:21.427969 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:21.427980 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:58:31.430050 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:58:31.863473 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:58:31.863538 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:58:31.888162 163433 cri.go:87] found id: ""
I0921 21:58:31.888184 163433 logs.go:274] 0 containers: []
W0921 21:58:31.888190 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:58:31.888195 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:58:31.888242 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:58:31.912569 163433 cri.go:87] found id: ""
I0921 21:58:31.912597 163433 logs.go:274] 0 containers: []
W0921 21:58:31.912606 163433 logs.go:276] No container was found matching "etcd"
I0921 21:58:31.912622 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:58:31.912678 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:58:31.937077 163433 cri.go:87] found id: ""
I0921 21:58:31.937101 163433 logs.go:274] 0 containers: []
W0921 21:58:31.937125 163433 logs.go:276] No container was found matching "coredns"
I0921 21:58:31.937133 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:58:31.937187 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:58:31.961115 163433 cri.go:87] found id: ""
I0921 21:58:31.961141 163433 logs.go:274] 0 containers: []
W0921 21:58:31.961150 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:58:31.961157 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:58:31.961220 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:58:31.984291 163433 cri.go:87] found id: ""
I0921 21:58:31.984313 163433 logs.go:274] 0 containers: []
W0921 21:58:31.984319 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:58:31.984325 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:58:31.984374 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:32.007119 163433 cri.go:87] found id: ""
I0921 21:58:32.007149 163433 logs.go:274] 0 containers: []
W0921 21:58:32.007158 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:32.007166 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:32.007227 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:32.030492 163433 cri.go:87] found id: ""
I0921 21:58:32.030525 163433 logs.go:274] 0 containers: []
W0921 21:58:32.030534 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:32.030543 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:32.030589 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:32.053566 163433 cri.go:87] found id: ""
I0921 21:58:32.053589 163433 logs.go:274] 0 containers: []
W0921 21:58:32.053597 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:32.053610 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:32.053623 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:32.098622 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:32.098659 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:32.127496 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:32.127535 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:32.145188 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:42 kubernetes-upgrade-20220921215522-10174 kubelet[2482]: E0921 21:57:42.649592 2482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.145597 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:43 kubernetes-upgrade-20220921215522-10174 kubelet[2492]: E0921 21:57:43.370917 2492 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.146009 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2503]: E0921 21:57:44.139479 2503 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.146402 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:44 kubernetes-upgrade-20220921215522-10174 kubelet[2514]: E0921 21:57:44.889947 2514 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.146792 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:45 kubernetes-upgrade-20220921215522-10174 kubelet[2524]: E0921 21:57:45.634469 2524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.147190 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:46 kubernetes-upgrade-20220921215522-10174 kubelet[2536]: E0921 21:57:46.371547 2536 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.147585 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2547]: E0921 21:57:47.146381 2547 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.148009 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:47 kubernetes-upgrade-20220921215522-10174 kubelet[2558]: E0921 21:57:47.871171 2558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.148408 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:48 kubernetes-upgrade-20220921215522-10174 kubelet[2569]: E0921 21:57:48.647840 2569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.148800 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:49 kubernetes-upgrade-20220921215522-10174 kubelet[2579]: E0921 21:57:49.373797 2579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.149205 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2729]: E0921 21:57:50.127167 2729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.149616 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:50 kubernetes-upgrade-20220921215522-10174 kubelet[2739]: E0921 21:57:50.874508 2739 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.150029 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:51 kubernetes-upgrade-20220921215522-10174 kubelet[2750]: E0921 21:57:51.642311 2750 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.150421 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:52 kubernetes-upgrade-20220921215522-10174 kubelet[2760]: E0921 21:57:52.371286 2760 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.150832 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2772]: E0921 21:57:53.128949 2772 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.151226 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2782]: E0921 21:57:53.875177 2782 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.151618 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:54 kubernetes-upgrade-20220921215522-10174 kubelet[2792]: E0921 21:57:54.628094 2792 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.152030 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:55 kubernetes-upgrade-20220921215522-10174 kubelet[2803]: E0921 21:57:55.390864 2803 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.152433 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2813]: E0921 21:57:56.138948 2813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.152829 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.153228 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.153644 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.154041 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.154440 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.154840 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:00 kubernetes-upgrade-20220921215522-10174 kubelet[3015]: E0921 21:58:00.621495 3015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.155238 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:01 kubernetes-upgrade-20220921215522-10174 kubelet[3026]: E0921 21:58:01.373196 3026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.155635 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3037]: E0921 21:58:02.136352 3037 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.156066 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3049]: E0921 21:58:02.882321 3049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.156460 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:03 kubernetes-upgrade-20220921215522-10174 kubelet[3060]: E0921 21:58:03.625330 3060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.156845 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:04 kubernetes-upgrade-20220921215522-10174 kubelet[3072]: E0921 21:58:04.375601 3072 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.157254 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3083]: E0921 21:58:05.130018 3083 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.157653 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3094]: E0921 21:58:05.880978 3094 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.158046 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:06 kubernetes-upgrade-20220921215522-10174 kubelet[3105]: E0921 21:58:06.623228 3105 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.158484 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.158872 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.159270 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.159658 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.160073 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.160476 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3306]: E0921 21:58:11.134783 3306 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.160870 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3318]: E0921 21:58:11.875201 3318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.161267 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:12 kubernetes-upgrade-20220921215522-10174 kubelet[3329]: E0921 21:58:12.632869 3329 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.161660 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:13 kubernetes-upgrade-20220921215522-10174 kubelet[3341]: E0921 21:58:13.374875 3341 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.162080 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3353]: E0921 21:58:14.123030 3353 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.162478 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3364]: E0921 21:58:14.876732 3364 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.162872 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:15 kubernetes-upgrade-20220921215522-10174 kubelet[3374]: E0921 21:58:15.624588 3374 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.163270 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:16 kubernetes-upgrade-20220921215522-10174 kubelet[3385]: E0921 21:58:16.371393 3385 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.163666 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3396]: E0921 21:58:17.137955 3396 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.164087 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.164481 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.164876 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.165276 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.165667 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.166060 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:21 kubernetes-upgrade-20220921215522-10174 kubelet[3597]: E0921 21:58:21.630652 3597 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.166455 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:22 kubernetes-upgrade-20220921215522-10174 kubelet[3607]: E0921 21:58:22.370329 3607 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.166863 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3618]: E0921 21:58:23.123363 3618 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.167267 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3629]: E0921 21:58:23.871937 3629 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.167666 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:24 kubernetes-upgrade-20220921215522-10174 kubelet[3640]: E0921 21:58:24.623599 3640 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.168082 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:25 kubernetes-upgrade-20220921215522-10174 kubelet[3651]: E0921 21:58:25.373329 3651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.168479 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3664]: E0921 21:58:26.122744 3664 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.168864 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3675]: E0921 21:58:26.871823 3675 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.169268 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:27 kubernetes-upgrade-20220921215522-10174 kubelet[3686]: E0921 21:58:27.622585 3686 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.169672 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:28 kubernetes-upgrade-20220921215522-10174 kubelet[3697]: E0921 21:58:28.376468 3697 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.170076 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.170479 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.170907 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.171314 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.171781 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:32.171915 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:32.171930 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:32.186117 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:32.186146 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:32.240672 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:32.240699 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:32.240713 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:32.240825 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:32.240839 163433 out.go:239] Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.240846 163433 out.go:239] Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.240854 163433 out.go:239] Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.240862 163433 out.go:239] Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:32.240874 163433 out.go:239] Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:32.240883 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:32.240892 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:58:42.241531 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:58:42.363477 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:58:42.363559 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:58:42.387547 163433 cri.go:87] found id: ""
I0921 21:58:42.387571 163433 logs.go:274] 0 containers: []
W0921 21:58:42.387578 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:58:42.387585 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:58:42.387639 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:58:42.409893 163433 cri.go:87] found id: ""
I0921 21:58:42.409924 163433 logs.go:274] 0 containers: []
W0921 21:58:42.409933 163433 logs.go:276] No container was found matching "etcd"
I0921 21:58:42.409940 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:58:42.409985 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:58:42.431936 163433 cri.go:87] found id: ""
I0921 21:58:42.431959 163433 logs.go:274] 0 containers: []
W0921 21:58:42.431974 163433 logs.go:276] No container was found matching "coredns"
I0921 21:58:42.431981 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:58:42.432034 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:58:42.454563 163433 cri.go:87] found id: ""
I0921 21:58:42.454587 163433 logs.go:274] 0 containers: []
W0921 21:58:42.454593 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:58:42.454598 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:58:42.454638 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:58:42.477095 163433 cri.go:87] found id: ""
I0921 21:58:42.477120 163433 logs.go:274] 0 containers: []
W0921 21:58:42.477127 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:58:42.477146 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:58:42.477202 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:42.501982 163433 cri.go:87] found id: ""
I0921 21:58:42.502006 163433 logs.go:274] 0 containers: []
W0921 21:58:42.502012 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:42.502018 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:42.502064 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:42.526071 163433 cri.go:87] found id: ""
I0921 21:58:42.526102 163433 logs.go:274] 0 containers: []
W0921 21:58:42.526110 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:42.526119 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:42.526173 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:42.550511 163433 cri.go:87] found id: ""
I0921 21:58:42.550541 163433 logs.go:274] 0 containers: []
W0921 21:58:42.550550 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:42.550562 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:42.550577 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:42.566357 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:42.566389 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:42.630777 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:42.630804 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:42.630816 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:42.665525 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:42.665563 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:42.696286 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:42.696323 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:42.714032 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2772]: E0921 21:57:53.128949 2772 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.714436 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:53 kubernetes-upgrade-20220921215522-10174 kubelet[2782]: E0921 21:57:53.875177 2782 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.714827 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:54 kubernetes-upgrade-20220921215522-10174 kubelet[2792]: E0921 21:57:54.628094 2792 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.715228 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:55 kubernetes-upgrade-20220921215522-10174 kubelet[2803]: E0921 21:57:55.390864 2803 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.715609 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2813]: E0921 21:57:56.138948 2813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.716027 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:56 kubernetes-upgrade-20220921215522-10174 kubelet[2823]: E0921 21:57:56.875520 2823 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.716418 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:57 kubernetes-upgrade-20220921215522-10174 kubelet[2834]: E0921 21:57:57.636715 2834 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.716798 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:58 kubernetes-upgrade-20220921215522-10174 kubelet[2844]: E0921 21:57:58.375773 2844 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.717186 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2855]: E0921 21:57:59.130098 2855 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.717588 163433 logs.go:138] Found kubelet problem: Sep 21 21:57:59 kubernetes-upgrade-20220921215522-10174 kubelet[2868]: E0921 21:57:59.873878 2868 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.717969 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:00 kubernetes-upgrade-20220921215522-10174 kubelet[3015]: E0921 21:58:00.621495 3015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.718352 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:01 kubernetes-upgrade-20220921215522-10174 kubelet[3026]: E0921 21:58:01.373196 3026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.718787 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3037]: E0921 21:58:02.136352 3037 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.719182 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3049]: E0921 21:58:02.882321 3049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.719557 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:03 kubernetes-upgrade-20220921215522-10174 kubelet[3060]: E0921 21:58:03.625330 3060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.719955 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:04 kubernetes-upgrade-20220921215522-10174 kubelet[3072]: E0921 21:58:04.375601 3072 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.720348 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3083]: E0921 21:58:05.130018 3083 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.720748 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3094]: E0921 21:58:05.880978 3094 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.721130 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:06 kubernetes-upgrade-20220921215522-10174 kubelet[3105]: E0921 21:58:06.623228 3105 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.721516 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.721937 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.722329 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.722705 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.723095 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.723477 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3306]: E0921 21:58:11.134783 3306 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.723889 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3318]: E0921 21:58:11.875201 3318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.724274 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:12 kubernetes-upgrade-20220921215522-10174 kubelet[3329]: E0921 21:58:12.632869 3329 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.724660 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:13 kubernetes-upgrade-20220921215522-10174 kubelet[3341]: E0921 21:58:13.374875 3341 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.725045 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3353]: E0921 21:58:14.123030 3353 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.725433 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3364]: E0921 21:58:14.876732 3364 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.725818 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:15 kubernetes-upgrade-20220921215522-10174 kubelet[3374]: E0921 21:58:15.624588 3374 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.726211 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:16 kubernetes-upgrade-20220921215522-10174 kubelet[3385]: E0921 21:58:16.371393 3385 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.726594 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3396]: E0921 21:58:17.137955 3396 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.726972 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.727353 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.727760 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.728147 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.728533 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.728909 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:21 kubernetes-upgrade-20220921215522-10174 kubelet[3597]: E0921 21:58:21.630652 3597 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.729305 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:22 kubernetes-upgrade-20220921215522-10174 kubelet[3607]: E0921 21:58:22.370329 3607 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.729683 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3618]: E0921 21:58:23.123363 3618 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.730060 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3629]: E0921 21:58:23.871937 3629 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.730449 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:24 kubernetes-upgrade-20220921215522-10174 kubelet[3640]: E0921 21:58:24.623599 3640 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.730842 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:25 kubernetes-upgrade-20220921215522-10174 kubelet[3651]: E0921 21:58:25.373329 3651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.731234 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3664]: E0921 21:58:26.122744 3664 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.731610 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3675]: E0921 21:58:26.871823 3675 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.732034 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:27 kubernetes-upgrade-20220921215522-10174 kubelet[3686]: E0921 21:58:27.622585 3686 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.732425 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:28 kubernetes-upgrade-20220921215522-10174 kubelet[3697]: E0921 21:58:28.376468 3697 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.732803 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.733184 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.733562 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.733937 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.734326 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.734711 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3898]: E0921 21:58:32.873032 3898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.735106 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:33 kubernetes-upgrade-20220921215522-10174 kubelet[3910]: E0921 21:58:33.628639 3910 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.735483 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:34 kubernetes-upgrade-20220921215522-10174 kubelet[3921]: E0921 21:58:34.371911 3921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.735920 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3932]: E0921 21:58:35.133679 3932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.736318 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3942]: E0921 21:58:35.894996 3942 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.736696 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:36 kubernetes-upgrade-20220921215522-10174 kubelet[3952]: E0921 21:58:36.645679 3952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.737112 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:37 kubernetes-upgrade-20220921215522-10174 kubelet[3962]: E0921 21:58:37.382599 3962 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.737513 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3972]: E0921 21:58:38.148834 3972 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.737892 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3982]: E0921 21:58:38.878739 3982 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.738275 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.738655 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.739036 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.739421 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.739825 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:42.739964 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:42.739978 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:42.740075 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:42.740093 163433 out.go:239] Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.740099 163433 out.go:239] Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.740104 163433 out.go:239] Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.740109 163433 out.go:239] Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:42.740112 163433 out.go:239] Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:42.740117 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:42.740127 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:58:52.740913 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:58:52.863793 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:58:52.863871 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:58:52.889997 163433 cri.go:87] found id: ""
I0921 21:58:52.890021 163433 logs.go:274] 0 containers: []
W0921 21:58:52.890028 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:58:52.890033 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:58:52.890087 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:58:52.914624 163433 cri.go:87] found id: ""
I0921 21:58:52.914652 163433 logs.go:274] 0 containers: []
W0921 21:58:52.914658 163433 logs.go:276] No container was found matching "etcd"
I0921 21:58:52.914664 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:58:52.914718 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:58:52.937740 163433 cri.go:87] found id: ""
I0921 21:58:52.937766 163433 logs.go:274] 0 containers: []
W0921 21:58:52.937772 163433 logs.go:276] No container was found matching "coredns"
I0921 21:58:52.937778 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:58:52.937852 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:58:52.962432 163433 cri.go:87] found id: ""
I0921 21:58:52.962460 163433 logs.go:274] 0 containers: []
W0921 21:58:52.962468 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:58:52.962475 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:58:52.962528 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:58:52.986468 163433 cri.go:87] found id: ""
I0921 21:58:52.986496 163433 logs.go:274] 0 containers: []
W0921 21:58:52.986502 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:58:52.986508 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:58:52.986560 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:58:53.009885 163433 cri.go:87] found id: ""
I0921 21:58:53.009914 163433 logs.go:274] 0 containers: []
W0921 21:58:53.009921 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:58:53.009927 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:58:53.009976 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:58:53.034075 163433 cri.go:87] found id: ""
I0921 21:58:53.034105 163433 logs.go:274] 0 containers: []
W0921 21:58:53.034115 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:58:53.034123 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:58:53.034173 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:58:53.056921 163433 cri.go:87] found id: ""
I0921 21:58:53.056945 163433 logs.go:274] 0 containers: []
W0921 21:58:53.056954 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:58:53.056966 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:58:53.056980 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:58:53.073625 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:02 kubernetes-upgrade-20220921215522-10174 kubelet[3049]: E0921 21:58:02.882321 3049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.074250 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:03 kubernetes-upgrade-20220921215522-10174 kubelet[3060]: E0921 21:58:03.625330 3060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.074937 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:04 kubernetes-upgrade-20220921215522-10174 kubelet[3072]: E0921 21:58:04.375601 3072 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.075567 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3083]: E0921 21:58:05.130018 3083 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.076166 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:05 kubernetes-upgrade-20220921215522-10174 kubelet[3094]: E0921 21:58:05.880978 3094 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.076609 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:06 kubernetes-upgrade-20220921215522-10174 kubelet[3105]: E0921 21:58:06.623228 3105 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.076993 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:07 kubernetes-upgrade-20220921215522-10174 kubelet[3118]: E0921 21:58:07.381913 3118 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.077377 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3130]: E0921 21:58:08.130002 3130 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.077751 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:08 kubernetes-upgrade-20220921215522-10174 kubelet[3141]: E0921 21:58:08.875549 3141 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.078127 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:09 kubernetes-upgrade-20220921215522-10174 kubelet[3151]: E0921 21:58:09.622166 3151 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.078510 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:10 kubernetes-upgrade-20220921215522-10174 kubelet[3164]: E0921 21:58:10.371341 3164 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.078894 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3306]: E0921 21:58:11.134783 3306 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.079283 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:11 kubernetes-upgrade-20220921215522-10174 kubelet[3318]: E0921 21:58:11.875201 3318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.079936 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:12 kubernetes-upgrade-20220921215522-10174 kubelet[3329]: E0921 21:58:12.632869 3329 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.080383 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:13 kubernetes-upgrade-20220921215522-10174 kubelet[3341]: E0921 21:58:13.374875 3341 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.080786 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3353]: E0921 21:58:14.123030 3353 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.081167 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3364]: E0921 21:58:14.876732 3364 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.081688 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:15 kubernetes-upgrade-20220921215522-10174 kubelet[3374]: E0921 21:58:15.624588 3374 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.082326 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:16 kubernetes-upgrade-20220921215522-10174 kubelet[3385]: E0921 21:58:16.371393 3385 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.082962 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3396]: E0921 21:58:17.137955 3396 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.083584 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.084102 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.084506 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.085070 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.085475 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.085860 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:21 kubernetes-upgrade-20220921215522-10174 kubelet[3597]: E0921 21:58:21.630652 3597 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.086235 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:22 kubernetes-upgrade-20220921215522-10174 kubelet[3607]: E0921 21:58:22.370329 3607 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.086633 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3618]: E0921 21:58:23.123363 3618 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.087007 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3629]: E0921 21:58:23.871937 3629 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.087393 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:24 kubernetes-upgrade-20220921215522-10174 kubelet[3640]: E0921 21:58:24.623599 3640 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.087792 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:25 kubernetes-upgrade-20220921215522-10174 kubelet[3651]: E0921 21:58:25.373329 3651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.088170 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3664]: E0921 21:58:26.122744 3664 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.088550 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3675]: E0921 21:58:26.871823 3675 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.088931 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:27 kubernetes-upgrade-20220921215522-10174 kubelet[3686]: E0921 21:58:27.622585 3686 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.089338 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:28 kubernetes-upgrade-20220921215522-10174 kubelet[3697]: E0921 21:58:28.376468 3697 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.089724 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.090099 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.090506 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.090889 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.091266 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.091825 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3898]: E0921 21:58:32.873032 3898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.092278 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:33 kubernetes-upgrade-20220921215522-10174 kubelet[3910]: E0921 21:58:33.628639 3910 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.092705 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:34 kubernetes-upgrade-20220921215522-10174 kubelet[3921]: E0921 21:58:34.371911 3921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.093102 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3932]: E0921 21:58:35.133679 3932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.093497 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3942]: E0921 21:58:35.894996 3942 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.093937 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:36 kubernetes-upgrade-20220921215522-10174 kubelet[3952]: E0921 21:58:36.645679 3952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.094354 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:37 kubernetes-upgrade-20220921215522-10174 kubelet[3962]: E0921 21:58:37.382599 3962 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.094757 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3972]: E0921 21:58:38.148834 3972 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.095151 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3982]: E0921 21:58:38.878739 3982 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.095711 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.096365 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.096812 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.097192 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.097683 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.098241 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:43 kubernetes-upgrade-20220921215522-10174 kubelet[4185]: E0921 21:58:43.372417 4185 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.098718 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4197]: E0921 21:58:44.122124 4197 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.099191 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4209]: E0921 21:58:44.872708 4209 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.099680 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:45 kubernetes-upgrade-20220921215522-10174 kubelet[4219]: E0921 21:58:45.620684 4219 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.100088 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:46 kubernetes-upgrade-20220921215522-10174 kubelet[4230]: E0921 21:58:46.371087 4230 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.100522 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4243]: E0921 21:58:47.121413 4243 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.100925 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4254]: E0921 21:58:47.872018 4254 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.101326 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:48 kubernetes-upgrade-20220921215522-10174 kubelet[4264]: E0921 21:58:48.621704 4264 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.101721 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.102110 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.102507 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.103084 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.103563 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:53.103713 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:58:53.103744 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:58:53.119036 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:58:53.119064 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:58:53.174025 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:58:53.174054 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:58:53.174069 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:58:53.212843 163433 logs.go:123] Gathering logs for container status ...
I0921 21:58:53.212877 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:58:53.244885 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:53.244915 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:58:53.245034 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:58:53.245050 163433 out.go:239] Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.245060 163433 out.go:239] Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.245072 163433 out.go:239] Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.245079 163433 out.go:239] Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:58:53.245094 163433 out.go:239] Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:58:53.245105 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:58:53.245133 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:03.245553 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:03.363405 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:03.363497 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:03.387969 163433 cri.go:87] found id: ""
I0921 21:59:03.388005 163433 logs.go:274] 0 containers: []
W0921 21:59:03.388017 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:03.388025 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:03.388086 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:03.411104 163433 cri.go:87] found id: ""
I0921 21:59:03.411137 163433 logs.go:274] 0 containers: []
W0921 21:59:03.411144 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:03.411149 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:03.411194 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:03.435637 163433 cri.go:87] found id: ""
I0921 21:59:03.435667 163433 logs.go:274] 0 containers: []
W0921 21:59:03.435676 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:03.435684 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:03.435816 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:03.460863 163433 cri.go:87] found id: ""
I0921 21:59:03.460896 163433 logs.go:274] 0 containers: []
W0921 21:59:03.460903 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:03.460909 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:03.460961 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:03.486244 163433 cri.go:87] found id: ""
I0921 21:59:03.486273 163433 logs.go:274] 0 containers: []
W0921 21:59:03.486282 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:03.486290 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:03.486345 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:03.512376 163433 cri.go:87] found id: ""
I0921 21:59:03.512406 163433 logs.go:274] 0 containers: []
W0921 21:59:03.512415 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:03.512423 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:03.512477 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:03.540553 163433 cri.go:87] found id: ""
I0921 21:59:03.540585 163433 logs.go:274] 0 containers: []
W0921 21:59:03.540593 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:03.540601 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:03.540658 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:03.570035 163433 cri.go:87] found id: ""
I0921 21:59:03.570063 163433 logs.go:274] 0 containers: []
W0921 21:59:03.570074 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:03.570087 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:03.570102 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:03.602311 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:03.602345 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:03.623938 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3353]: E0921 21:58:14.123030 3353 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.624591 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:14 kubernetes-upgrade-20220921215522-10174 kubelet[3364]: E0921 21:58:14.876732 3364 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.625292 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:15 kubernetes-upgrade-20220921215522-10174 kubelet[3374]: E0921 21:58:15.624588 3374 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.625985 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:16 kubernetes-upgrade-20220921215522-10174 kubelet[3385]: E0921 21:58:16.371393 3385 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.626674 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3396]: E0921 21:58:17.137955 3396 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.627359 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:17 kubernetes-upgrade-20220921215522-10174 kubelet[3407]: E0921 21:58:17.890930 3407 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.628139 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:18 kubernetes-upgrade-20220921215522-10174 kubelet[3417]: E0921 21:58:18.632034 3417 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.628856 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:19 kubernetes-upgrade-20220921215522-10174 kubelet[3428]: E0921 21:58:19.375785 3428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.629550 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3439]: E0921 21:58:20.129184 3439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.630208 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:20 kubernetes-upgrade-20220921215522-10174 kubelet[3451]: E0921 21:58:20.880762 3451 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.630758 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:21 kubernetes-upgrade-20220921215522-10174 kubelet[3597]: E0921 21:58:21.630652 3597 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.631213 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:22 kubernetes-upgrade-20220921215522-10174 kubelet[3607]: E0921 21:58:22.370329 3607 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.631688 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3618]: E0921 21:58:23.123363 3618 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.632162 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:23 kubernetes-upgrade-20220921215522-10174 kubelet[3629]: E0921 21:58:23.871937 3629 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.632586 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:24 kubernetes-upgrade-20220921215522-10174 kubelet[3640]: E0921 21:58:24.623599 3640 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.633009 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:25 kubernetes-upgrade-20220921215522-10174 kubelet[3651]: E0921 21:58:25.373329 3651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.633558 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3664]: E0921 21:58:26.122744 3664 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.633977 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3675]: E0921 21:58:26.871823 3675 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.634384 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:27 kubernetes-upgrade-20220921215522-10174 kubelet[3686]: E0921 21:58:27.622585 3686 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.634790 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:28 kubernetes-upgrade-20220921215522-10174 kubelet[3697]: E0921 21:58:28.376468 3697 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.635192 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.635607 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.636180 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.636584 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.637080 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.637491 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3898]: E0921 21:58:32.873032 3898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.637903 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:33 kubernetes-upgrade-20220921215522-10174 kubelet[3910]: E0921 21:58:33.628639 3910 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.638538 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:34 kubernetes-upgrade-20220921215522-10174 kubelet[3921]: E0921 21:58:34.371911 3921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.639227 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3932]: E0921 21:58:35.133679 3932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.639993 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3942]: E0921 21:58:35.894996 3942 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.640608 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:36 kubernetes-upgrade-20220921215522-10174 kubelet[3952]: E0921 21:58:36.645679 3952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.641273 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:37 kubernetes-upgrade-20220921215522-10174 kubelet[3962]: E0921 21:58:37.382599 3962 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.641904 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3972]: E0921 21:58:38.148834 3972 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.642580 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3982]: E0921 21:58:38.878739 3982 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.643255 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.643980 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.644717 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.645453 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.646186 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.646881 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:43 kubernetes-upgrade-20220921215522-10174 kubelet[4185]: E0921 21:58:43.372417 4185 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.647585 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4197]: E0921 21:58:44.122124 4197 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.648274 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4209]: E0921 21:58:44.872708 4209 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.648983 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:45 kubernetes-upgrade-20220921215522-10174 kubelet[4219]: E0921 21:58:45.620684 4219 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.649729 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:46 kubernetes-upgrade-20220921215522-10174 kubelet[4230]: E0921 21:58:46.371087 4230 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.650412 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4243]: E0921 21:58:47.121413 4243 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.651088 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4254]: E0921 21:58:47.872018 4254 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.651735 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:48 kubernetes-upgrade-20220921215522-10174 kubelet[4264]: E0921 21:58:48.621704 4264 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.652208 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.652625 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.653051 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.653460 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.653922 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.654371 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4436]: E0921 21:58:53.129656 4436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.654790 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4480]: E0921 21:58:53.875326 4480 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.655199 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:54 kubernetes-upgrade-20220921215522-10174 kubelet[4491]: E0921 21:58:54.630207 4491 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.655596 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:55 kubernetes-upgrade-20220921215522-10174 kubelet[4505]: E0921 21:58:55.382351 4505 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.656025 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4515]: E0921 21:58:56.132997 4515 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.656425 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4524]: E0921 21:58:56.895570 4524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.656842 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:57 kubernetes-upgrade-20220921215522-10174 kubelet[4533]: E0921 21:58:57.642639 4533 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.657313 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:58 kubernetes-upgrade-20220921215522-10174 kubelet[4545]: E0921 21:58:58.392797 4545 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.657706 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4554]: E0921 21:58:59.134963 4554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.658136 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.658687 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.659307 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.659938 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.660598 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:03.661101 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:03.661121 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:03.684737 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:03.684833 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:03.774500 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:03.774530 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:03.774542 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:03.826613 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:03.826652 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:03.826794 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:03.826811 163433 out.go:239] Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.826822 163433 out.go:239] Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.826833 163433 out.go:239] Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.826842 163433 out.go:239] Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:03.826854 163433 out.go:239] Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:03.826864 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:03.826871 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:13.827459 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:13.862951 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:13.863043 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:13.889021 163433 cri.go:87] found id: ""
I0921 21:59:13.889049 163433 logs.go:274] 0 containers: []
W0921 21:59:13.889059 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:13.889068 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:13.889132 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:13.917766 163433 cri.go:87] found id: ""
I0921 21:59:13.917797 163433 logs.go:274] 0 containers: []
W0921 21:59:13.917806 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:13.917813 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:13.917869 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:13.944016 163433 cri.go:87] found id: ""
I0921 21:59:13.944044 163433 logs.go:274] 0 containers: []
W0921 21:59:13.944051 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:13.944057 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:13.944109 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:13.997324 163433 cri.go:87] found id: ""
I0921 21:59:13.997351 163433 logs.go:274] 0 containers: []
W0921 21:59:13.997357 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:13.997362 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:13.997412 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:14.029255 163433 cri.go:87] found id: ""
I0921 21:59:14.029281 163433 logs.go:274] 0 containers: []
W0921 21:59:14.029290 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:14.029297 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:14.029353 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:14.058305 163433 cri.go:87] found id: ""
I0921 21:59:14.058337 163433 logs.go:274] 0 containers: []
W0921 21:59:14.058346 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:14.058353 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:14.058408 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:14.084834 163433 cri.go:87] found id: ""
I0921 21:59:14.084869 163433 logs.go:274] 0 containers: []
W0921 21:59:14.084880 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:14.084889 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:14.084943 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:14.120731 163433 cri.go:87] found id: ""
I0921 21:59:14.120756 163433 logs.go:274] 0 containers: []
W0921 21:59:14.120765 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:14.120778 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:14.120792 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:14.144355 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:14.144399 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:14.228833 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:14.228871 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:14.228884 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:14.281289 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:14.281436 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:14.358951 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:14.358980 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:14.380506 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:24 kubernetes-upgrade-20220921215522-10174 kubelet[3640]: E0921 21:58:24.623599 3640 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.381201 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:25 kubernetes-upgrade-20220921215522-10174 kubelet[3651]: E0921 21:58:25.373329 3651 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.381905 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3664]: E0921 21:58:26.122744 3664 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.382622 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:26 kubernetes-upgrade-20220921215522-10174 kubelet[3675]: E0921 21:58:26.871823 3675 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.383327 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:27 kubernetes-upgrade-20220921215522-10174 kubelet[3686]: E0921 21:58:27.622585 3686 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.384067 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:28 kubernetes-upgrade-20220921215522-10174 kubelet[3697]: E0921 21:58:28.376468 3697 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.384815 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3708]: E0921 21:58:29.132047 3708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.385560 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:29 kubernetes-upgrade-20220921215522-10174 kubelet[3719]: E0921 21:58:29.877817 3719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.386268 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:30 kubernetes-upgrade-20220921215522-10174 kubelet[3730]: E0921 21:58:30.636585 3730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.387018 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:31 kubernetes-upgrade-20220921215522-10174 kubelet[3740]: E0921 21:58:31.375404 3740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.387826 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3854]: E0921 21:58:32.129853 3854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.388593 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:32 kubernetes-upgrade-20220921215522-10174 kubelet[3898]: E0921 21:58:32.873032 3898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.389294 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:33 kubernetes-upgrade-20220921215522-10174 kubelet[3910]: E0921 21:58:33.628639 3910 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.389995 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:34 kubernetes-upgrade-20220921215522-10174 kubelet[3921]: E0921 21:58:34.371911 3921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.390705 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3932]: E0921 21:58:35.133679 3932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.391412 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3942]: E0921 21:58:35.894996 3942 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.392138 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:36 kubernetes-upgrade-20220921215522-10174 kubelet[3952]: E0921 21:58:36.645679 3952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.392872 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:37 kubernetes-upgrade-20220921215522-10174 kubelet[3962]: E0921 21:58:37.382599 3962 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.393588 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3972]: E0921 21:58:38.148834 3972 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.394295 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3982]: E0921 21:58:38.878739 3982 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.395016 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.395734 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.396470 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.397079 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.397537 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.397968 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:43 kubernetes-upgrade-20220921215522-10174 kubelet[4185]: E0921 21:58:43.372417 4185 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.398701 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4197]: E0921 21:58:44.122124 4197 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.399391 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4209]: E0921 21:58:44.872708 4209 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.400161 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:45 kubernetes-upgrade-20220921215522-10174 kubelet[4219]: E0921 21:58:45.620684 4219 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.400815 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:46 kubernetes-upgrade-20220921215522-10174 kubelet[4230]: E0921 21:58:46.371087 4230 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.401491 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4243]: E0921 21:58:47.121413 4243 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.402259 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4254]: E0921 21:58:47.872018 4254 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.402699 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:48 kubernetes-upgrade-20220921215522-10174 kubelet[4264]: E0921 21:58:48.621704 4264 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.403097 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.403494 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.403967 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.404610 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.405259 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.405932 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4436]: E0921 21:58:53.129656 4436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.406692 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4480]: E0921 21:58:53.875326 4480 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.407374 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:54 kubernetes-upgrade-20220921215522-10174 kubelet[4491]: E0921 21:58:54.630207 4491 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.408179 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:55 kubernetes-upgrade-20220921215522-10174 kubelet[4505]: E0921 21:58:55.382351 4505 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.408997 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4515]: E0921 21:58:56.132997 4515 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.409755 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4524]: E0921 21:58:56.895570 4524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.410477 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:57 kubernetes-upgrade-20220921215522-10174 kubelet[4533]: E0921 21:58:57.642639 4533 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.411142 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:58 kubernetes-upgrade-20220921215522-10174 kubelet[4545]: E0921 21:58:58.392797 4545 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.411530 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4554]: E0921 21:58:59.134963 4554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.411994 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.412616 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.413275 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.413938 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.414568 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.415219 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:03 kubernetes-upgrade-20220921215522-10174 kubelet[4727]: E0921 21:59:03.637908 4727 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.415888 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:04 kubernetes-upgrade-20220921215522-10174 kubelet[4764]: E0921 21:59:04.376354 4764 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.416586 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4774]: E0921 21:59:05.130669 4774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.417308 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4786]: E0921 21:59:05.874316 4786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.417981 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:06 kubernetes-upgrade-20220921215522-10174 kubelet[4797]: E0921 21:59:06.621813 4797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.418648 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:07 kubernetes-upgrade-20220921215522-10174 kubelet[4808]: E0921 21:59:07.380124 4808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.419353 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4819]: E0921 21:59:08.126226 4819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.420065 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4830]: E0921 21:59:08.872713 4830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.420761 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:09 kubernetes-upgrade-20220921215522-10174 kubelet[4840]: E0921 21:59:09.623509 4840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.421474 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:10 kubernetes-upgrade-20220921215522-10174 kubelet[4850]: E0921 21:59:10.374150 4850 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.422180 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.422889 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.423592 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.424317 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.425029 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:14.425264 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:14.425281 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:14.425406 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:14.425461 163433 out.go:239] Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.425486 163433 out.go:239] Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.425508 163433 out.go:239] Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.425531 163433 out.go:239] Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:14.425553 163433 out.go:239] Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:14.425566 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:14.425574 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:24.427329 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:24.862945 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:24.863012 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:24.889625 163433 cri.go:87] found id: ""
I0921 21:59:24.889655 163433 logs.go:274] 0 containers: []
W0921 21:59:24.889664 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:24.889672 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:24.889719 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:24.916752 163433 cri.go:87] found id: ""
I0921 21:59:24.916775 163433 logs.go:274] 0 containers: []
W0921 21:59:24.916781 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:24.916787 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:24.916843 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:24.942910 163433 cri.go:87] found id: ""
I0921 21:59:24.942936 163433 logs.go:274] 0 containers: []
W0921 21:59:24.942942 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:24.942948 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:24.943006 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:24.967748 163433 cri.go:87] found id: ""
I0921 21:59:24.967781 163433 logs.go:274] 0 containers: []
W0921 21:59:24.967790 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:24.967797 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:24.967879 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:24.992649 163433 cri.go:87] found id: ""
I0921 21:59:24.992683 163433 logs.go:274] 0 containers: []
W0921 21:59:24.992693 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:24.992701 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:24.992788 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:25.017584 163433 cri.go:87] found id: ""
I0921 21:59:25.017606 163433 logs.go:274] 0 containers: []
W0921 21:59:25.017612 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:25.017618 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:25.017662 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:25.045775 163433 cri.go:87] found id: ""
I0921 21:59:25.045806 163433 logs.go:274] 0 containers: []
W0921 21:59:25.045816 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:25.045825 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:25.045904 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:25.075250 163433 cri.go:87] found id: ""
I0921 21:59:25.075273 163433 logs.go:274] 0 containers: []
W0921 21:59:25.075279 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:25.075288 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:25.075297 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:25.092668 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3932]: E0921 21:58:35.133679 3932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.093155 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:35 kubernetes-upgrade-20220921215522-10174 kubelet[3942]: E0921 21:58:35.894996 3942 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.093586 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:36 kubernetes-upgrade-20220921215522-10174 kubelet[3952]: E0921 21:58:36.645679 3952 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.094014 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:37 kubernetes-upgrade-20220921215522-10174 kubelet[3962]: E0921 21:58:37.382599 3962 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.094455 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3972]: E0921 21:58:38.148834 3972 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.094889 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:38 kubernetes-upgrade-20220921215522-10174 kubelet[3982]: E0921 21:58:38.878739 3982 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.095311 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:39 kubernetes-upgrade-20220921215522-10174 kubelet[3993]: E0921 21:58:39.634330 3993 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.095949 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:40 kubernetes-upgrade-20220921215522-10174 kubelet[4003]: E0921 21:58:40.371691 4003 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.096399 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4015]: E0921 21:58:41.121950 4015 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.096826 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:41 kubernetes-upgrade-20220921215522-10174 kubelet[4026]: E0921 21:58:41.871405 4026 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.097291 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:42 kubernetes-upgrade-20220921215522-10174 kubelet[4150]: E0921 21:58:42.630550 4150 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.097736 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:43 kubernetes-upgrade-20220921215522-10174 kubelet[4185]: E0921 21:58:43.372417 4185 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.098287 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4197]: E0921 21:58:44.122124 4197 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.098745 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:44 kubernetes-upgrade-20220921215522-10174 kubelet[4209]: E0921 21:58:44.872708 4209 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.099303 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:45 kubernetes-upgrade-20220921215522-10174 kubelet[4219]: E0921 21:58:45.620684 4219 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.099973 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:46 kubernetes-upgrade-20220921215522-10174 kubelet[4230]: E0921 21:58:46.371087 4230 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.100428 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4243]: E0921 21:58:47.121413 4243 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.100852 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4254]: E0921 21:58:47.872018 4254 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.101336 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:48 kubernetes-upgrade-20220921215522-10174 kubelet[4264]: E0921 21:58:48.621704 4264 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.101816 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.102368 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.102904 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.103553 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.104130 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.104527 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4436]: E0921 21:58:53.129656 4436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.104940 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4480]: E0921 21:58:53.875326 4480 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.105401 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:54 kubernetes-upgrade-20220921215522-10174 kubelet[4491]: E0921 21:58:54.630207 4491 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.105865 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:55 kubernetes-upgrade-20220921215522-10174 kubelet[4505]: E0921 21:58:55.382351 4505 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.106302 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4515]: E0921 21:58:56.132997 4515 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.106708 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4524]: E0921 21:58:56.895570 4524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.107116 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:57 kubernetes-upgrade-20220921215522-10174 kubelet[4533]: E0921 21:58:57.642639 4533 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.107506 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:58 kubernetes-upgrade-20220921215522-10174 kubelet[4545]: E0921 21:58:58.392797 4545 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.107963 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4554]: E0921 21:58:59.134963 4554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.108361 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.108746 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.109154 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.109547 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.109950 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.110350 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:03 kubernetes-upgrade-20220921215522-10174 kubelet[4727]: E0921 21:59:03.637908 4727 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.110741 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:04 kubernetes-upgrade-20220921215522-10174 kubelet[4764]: E0921 21:59:04.376354 4764 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.111140 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4774]: E0921 21:59:05.130669 4774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.111527 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4786]: E0921 21:59:05.874316 4786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.111945 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:06 kubernetes-upgrade-20220921215522-10174 kubelet[4797]: E0921 21:59:06.621813 4797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.112383 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:07 kubernetes-upgrade-20220921215522-10174 kubelet[4808]: E0921 21:59:07.380124 4808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.112788 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4819]: E0921 21:59:08.126226 4819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.113195 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4830]: E0921 21:59:08.872713 4830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.113588 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:09 kubernetes-upgrade-20220921215522-10174 kubelet[4840]: E0921 21:59:09.623509 4840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.113982 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:10 kubernetes-upgrade-20220921215522-10174 kubelet[4850]: E0921 21:59:10.374150 4850 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.114407 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.114807 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.115196 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.115624 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.116047 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.116444 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[5049]: E0921 21:59:14.875596 5049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.116838 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:15 kubernetes-upgrade-20220921215522-10174 kubelet[5060]: E0921 21:59:15.632288 5060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.117230 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:16 kubernetes-upgrade-20220921215522-10174 kubelet[5071]: E0921 21:59:16.381461 5071 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.117643 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5082]: E0921 21:59:17.132884 5082 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.118148 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5093]: E0921 21:59:17.876596 5093 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.118805 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:18 kubernetes-upgrade-20220921215522-10174 kubelet[5104]: E0921 21:59:18.633938 5104 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.119238 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:19 kubernetes-upgrade-20220921215522-10174 kubelet[5115]: E0921 21:59:19.373556 5115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.119633 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5125]: E0921 21:59:20.124415 5125 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.120086 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5136]: E0921 21:59:20.873303 5136 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.120485 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.120888 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.121289 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.121680 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.122128 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:25.122261 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:25.122278 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:25.137915 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:25.137947 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:25.200414 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:25.200436 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:25.200448 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:25.236786 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:25.236821 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:25.261691 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:25.261716 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:25.261828 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:25.261845 163433 out.go:239] Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.261853 163433 out.go:239] Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.261866 163433 out.go:239] Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.261877 163433 out.go:239] Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:25.261886 163433 out.go:239] Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:25.261890 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:25.261895 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:35.263224 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:35.362980 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:35.363055 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:35.386410 163433 cri.go:87] found id: ""
I0921 21:59:35.386436 163433 logs.go:274] 0 containers: []
W0921 21:59:35.386444 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:35.386452 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:35.386505 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:35.409479 163433 cri.go:87] found id: ""
I0921 21:59:35.409509 163433 logs.go:274] 0 containers: []
W0921 21:59:35.409519 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:35.409527 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:35.409581 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:35.433409 163433 cri.go:87] found id: ""
I0921 21:59:35.433457 163433 logs.go:274] 0 containers: []
W0921 21:59:35.433467 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:35.433476 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:35.433532 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:35.456657 163433 cri.go:87] found id: ""
I0921 21:59:35.456688 163433 logs.go:274] 0 containers: []
W0921 21:59:35.456696 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:35.456702 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:35.456746 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:35.479638 163433 cri.go:87] found id: ""
I0921 21:59:35.479663 163433 logs.go:274] 0 containers: []
W0921 21:59:35.479671 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:35.479679 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:35.479764 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:35.503409 163433 cri.go:87] found id: ""
I0921 21:59:35.503432 163433 logs.go:274] 0 containers: []
W0921 21:59:35.503438 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:35.503444 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:35.503484 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:35.527852 163433 cri.go:87] found id: ""
I0921 21:59:35.527884 163433 logs.go:274] 0 containers: []
W0921 21:59:35.527893 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:35.527902 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:35.527958 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:35.552383 163433 cri.go:87] found id: ""
I0921 21:59:35.552405 163433 logs.go:274] 0 containers: []
W0921 21:59:35.552411 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:35.552420 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:35.552435 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:35.568142 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:45 kubernetes-upgrade-20220921215522-10174 kubelet[4219]: E0921 21:58:45.620684 4219 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.568545 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:46 kubernetes-upgrade-20220921215522-10174 kubelet[4230]: E0921 21:58:46.371087 4230 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.568942 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4243]: E0921 21:58:47.121413 4243 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.569327 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:47 kubernetes-upgrade-20220921215522-10174 kubelet[4254]: E0921 21:58:47.872018 4254 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.569713 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:48 kubernetes-upgrade-20220921215522-10174 kubelet[4264]: E0921 21:58:48.621704 4264 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.570092 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:49 kubernetes-upgrade-20220921215522-10174 kubelet[4275]: E0921 21:58:49.371878 4275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.570470 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4286]: E0921 21:58:50.122084 4286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.570851 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:50 kubernetes-upgrade-20220921215522-10174 kubelet[4297]: E0921 21:58:50.876452 4297 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.571237 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:51 kubernetes-upgrade-20220921215522-10174 kubelet[4307]: E0921 21:58:51.627411 4307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.571613 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:52 kubernetes-upgrade-20220921215522-10174 kubelet[4318]: E0921 21:58:52.373933 4318 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.572063 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4436]: E0921 21:58:53.129656 4436 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.572450 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:53 kubernetes-upgrade-20220921215522-10174 kubelet[4480]: E0921 21:58:53.875326 4480 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.572830 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:54 kubernetes-upgrade-20220921215522-10174 kubelet[4491]: E0921 21:58:54.630207 4491 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.573209 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:55 kubernetes-upgrade-20220921215522-10174 kubelet[4505]: E0921 21:58:55.382351 4505 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.573586 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4515]: E0921 21:58:56.132997 4515 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.573966 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4524]: E0921 21:58:56.895570 4524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.574354 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:57 kubernetes-upgrade-20220921215522-10174 kubelet[4533]: E0921 21:58:57.642639 4533 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.574740 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:58 kubernetes-upgrade-20220921215522-10174 kubelet[4545]: E0921 21:58:58.392797 4545 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.575121 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4554]: E0921 21:58:59.134963 4554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.575510 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.575925 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.576302 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.576675 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.577073 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.577462 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:03 kubernetes-upgrade-20220921215522-10174 kubelet[4727]: E0921 21:59:03.637908 4727 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.577843 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:04 kubernetes-upgrade-20220921215522-10174 kubelet[4764]: E0921 21:59:04.376354 4764 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.578231 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4774]: E0921 21:59:05.130669 4774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.578613 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4786]: E0921 21:59:05.874316 4786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.578996 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:06 kubernetes-upgrade-20220921215522-10174 kubelet[4797]: E0921 21:59:06.621813 4797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.579378 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:07 kubernetes-upgrade-20220921215522-10174 kubelet[4808]: E0921 21:59:07.380124 4808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.579842 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4819]: E0921 21:59:08.126226 4819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.580238 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4830]: E0921 21:59:08.872713 4830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.580613 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:09 kubernetes-upgrade-20220921215522-10174 kubelet[4840]: E0921 21:59:09.623509 4840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.581021 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:10 kubernetes-upgrade-20220921215522-10174 kubelet[4850]: E0921 21:59:10.374150 4850 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.581407 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.581784 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.582154 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.582645 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.583026 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.583443 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[5049]: E0921 21:59:14.875596 5049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.583923 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:15 kubernetes-upgrade-20220921215522-10174 kubelet[5060]: E0921 21:59:15.632288 5060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.584306 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:16 kubernetes-upgrade-20220921215522-10174 kubelet[5071]: E0921 21:59:16.381461 5071 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.584686 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5082]: E0921 21:59:17.132884 5082 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.585067 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5093]: E0921 21:59:17.876596 5093 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.585453 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:18 kubernetes-upgrade-20220921215522-10174 kubelet[5104]: E0921 21:59:18.633938 5104 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.585840 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:19 kubernetes-upgrade-20220921215522-10174 kubelet[5115]: E0921 21:59:19.373556 5115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.586229 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5125]: E0921 21:59:20.124415 5125 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.586607 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5136]: E0921 21:59:20.873303 5136 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.586989 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.587380 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.587789 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.588178 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.588556 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.588940 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:25 kubernetes-upgrade-20220921215522-10174 kubelet[5337]: E0921 21:59:25.370164 5337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.589334 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5348]: E0921 21:59:26.124915 5348 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.589723 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5359]: E0921 21:59:26.873809 5359 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.590103 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:27 kubernetes-upgrade-20220921215522-10174 kubelet[5370]: E0921 21:59:27.628142 5370 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.590504 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:28 kubernetes-upgrade-20220921215522-10174 kubelet[5382]: E0921 21:59:28.390222 5382 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.590886 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5393]: E0921 21:59:29.137158 5393 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.591261 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5402]: E0921 21:59:29.876549 5402 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.591656 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:30 kubernetes-upgrade-20220921215522-10174 kubelet[5412]: E0921 21:59:30.626109 5412 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.592093 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:31 kubernetes-upgrade-20220921215522-10174 kubelet[5423]: E0921 21:59:31.374600 5423 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.592482 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.592860 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.593241 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.593620 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.594034 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:35.594196 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:35.594212 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:35.608913 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:35.608939 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:35.661660 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:35.661689 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:35.661701 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:35.698240 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:35.698271 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:35.724050 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:35.724077 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:35.724232 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:35.724252 163433 out.go:239] Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.724263 163433 out.go:239] Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.724269 163433 out.go:239] Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.724274 163433 out.go:239] Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:35.724279 163433 out.go:239] Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:35.724283 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:35.724288 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:45.725263 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:45.863196 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:45.863285 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:45.888025 163433 cri.go:87] found id: ""
I0921 21:59:45.888053 163433 logs.go:274] 0 containers: []
W0921 21:59:45.888060 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:45.888066 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:45.888126 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:45.910603 163433 cri.go:87] found id: ""
I0921 21:59:45.910635 163433 logs.go:274] 0 containers: []
W0921 21:59:45.910644 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:45.910651 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:45.910695 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:45.934248 163433 cri.go:87] found id: ""
I0921 21:59:45.934274 163433 logs.go:274] 0 containers: []
W0921 21:59:45.934283 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:45.934291 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:45.934348 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:45.956614 163433 cri.go:87] found id: ""
I0921 21:59:45.956643 163433 logs.go:274] 0 containers: []
W0921 21:59:45.956653 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:45.956659 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:45.956703 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:45.978721 163433 cri.go:87] found id: ""
I0921 21:59:45.978748 163433 logs.go:274] 0 containers: []
W0921 21:59:45.978756 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:45.978763 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:45.978831 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:46.005441 163433 cri.go:87] found id: ""
I0921 21:59:46.005480 163433 logs.go:274] 0 containers: []
W0921 21:59:46.005490 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:46.005498 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:46.005548 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:46.029762 163433 cri.go:87] found id: ""
I0921 21:59:46.029788 163433 logs.go:274] 0 containers: []
W0921 21:59:46.029794 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:46.029799 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:46.029853 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:46.054492 163433 cri.go:87] found id: ""
I0921 21:59:46.054520 163433 logs.go:274] 0 containers: []
W0921 21:59:46.054527 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:46.054536 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:46.054545 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:46.071489 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4515]: E0921 21:58:56.132997 4515 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.071963 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:56 kubernetes-upgrade-20220921215522-10174 kubelet[4524]: E0921 21:58:56.895570 4524 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.072364 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:57 kubernetes-upgrade-20220921215522-10174 kubelet[4533]: E0921 21:58:57.642639 4533 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.072764 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:58 kubernetes-upgrade-20220921215522-10174 kubelet[4545]: E0921 21:58:58.392797 4545 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.073157 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4554]: E0921 21:58:59.134963 4554 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.073541 163433 logs.go:138] Found kubelet problem: Sep 21 21:58:59 kubernetes-upgrade-20220921215522-10174 kubelet[4566]: E0921 21:58:59.874689 4566 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.073922 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:00 kubernetes-upgrade-20220921215522-10174 kubelet[4577]: E0921 21:59:00.646186 4577 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.074333 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:01 kubernetes-upgrade-20220921215522-10174 kubelet[4587]: E0921 21:59:01.371592 4587 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.074725 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4598]: E0921 21:59:02.124561 4598 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.075109 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:02 kubernetes-upgrade-20220921215522-10174 kubelet[4610]: E0921 21:59:02.871389 4610 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.075506 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:03 kubernetes-upgrade-20220921215522-10174 kubelet[4727]: E0921 21:59:03.637908 4727 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.075927 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:04 kubernetes-upgrade-20220921215522-10174 kubelet[4764]: E0921 21:59:04.376354 4764 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.076403 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4774]: E0921 21:59:05.130669 4774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.076802 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:05 kubernetes-upgrade-20220921215522-10174 kubelet[4786]: E0921 21:59:05.874316 4786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.077187 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:06 kubernetes-upgrade-20220921215522-10174 kubelet[4797]: E0921 21:59:06.621813 4797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.077614 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:07 kubernetes-upgrade-20220921215522-10174 kubelet[4808]: E0921 21:59:07.380124 4808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.078027 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4819]: E0921 21:59:08.126226 4819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.078404 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4830]: E0921 21:59:08.872713 4830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.078813 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:09 kubernetes-upgrade-20220921215522-10174 kubelet[4840]: E0921 21:59:09.623509 4840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.079208 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:10 kubernetes-upgrade-20220921215522-10174 kubelet[4850]: E0921 21:59:10.374150 4850 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.079629 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.080047 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.080439 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.080847 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.081244 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.081627 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[5049]: E0921 21:59:14.875596 5049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.082033 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:15 kubernetes-upgrade-20220921215522-10174 kubelet[5060]: E0921 21:59:15.632288 5060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.082440 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:16 kubernetes-upgrade-20220921215522-10174 kubelet[5071]: E0921 21:59:16.381461 5071 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.082830 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5082]: E0921 21:59:17.132884 5082 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.083234 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5093]: E0921 21:59:17.876596 5093 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.083657 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:18 kubernetes-upgrade-20220921215522-10174 kubelet[5104]: E0921 21:59:18.633938 5104 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.084123 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:19 kubernetes-upgrade-20220921215522-10174 kubelet[5115]: E0921 21:59:19.373556 5115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.084503 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5125]: E0921 21:59:20.124415 5125 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.084908 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5136]: E0921 21:59:20.873303 5136 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.085295 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.085678 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.086084 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.086490 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.086877 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.087274 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:25 kubernetes-upgrade-20220921215522-10174 kubelet[5337]: E0921 21:59:25.370164 5337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.087662 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5348]: E0921 21:59:26.124915 5348 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.088082 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5359]: E0921 21:59:26.873809 5359 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.088489 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:27 kubernetes-upgrade-20220921215522-10174 kubelet[5370]: E0921 21:59:27.628142 5370 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.088873 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:28 kubernetes-upgrade-20220921215522-10174 kubelet[5382]: E0921 21:59:28.390222 5382 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.089268 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5393]: E0921 21:59:29.137158 5393 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.089672 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5402]: E0921 21:59:29.876549 5402 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.090059 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:30 kubernetes-upgrade-20220921215522-10174 kubelet[5412]: E0921 21:59:30.626109 5412 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.090437 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:31 kubernetes-upgrade-20220921215522-10174 kubelet[5423]: E0921 21:59:31.374600 5423 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.090832 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.091270 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.091648 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.092083 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.092459 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.092877 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5630]: E0921 21:59:35.871827 5630 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.093257 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:36 kubernetes-upgrade-20220921215522-10174 kubelet[5641]: E0921 21:59:36.621845 5641 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.093648 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:37 kubernetes-upgrade-20220921215522-10174 kubelet[5652]: E0921 21:59:37.378142 5652 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.094039 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5663]: E0921 21:59:38.135567 5663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.094431 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5674]: E0921 21:59:38.874212 5674 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.094827 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:39 kubernetes-upgrade-20220921215522-10174 kubelet[5685]: E0921 21:59:39.623075 5685 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.095214 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:40 kubernetes-upgrade-20220921215522-10174 kubelet[5696]: E0921 21:59:40.374289 5696 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.095594 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5708]: E0921 21:59:41.126179 5708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.096006 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5719]: E0921 21:59:41.879591 5719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.096405 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.096801 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.097204 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.097605 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.098014 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:46.098145 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:46.098163 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:46.114435 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:46.114473 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:46.168644 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:46.168667 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:46.168679 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:46.204920 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:46.204953 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:46.231338 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:46.231372 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:46.231484 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:46.231501 163433 out.go:239] Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.231509 163433 out.go:239] Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.231522 163433 out.go:239] Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.231533 163433 out.go:239] Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:46.231544 163433 out.go:239] Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:46.231556 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:46.231568 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 21:59:56.232362 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 21:59:56.363254 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 21:59:56.363320 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 21:59:56.386079 163433 cri.go:87] found id: ""
I0921 21:59:56.386105 163433 logs.go:274] 0 containers: []
W0921 21:59:56.386112 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 21:59:56.386119 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 21:59:56.386160 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 21:59:56.408178 163433 cri.go:87] found id: ""
I0921 21:59:56.408206 163433 logs.go:274] 0 containers: []
W0921 21:59:56.408215 163433 logs.go:276] No container was found matching "etcd"
I0921 21:59:56.408222 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 21:59:56.408273 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 21:59:56.431215 163433 cri.go:87] found id: ""
I0921 21:59:56.431240 163433 logs.go:274] 0 containers: []
W0921 21:59:56.431246 163433 logs.go:276] No container was found matching "coredns"
I0921 21:59:56.431251 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 21:59:56.431308 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 21:59:56.453885 163433 cri.go:87] found id: ""
I0921 21:59:56.453913 163433 logs.go:274] 0 containers: []
W0921 21:59:56.453919 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 21:59:56.453925 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 21:59:56.453976 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 21:59:56.476089 163433 cri.go:87] found id: ""
I0921 21:59:56.476110 163433 logs.go:274] 0 containers: []
W0921 21:59:56.476116 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 21:59:56.476121 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 21:59:56.476162 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 21:59:56.498717 163433 cri.go:87] found id: ""
I0921 21:59:56.498748 163433 logs.go:274] 0 containers: []
W0921 21:59:56.498755 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 21:59:56.498761 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 21:59:56.498803 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 21:59:56.521339 163433 cri.go:87] found id: ""
I0921 21:59:56.521363 163433 logs.go:274] 0 containers: []
W0921 21:59:56.521369 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 21:59:56.521375 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 21:59:56.521415 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 21:59:56.544377 163433 cri.go:87] found id: ""
I0921 21:59:56.544405 163433 logs.go:274] 0 containers: []
W0921 21:59:56.544414 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 21:59:56.544426 163433 logs.go:123] Gathering logs for dmesg ...
I0921 21:59:56.544443 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 21:59:56.558351 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 21:59:56.558375 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 21:59:56.615428 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 21:59:56.615459 163433 logs.go:123] Gathering logs for containerd ...
I0921 21:59:56.615473 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 21:59:56.658412 163433 logs.go:123] Gathering logs for container status ...
I0921 21:59:56.658459 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 21:59:56.710314 163433 logs.go:123] Gathering logs for kubelet ...
I0921 21:59:56.710351 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 21:59:56.728263 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:06 kubernetes-upgrade-20220921215522-10174 kubelet[4797]: E0921 21:59:06.621813 4797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.728925 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:07 kubernetes-upgrade-20220921215522-10174 kubelet[4808]: E0921 21:59:07.380124 4808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.729566 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4819]: E0921 21:59:08.126226 4819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.730201 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:08 kubernetes-upgrade-20220921215522-10174 kubelet[4830]: E0921 21:59:08.872713 4830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.730803 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:09 kubernetes-upgrade-20220921215522-10174 kubelet[4840]: E0921 21:59:09.623509 4840 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.731276 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:10 kubernetes-upgrade-20220921215522-10174 kubelet[4850]: E0921 21:59:10.374150 4850 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.731946 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4861]: E0921 21:59:11.139623 4861 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.732470 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:11 kubernetes-upgrade-20220921215522-10174 kubelet[4872]: E0921 21:59:11.874420 4872 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.732892 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:12 kubernetes-upgrade-20220921215522-10174 kubelet[4883]: E0921 21:59:12.643620 4883 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.733286 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:13 kubernetes-upgrade-20220921215522-10174 kubelet[4893]: E0921 21:59:13.393673 4893 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.733674 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[4988]: E0921 21:59:14.157483 4988 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.734055 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:14 kubernetes-upgrade-20220921215522-10174 kubelet[5049]: E0921 21:59:14.875596 5049 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.734445 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:15 kubernetes-upgrade-20220921215522-10174 kubelet[5060]: E0921 21:59:15.632288 5060 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.734851 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:16 kubernetes-upgrade-20220921215522-10174 kubelet[5071]: E0921 21:59:16.381461 5071 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.735284 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5082]: E0921 21:59:17.132884 5082 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.735667 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5093]: E0921 21:59:17.876596 5093 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.736136 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:18 kubernetes-upgrade-20220921215522-10174 kubelet[5104]: E0921 21:59:18.633938 5104 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.736526 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:19 kubernetes-upgrade-20220921215522-10174 kubelet[5115]: E0921 21:59:19.373556 5115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.737107 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5125]: E0921 21:59:20.124415 5125 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.737773 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5136]: E0921 21:59:20.873303 5136 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.738249 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.738650 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.739025 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.739412 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.739842 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.740230 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:25 kubernetes-upgrade-20220921215522-10174 kubelet[5337]: E0921 21:59:25.370164 5337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.740617 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5348]: E0921 21:59:26.124915 5348 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.741134 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5359]: E0921 21:59:26.873809 5359 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.741597 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:27 kubernetes-upgrade-20220921215522-10174 kubelet[5370]: E0921 21:59:27.628142 5370 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.742008 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:28 kubernetes-upgrade-20220921215522-10174 kubelet[5382]: E0921 21:59:28.390222 5382 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.742422 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5393]: E0921 21:59:29.137158 5393 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.742856 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5402]: E0921 21:59:29.876549 5402 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.743281 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:30 kubernetes-upgrade-20220921215522-10174 kubelet[5412]: E0921 21:59:30.626109 5412 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.743697 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:31 kubernetes-upgrade-20220921215522-10174 kubelet[5423]: E0921 21:59:31.374600 5423 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.744123 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.744544 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.744979 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.745428 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.745845 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.746256 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5630]: E0921 21:59:35.871827 5630 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.746674 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:36 kubernetes-upgrade-20220921215522-10174 kubelet[5641]: E0921 21:59:36.621845 5641 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.747075 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:37 kubernetes-upgrade-20220921215522-10174 kubelet[5652]: E0921 21:59:37.378142 5652 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.747486 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5663]: E0921 21:59:38.135567 5663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.747975 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5674]: E0921 21:59:38.874212 5674 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.748366 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:39 kubernetes-upgrade-20220921215522-10174 kubelet[5685]: E0921 21:59:39.623075 5685 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.748755 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:40 kubernetes-upgrade-20220921215522-10174 kubelet[5696]: E0921 21:59:40.374289 5696 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.749149 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5708]: E0921 21:59:41.126179 5708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.749526 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5719]: E0921 21:59:41.879591 5719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.749907 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.750285 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.750682 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.751068 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.751450 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.751893 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:46 kubernetes-upgrade-20220921215522-10174 kubelet[5921]: E0921 21:59:46.369661 5921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.752294 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5932]: E0921 21:59:47.127018 5932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.752950 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5943]: E0921 21:59:47.882291 5943 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.753358 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:48 kubernetes-upgrade-20220921215522-10174 kubelet[5954]: E0921 21:59:48.659376 5954 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.753756 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:49 kubernetes-upgrade-20220921215522-10174 kubelet[5966]: E0921 21:59:49.395999 5966 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.754158 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5977]: E0921 21:59:50.139088 5977 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.754565 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5987]: E0921 21:59:50.872398 5987 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.754971 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:51 kubernetes-upgrade-20220921215522-10174 kubelet[5999]: E0921 21:59:51.621693 5999 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.755372 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:52 kubernetes-upgrade-20220921215522-10174 kubelet[6010]: E0921 21:59:52.378257 6010 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.755840 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.756262 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.756643 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757086 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757477 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:56.757607 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:56.757621 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 21:59:56.757724 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 21:59:56.757738 163433 out.go:239] Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757743 163433 out.go:239] Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757749 163433 out.go:239] Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757754 163433 out.go:239] Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 21:59:56.757760 163433 out.go:239] Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 21:59:56.757766 163433 out.go:309] Setting ErrFile to fd 2...
I0921 21:59:56.757774 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:00:06.759265 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 22:00:06.863333 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 22:00:06.863401 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 22:00:06.889775 163433 cri.go:87] found id: ""
I0921 22:00:06.889804 163433 logs.go:274] 0 containers: []
W0921 22:00:06.889814 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 22:00:06.889822 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 22:00:06.889870 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 22:00:06.911963 163433 cri.go:87] found id: ""
I0921 22:00:06.911994 163433 logs.go:274] 0 containers: []
W0921 22:00:06.912003 163433 logs.go:276] No container was found matching "etcd"
I0921 22:00:06.912011 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 22:00:06.912055 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 22:00:06.934612 163433 cri.go:87] found id: ""
I0921 22:00:06.934646 163433 logs.go:274] 0 containers: []
W0921 22:00:06.934655 163433 logs.go:276] No container was found matching "coredns"
I0921 22:00:06.934663 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 22:00:06.934707 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 22:00:06.961709 163433 cri.go:87] found id: ""
I0921 22:00:06.961739 163433 logs.go:274] 0 containers: []
W0921 22:00:06.961749 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 22:00:06.961756 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 22:00:06.961810 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 22:00:07.006619 163433 cri.go:87] found id: ""
I0921 22:00:07.006647 163433 logs.go:274] 0 containers: []
W0921 22:00:07.006656 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 22:00:07.006667 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 22:00:07.006722 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 22:00:07.041876 163433 cri.go:87] found id: ""
I0921 22:00:07.041904 163433 logs.go:274] 0 containers: []
W0921 22:00:07.041942 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 22:00:07.041952 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 22:00:07.042006 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 22:00:07.115611 163433 cri.go:87] found id: ""
I0921 22:00:07.115635 163433 logs.go:274] 0 containers: []
W0921 22:00:07.115641 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 22:00:07.115647 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 22:00:07.115688 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 22:00:07.217253 163433 cri.go:87] found id: ""
I0921 22:00:07.217289 163433 logs.go:274] 0 containers: []
W0921 22:00:07.217297 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 22:00:07.217309 163433 logs.go:123] Gathering logs for container status ...
I0921 22:00:07.217323 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 22:00:07.246160 163433 logs.go:123] Gathering logs for kubelet ...
I0921 22:00:07.246193 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 22:00:07.262860 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5082]: E0921 21:59:17.132884 5082 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.263258 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:17 kubernetes-upgrade-20220921215522-10174 kubelet[5093]: E0921 21:59:17.876596 5093 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.263645 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:18 kubernetes-upgrade-20220921215522-10174 kubelet[5104]: E0921 21:59:18.633938 5104 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.264094 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:19 kubernetes-upgrade-20220921215522-10174 kubelet[5115]: E0921 21:59:19.373556 5115 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.264506 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5125]: E0921 21:59:20.124415 5125 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.265029 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:20 kubernetes-upgrade-20220921215522-10174 kubelet[5136]: E0921 21:59:20.873303 5136 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.265473 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:21 kubernetes-upgrade-20220921215522-10174 kubelet[5147]: E0921 21:59:21.625541 5147 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.265972 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:22 kubernetes-upgrade-20220921215522-10174 kubelet[5158]: E0921 21:59:22.381102 5158 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.266351 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5170]: E0921 21:59:23.127688 5170 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.266797 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:23 kubernetes-upgrade-20220921215522-10174 kubelet[5181]: E0921 21:59:23.885323 5181 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.267203 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:24 kubernetes-upgrade-20220921215522-10174 kubelet[5194]: E0921 21:59:24.636411 5194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.267587 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:25 kubernetes-upgrade-20220921215522-10174 kubelet[5337]: E0921 21:59:25.370164 5337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.268012 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5348]: E0921 21:59:26.124915 5348 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.268398 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:26 kubernetes-upgrade-20220921215522-10174 kubelet[5359]: E0921 21:59:26.873809 5359 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.268788 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:27 kubernetes-upgrade-20220921215522-10174 kubelet[5370]: E0921 21:59:27.628142 5370 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.269208 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:28 kubernetes-upgrade-20220921215522-10174 kubelet[5382]: E0921 21:59:28.390222 5382 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.269596 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5393]: E0921 21:59:29.137158 5393 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.269980 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5402]: E0921 21:59:29.876549 5402 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.270373 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:30 kubernetes-upgrade-20220921215522-10174 kubelet[5412]: E0921 21:59:30.626109 5412 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.270746 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:31 kubernetes-upgrade-20220921215522-10174 kubelet[5423]: E0921 21:59:31.374600 5423 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.271126 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.271517 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.271916 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.272296 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.272689 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.273082 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5630]: E0921 21:59:35.871827 5630 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.273499 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:36 kubernetes-upgrade-20220921215522-10174 kubelet[5641]: E0921 21:59:36.621845 5641 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.273874 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:37 kubernetes-upgrade-20220921215522-10174 kubelet[5652]: E0921 21:59:37.378142 5652 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.274252 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5663]: E0921 21:59:38.135567 5663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.274643 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5674]: E0921 21:59:38.874212 5674 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.275020 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:39 kubernetes-upgrade-20220921215522-10174 kubelet[5685]: E0921 21:59:39.623075 5685 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.275399 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:40 kubernetes-upgrade-20220921215522-10174 kubelet[5696]: E0921 21:59:40.374289 5696 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.275932 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5708]: E0921 21:59:41.126179 5708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.276591 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5719]: E0921 21:59:41.879591 5719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.277278 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.277949 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.278598 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.279294 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.279997 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.280655 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:46 kubernetes-upgrade-20220921215522-10174 kubelet[5921]: E0921 21:59:46.369661 5921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.281361 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5932]: E0921 21:59:47.127018 5932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.282053 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5943]: E0921 21:59:47.882291 5943 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.284751 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:48 kubernetes-upgrade-20220921215522-10174 kubelet[5954]: E0921 21:59:48.659376 5954 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.285448 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:49 kubernetes-upgrade-20220921215522-10174 kubelet[5966]: E0921 21:59:49.395999 5966 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.286160 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5977]: E0921 21:59:50.139088 5977 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.286961 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5987]: E0921 21:59:50.872398 5987 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.287922 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:51 kubernetes-upgrade-20220921215522-10174 kubelet[5999]: E0921 21:59:51.621693 5999 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.288632 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:52 kubernetes-upgrade-20220921215522-10174 kubelet[6010]: E0921 21:59:52.378257 6010 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.289352 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.290144 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.290821 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.294823 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.295499 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.296201 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6212]: E0921 21:59:56.912660 6212 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.296909 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:57 kubernetes-upgrade-20220921215522-10174 kubelet[6223]: E0921 21:59:57.625738 6223 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.297570 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:58 kubernetes-upgrade-20220921215522-10174 kubelet[6233]: E0921 21:59:58.376248 6233 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.298245 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6244]: E0921 21:59:59.124640 6244 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.298958 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6255]: E0921 21:59:59.885711 6255 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.299673 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:00 kubernetes-upgrade-20220921215522-10174 kubelet[6265]: E0921 22:00:00.628088 6265 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.305340 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:01 kubernetes-upgrade-20220921215522-10174 kubelet[6275]: E0921 22:00:01.381353 6275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.306041 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6286]: E0921 22:00:02.136330 6286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.306704 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6296]: E0921 22:00:02.884102 6296 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.307360 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:03 kubernetes-upgrade-20220921215522-10174 kubelet[6307]: E0921 22:00:03.631068 6307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.308033 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:04 kubernetes-upgrade-20220921215522-10174 kubelet[6317]: E0921 22:00:04.376091 6317 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.308686 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6328]: E0921 22:00:05.124355 6328 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.309364 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6339]: E0921 22:00:05.878766 6339 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.310002 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:06 kubernetes-upgrade-20220921215522-10174 kubelet[6349]: E0921 22:00:06.623312 6349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:07.310233 163433 logs.go:123] Gathering logs for dmesg ...
I0921 22:00:07.310251 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 22:00:07.334736 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 22:00:07.334835 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 22:00:07.447250 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 22:00:07.447293 163433 logs.go:123] Gathering logs for containerd ...
I0921 22:00:07.447308 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 22:00:07.503668 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:07.503704 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 22:00:07.503936 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 22:00:07.503973 163433 out.go:239] Sep 21 22:00:03 kubernetes-upgrade-20220921215522-10174 kubelet[6307]: E0921 22:00:03.631068 6307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:03 kubernetes-upgrade-20220921215522-10174 kubelet[6307]: E0921 22:00:03.631068 6307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.503981 163433 out.go:239] Sep 21 22:00:04 kubernetes-upgrade-20220921215522-10174 kubelet[6317]: E0921 22:00:04.376091 6317 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:04 kubernetes-upgrade-20220921215522-10174 kubelet[6317]: E0921 22:00:04.376091 6317 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.503991 163433 out.go:239] Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6328]: E0921 22:00:05.124355 6328 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6328]: E0921 22:00:05.124355 6328 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.504001 163433 out.go:239] Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6339]: E0921 22:00:05.878766 6339 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6339]: E0921 22:00:05.878766 6339 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:07.504009 163433 out.go:239] Sep 21 22:00:06 kubernetes-upgrade-20220921215522-10174 kubelet[6349]: E0921 22:00:06.623312 6349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:06 kubernetes-upgrade-20220921215522-10174 kubelet[6349]: E0921 22:00:06.623312 6349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:07.504079 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:07.504095 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:00:17.506037 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 22:00:17.862999 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 22:00:17.863091 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 22:00:17.890706 163433 cri.go:87] found id: ""
I0921 22:00:17.890735 163433 logs.go:274] 0 containers: []
W0921 22:00:17.890752 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 22:00:17.890759 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 22:00:17.890818 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 22:00:17.919086 163433 cri.go:87] found id: ""
I0921 22:00:17.919119 163433 logs.go:274] 0 containers: []
W0921 22:00:17.919127 163433 logs.go:276] No container was found matching "etcd"
I0921 22:00:17.919136 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 22:00:17.919189 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 22:00:17.943931 163433 cri.go:87] found id: ""
I0921 22:00:17.943966 163433 logs.go:274] 0 containers: []
W0921 22:00:17.943975 163433 logs.go:276] No container was found matching "coredns"
I0921 22:00:17.943982 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 22:00:17.944037 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 22:00:17.970927 163433 cri.go:87] found id: ""
I0921 22:00:17.970954 163433 logs.go:274] 0 containers: []
W0921 22:00:17.970961 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 22:00:17.970966 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 22:00:17.971044 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 22:00:17.997937 163433 cri.go:87] found id: ""
I0921 22:00:17.997960 163433 logs.go:274] 0 containers: []
W0921 22:00:17.997967 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 22:00:17.997972 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 22:00:17.998027 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 22:00:18.025139 163433 cri.go:87] found id: ""
I0921 22:00:18.025167 163433 logs.go:274] 0 containers: []
W0921 22:00:18.025175 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 22:00:18.025183 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 22:00:18.025253 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 22:00:18.057790 163433 cri.go:87] found id: ""
I0921 22:00:18.057815 163433 logs.go:274] 0 containers: []
W0921 22:00:18.057821 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 22:00:18.057827 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 22:00:18.057878 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 22:00:18.086006 163433 cri.go:87] found id: ""
I0921 22:00:18.086037 163433 logs.go:274] 0 containers: []
W0921 22:00:18.086046 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 22:00:18.086059 163433 logs.go:123] Gathering logs for kubelet ...
I0921 22:00:18.086073 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 22:00:18.104469 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:28 kubernetes-upgrade-20220921215522-10174 kubelet[5382]: E0921 21:59:28.390222 5382 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.104889 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5393]: E0921 21:59:29.137158 5393 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.105311 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:29 kubernetes-upgrade-20220921215522-10174 kubelet[5402]: E0921 21:59:29.876549 5402 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.105879 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:30 kubernetes-upgrade-20220921215522-10174 kubelet[5412]: E0921 21:59:30.626109 5412 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.106541 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:31 kubernetes-upgrade-20220921215522-10174 kubelet[5423]: E0921 21:59:31.374600 5423 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.107212 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5435]: E0921 21:59:32.145394 5435 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.107920 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:32 kubernetes-upgrade-20220921215522-10174 kubelet[5446]: E0921 21:59:32.870591 5446 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.108433 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:33 kubernetes-upgrade-20220921215522-10174 kubelet[5457]: E0921 21:59:33.620542 5457 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.109004 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:34 kubernetes-upgrade-20220921215522-10174 kubelet[5468]: E0921 21:59:34.371200 5468 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.109557 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5479]: E0921 21:59:35.124735 5479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.109978 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:35 kubernetes-upgrade-20220921215522-10174 kubelet[5630]: E0921 21:59:35.871827 5630 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.110373 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:36 kubernetes-upgrade-20220921215522-10174 kubelet[5641]: E0921 21:59:36.621845 5641 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.110933 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:37 kubernetes-upgrade-20220921215522-10174 kubelet[5652]: E0921 21:59:37.378142 5652 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.111637 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5663]: E0921 21:59:38.135567 5663 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.112285 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5674]: E0921 21:59:38.874212 5674 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.112735 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:39 kubernetes-upgrade-20220921215522-10174 kubelet[5685]: E0921 21:59:39.623075 5685 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.113111 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:40 kubernetes-upgrade-20220921215522-10174 kubelet[5696]: E0921 21:59:40.374289 5696 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.113571 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5708]: E0921 21:59:41.126179 5708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.114242 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5719]: E0921 21:59:41.879591 5719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.114945 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.115599 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.116289 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.116930 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.117607 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.118211 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:46 kubernetes-upgrade-20220921215522-10174 kubelet[5921]: E0921 21:59:46.369661 5921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.118699 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5932]: E0921 21:59:47.127018 5932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.119196 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5943]: E0921 21:59:47.882291 5943 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.119653 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:48 kubernetes-upgrade-20220921215522-10174 kubelet[5954]: E0921 21:59:48.659376 5954 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.120146 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:49 kubernetes-upgrade-20220921215522-10174 kubelet[5966]: E0921 21:59:49.395999 5966 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.120691 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5977]: E0921 21:59:50.139088 5977 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.121407 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5987]: E0921 21:59:50.872398 5987 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.122061 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:51 kubernetes-upgrade-20220921215522-10174 kubelet[5999]: E0921 21:59:51.621693 5999 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.122739 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:52 kubernetes-upgrade-20220921215522-10174 kubelet[6010]: E0921 21:59:52.378257 6010 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.123391 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.124080 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.124802 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.125512 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.126217 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.126789 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6212]: E0921 21:59:56.912660 6212 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.127191 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:57 kubernetes-upgrade-20220921215522-10174 kubelet[6223]: E0921 21:59:57.625738 6223 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.127786 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:58 kubernetes-upgrade-20220921215522-10174 kubelet[6233]: E0921 21:59:58.376248 6233 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.128284 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6244]: E0921 21:59:59.124640 6244 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.128751 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6255]: E0921 21:59:59.885711 6255 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.129343 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:00 kubernetes-upgrade-20220921215522-10174 kubelet[6265]: E0921 22:00:00.628088 6265 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.129915 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:01 kubernetes-upgrade-20220921215522-10174 kubelet[6275]: E0921 22:00:01.381353 6275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.130316 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6286]: E0921 22:00:02.136330 6286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.130910 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6296]: E0921 22:00:02.884102 6296 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.131557 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:03 kubernetes-upgrade-20220921215522-10174 kubelet[6307]: E0921 22:00:03.631068 6307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.132039 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:04 kubernetes-upgrade-20220921215522-10174 kubelet[6317]: E0921 22:00:04.376091 6317 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.132443 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6328]: E0921 22:00:05.124355 6328 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.132860 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6339]: E0921 22:00:05.878766 6339 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.133252 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:06 kubernetes-upgrade-20220921215522-10174 kubelet[6349]: E0921 22:00:06.623312 6349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.133634 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:07 kubernetes-upgrade-20220921215522-10174 kubelet[6479]: E0921 22:00:07.413586 6479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.134039 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:08 kubernetes-upgrade-20220921215522-10174 kubelet[6498]: E0921 22:00:08.203915 6498 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.134422 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:08 kubernetes-upgrade-20220921215522-10174 kubelet[6506]: E0921 22:00:08.896024 6506 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.134801 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:09 kubernetes-upgrade-20220921215522-10174 kubelet[6517]: E0921 22:00:09.631187 6517 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.135191 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:10 kubernetes-upgrade-20220921215522-10174 kubelet[6528]: E0921 22:00:10.378072 6528 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.135583 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:11 kubernetes-upgrade-20220921215522-10174 kubelet[6537]: E0921 22:00:11.154422 6537 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.135993 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:11 kubernetes-upgrade-20220921215522-10174 kubelet[6548]: E0921 22:00:11.883832 6548 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.136387 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:12 kubernetes-upgrade-20220921215522-10174 kubelet[6558]: E0921 22:00:12.650188 6558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.136762 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:13 kubernetes-upgrade-20220921215522-10174 kubelet[6569]: E0921 22:00:13.393024 6569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.137146 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6579]: E0921 22:00:14.131211 6579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.137557 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6589]: E0921 22:00:14.887145 6589 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.137949 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:15 kubernetes-upgrade-20220921215522-10174 kubelet[6601]: E0921 22:00:15.671116 6601 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.138338 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:16 kubernetes-upgrade-20220921215522-10174 kubelet[6612]: E0921 22:00:16.392213 6612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.138719 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6621]: E0921 22:00:17.141076 6621 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.139103 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6634]: E0921 22:00:17.877141 6634 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:18.139248 163433 logs.go:123] Gathering logs for dmesg ...
I0921 22:00:18.139308 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 22:00:18.154371 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 22:00:18.154399 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 22:00:18.223787 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 22:00:18.223812 163433 logs.go:123] Gathering logs for containerd ...
I0921 22:00:18.223822 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 22:00:18.259233 163433 logs.go:123] Gathering logs for container status ...
I0921 22:00:18.259265 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 22:00:18.296290 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:18.296316 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 22:00:18.296434 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 22:00:18.296447 163433 out.go:239] Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6589]: E0921 22:00:14.887145 6589 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6589]: E0921 22:00:14.887145 6589 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.296455 163433 out.go:239] Sep 21 22:00:15 kubernetes-upgrade-20220921215522-10174 kubelet[6601]: E0921 22:00:15.671116 6601 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:15 kubernetes-upgrade-20220921215522-10174 kubelet[6601]: E0921 22:00:15.671116 6601 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.296464 163433 out.go:239] Sep 21 22:00:16 kubernetes-upgrade-20220921215522-10174 kubelet[6612]: E0921 22:00:16.392213 6612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:16 kubernetes-upgrade-20220921215522-10174 kubelet[6612]: E0921 22:00:16.392213 6612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.296474 163433 out.go:239] Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6621]: E0921 22:00:17.141076 6621 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6621]: E0921 22:00:17.141076 6621 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:18.296487 163433 out.go:239] Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6634]: E0921 22:00:17.877141 6634 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6634]: E0921 22:00:17.877141 6634 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:18.296493 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:18.296505 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:00:28.297531 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 22:00:28.363064 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 22:00:28.363176 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 22:00:28.393968 163433 cri.go:87] found id: ""
I0921 22:00:28.393997 163433 logs.go:274] 0 containers: []
W0921 22:00:28.394006 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 22:00:28.394015 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 22:00:28.394077 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 22:00:28.425199 163433 cri.go:87] found id: ""
I0921 22:00:28.425228 163433 logs.go:274] 0 containers: []
W0921 22:00:28.425236 163433 logs.go:276] No container was found matching "etcd"
I0921 22:00:28.425244 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 22:00:28.425305 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 22:00:28.458022 163433 cri.go:87] found id: ""
I0921 22:00:28.458058 163433 logs.go:274] 0 containers: []
W0921 22:00:28.458069 163433 logs.go:276] No container was found matching "coredns"
I0921 22:00:28.458077 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 22:00:28.458141 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 22:00:28.484520 163433 cri.go:87] found id: ""
I0921 22:00:28.484560 163433 logs.go:274] 0 containers: []
W0921 22:00:28.484569 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 22:00:28.484577 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 22:00:28.484642 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 22:00:28.510468 163433 cri.go:87] found id: ""
I0921 22:00:28.510498 163433 logs.go:274] 0 containers: []
W0921 22:00:28.510508 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 22:00:28.510516 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 22:00:28.510564 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 22:00:28.535911 163433 cri.go:87] found id: ""
I0921 22:00:28.535934 163433 logs.go:274] 0 containers: []
W0921 22:00:28.535940 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 22:00:28.535946 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 22:00:28.535997 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 22:00:28.573959 163433 cri.go:87] found id: ""
I0921 22:00:28.573988 163433 logs.go:274] 0 containers: []
W0921 22:00:28.573995 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 22:00:28.574001 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 22:00:28.574044 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 22:00:28.607709 163433 cri.go:87] found id: ""
I0921 22:00:28.607756 163433 logs.go:274] 0 containers: []
W0921 22:00:28.607763 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 22:00:28.607773 163433 logs.go:123] Gathering logs for kubelet ...
I0921 22:00:28.607784 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 22:00:28.624882 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:38 kubernetes-upgrade-20220921215522-10174 kubelet[5674]: E0921 21:59:38.874212 5674 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.625602 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:39 kubernetes-upgrade-20220921215522-10174 kubelet[5685]: E0921 21:59:39.623075 5685 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.626333 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:40 kubernetes-upgrade-20220921215522-10174 kubelet[5696]: E0921 21:59:40.374289 5696 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.627034 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5708]: E0921 21:59:41.126179 5708 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.627769 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:41 kubernetes-upgrade-20220921215522-10174 kubelet[5719]: E0921 21:59:41.879591 5719 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.628496 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:42 kubernetes-upgrade-20220921215522-10174 kubelet[5729]: E0921 21:59:42.633853 5729 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.629220 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:43 kubernetes-upgrade-20220921215522-10174 kubelet[5740]: E0921 21:59:43.387451 5740 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.629933 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5752]: E0921 21:59:44.128455 5752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.630646 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:44 kubernetes-upgrade-20220921215522-10174 kubelet[5762]: E0921 21:59:44.874407 5762 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.631358 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:45 kubernetes-upgrade-20220921215522-10174 kubelet[5774]: E0921 21:59:45.623741 5774 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.632109 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:46 kubernetes-upgrade-20220921215522-10174 kubelet[5921]: E0921 21:59:46.369661 5921 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.632826 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5932]: E0921 21:59:47.127018 5932 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.633525 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:47 kubernetes-upgrade-20220921215522-10174 kubelet[5943]: E0921 21:59:47.882291 5943 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.634234 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:48 kubernetes-upgrade-20220921215522-10174 kubelet[5954]: E0921 21:59:48.659376 5954 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.634960 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:49 kubernetes-upgrade-20220921215522-10174 kubelet[5966]: E0921 21:59:49.395999 5966 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.635661 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5977]: E0921 21:59:50.139088 5977 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.636379 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:50 kubernetes-upgrade-20220921215522-10174 kubelet[5987]: E0921 21:59:50.872398 5987 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.637093 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:51 kubernetes-upgrade-20220921215522-10174 kubelet[5999]: E0921 21:59:51.621693 5999 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.637798 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:52 kubernetes-upgrade-20220921215522-10174 kubelet[6010]: E0921 21:59:52.378257 6010 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.638504 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6021]: E0921 21:59:53.122030 6021 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.639234 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:53 kubernetes-upgrade-20220921215522-10174 kubelet[6032]: E0921 21:59:53.883474 6032 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.639957 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:54 kubernetes-upgrade-20220921215522-10174 kubelet[6043]: E0921 21:59:54.623656 6043 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.640663 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:55 kubernetes-upgrade-20220921215522-10174 kubelet[6053]: E0921 21:59:55.373880 6053 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.641375 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6064]: E0921 21:59:56.122849 6064 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.642089 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:56 kubernetes-upgrade-20220921215522-10174 kubelet[6212]: E0921 21:59:56.912660 6212 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.642802 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:57 kubernetes-upgrade-20220921215522-10174 kubelet[6223]: E0921 21:59:57.625738 6223 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.643510 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:58 kubernetes-upgrade-20220921215522-10174 kubelet[6233]: E0921 21:59:58.376248 6233 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.644235 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6244]: E0921 21:59:59.124640 6244 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.644946 163433 logs.go:138] Found kubelet problem: Sep 21 21:59:59 kubernetes-upgrade-20220921215522-10174 kubelet[6255]: E0921 21:59:59.885711 6255 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.645654 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:00 kubernetes-upgrade-20220921215522-10174 kubelet[6265]: E0921 22:00:00.628088 6265 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.646365 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:01 kubernetes-upgrade-20220921215522-10174 kubelet[6275]: E0921 22:00:01.381353 6275 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.647081 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6286]: E0921 22:00:02.136330 6286 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.647797 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:02 kubernetes-upgrade-20220921215522-10174 kubelet[6296]: E0921 22:00:02.884102 6296 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.648511 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:03 kubernetes-upgrade-20220921215522-10174 kubelet[6307]: E0921 22:00:03.631068 6307 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.649237 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:04 kubernetes-upgrade-20220921215522-10174 kubelet[6317]: E0921 22:00:04.376091 6317 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.649959 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6328]: E0921 22:00:05.124355 6328 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.650666 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:05 kubernetes-upgrade-20220921215522-10174 kubelet[6339]: E0921 22:00:05.878766 6339 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.651377 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:06 kubernetes-upgrade-20220921215522-10174 kubelet[6349]: E0921 22:00:06.623312 6349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.652101 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:07 kubernetes-upgrade-20220921215522-10174 kubelet[6479]: E0921 22:00:07.413586 6479 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.652815 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:08 kubernetes-upgrade-20220921215522-10174 kubelet[6498]: E0921 22:00:08.203915 6498 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.653518 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:08 kubernetes-upgrade-20220921215522-10174 kubelet[6506]: E0921 22:00:08.896024 6506 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.654229 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:09 kubernetes-upgrade-20220921215522-10174 kubelet[6517]: E0921 22:00:09.631187 6517 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.654943 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:10 kubernetes-upgrade-20220921215522-10174 kubelet[6528]: E0921 22:00:10.378072 6528 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.655651 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:11 kubernetes-upgrade-20220921215522-10174 kubelet[6537]: E0921 22:00:11.154422 6537 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.656373 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:11 kubernetes-upgrade-20220921215522-10174 kubelet[6548]: E0921 22:00:11.883832 6548 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.657089 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:12 kubernetes-upgrade-20220921215522-10174 kubelet[6558]: E0921 22:00:12.650188 6558 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.657804 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:13 kubernetes-upgrade-20220921215522-10174 kubelet[6569]: E0921 22:00:13.393024 6569 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.658512 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6579]: E0921 22:00:14.131211 6579 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.659226 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:14 kubernetes-upgrade-20220921215522-10174 kubelet[6589]: E0921 22:00:14.887145 6589 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.659962 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:15 kubernetes-upgrade-20220921215522-10174 kubelet[6601]: E0921 22:00:15.671116 6601 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.660686 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:16 kubernetes-upgrade-20220921215522-10174 kubelet[6612]: E0921 22:00:16.392213 6612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.661399 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6621]: E0921 22:00:17.141076 6621 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.662117 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:17 kubernetes-upgrade-20220921215522-10174 kubelet[6634]: E0921 22:00:17.877141 6634 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.662824 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:18 kubernetes-upgrade-20220921215522-10174 kubelet[6780]: E0921 22:00:18.628375 6780 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.663557 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:19 kubernetes-upgrade-20220921215522-10174 kubelet[6791]: E0921 22:00:19.403500 6791 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.664274 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:20 kubernetes-upgrade-20220921215522-10174 kubelet[6802]: E0921 22:00:20.126094 6802 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.664984 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:20 kubernetes-upgrade-20220921215522-10174 kubelet[6813]: E0921 22:00:20.880801 6813 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.665689 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:21 kubernetes-upgrade-20220921215522-10174 kubelet[6825]: E0921 22:00:21.625219 6825 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.666401 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:22 kubernetes-upgrade-20220921215522-10174 kubelet[6836]: E0921 22:00:22.375925 6836 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.667115 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:23 kubernetes-upgrade-20220921215522-10174 kubelet[6847]: E0921 22:00:23.123479 6847 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.667838 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:23 kubernetes-upgrade-20220921215522-10174 kubelet[6858]: E0921 22:00:23.879137 6858 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.668549 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:24 kubernetes-upgrade-20220921215522-10174 kubelet[6869]: E0921 22:00:24.632766 6869 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.669270 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:25 kubernetes-upgrade-20220921215522-10174 kubelet[6880]: E0921 22:00:25.374278 6880 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.669991 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6891]: E0921 22:00:26.138502 6891 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.670701 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6901]: E0921 22:00:26.871756 6901 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.671420 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:27 kubernetes-upgrade-20220921215522-10174 kubelet[6913]: E0921 22:00:27.648560 6913 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.672138 163433 logs.go:138] Found kubelet problem: Sep 21 22:00:28 kubernetes-upgrade-20220921215522-10174 kubelet[6925]: E0921 22:00:28.391197 6925 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:28.672374 163433 logs.go:123] Gathering logs for dmesg ...
I0921 22:00:28.672396 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 22:00:28.688119 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 22:00:28.688154 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 22:00:28.755946 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 22:00:28.755970 163433 logs.go:123] Gathering logs for containerd ...
I0921 22:00:28.755980 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 22:00:28.795608 163433 logs.go:123] Gathering logs for container status ...
I0921 22:00:28.795641 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0921 22:00:28.822712 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:28.822737 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0921 22:00:28.822861 163433 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0921 22:00:28.822878 163433 out.go:239] Sep 21 22:00:25 kubernetes-upgrade-20220921215522-10174 kubelet[6880]: E0921 22:00:25.374278 6880 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:25 kubernetes-upgrade-20220921215522-10174 kubelet[6880]: E0921 22:00:25.374278 6880 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.822885 163433 out.go:239] Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6891]: E0921 22:00:26.138502 6891 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6891]: E0921 22:00:26.138502 6891 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.822892 163433 out.go:239] Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6901]: E0921 22:00:26.871756 6901 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:26 kubernetes-upgrade-20220921215522-10174 kubelet[6901]: E0921 22:00:26.871756 6901 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.822901 163433 out.go:239] Sep 21 22:00:27 kubernetes-upgrade-20220921215522-10174 kubelet[6913]: E0921 22:00:27.648560 6913 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:27 kubernetes-upgrade-20220921215522-10174 kubelet[6913]: E0921 22:00:27.648560 6913 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:00:28.822907 163433 out.go:239] Sep 21 22:00:28 kubernetes-upgrade-20220921215522-10174 kubelet[6925]: E0921 22:00:28.391197 6925 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:00:28 kubernetes-upgrade-20220921215522-10174 kubelet[6925]: E0921 22:00:28.391197 6925 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:00:28.822913 163433 out.go:309] Setting ErrFile to fd 2...
I0921 22:00:28.822919 163433 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:00:38.823268 163433 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 22:00:38.863544 163433 kubeadm.go:631] restartCluster took 4m1.12814734s
W0921 22:00:38.863699 163433 out.go:239] ! Unable to restart cluster, will reset it: apiserver healthz: apiserver process never appeared
! Unable to restart cluster, will reset it: apiserver healthz: apiserver process never appeared
I0921 22:00:38.863769 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0921 22:00:40.855466 163433 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force": (1.991670203s)
I0921 22:00:40.855539 163433 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0921 22:00:40.866648 163433 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0921 22:00:40.873760 163433 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0921 22:00:40.873823 163433 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0921 22:00:40.881453 163433 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0921 22:00:40.881506 163433 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0921 22:00:40.924774 163433 kubeadm.go:317] W0921 22:00:40.923939 8214 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
I0921 22:00:40.961370 163433 kubeadm.go:317] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
I0921 22:00:41.029465 163433 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0921 22:02:37.018351 163433 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0921 22:02:37.018524 163433 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0921 22:02:37.021344 163433 kubeadm.go:317] [init] Using Kubernetes version: v1.25.2
I0921 22:02:37.021412 163433 kubeadm.go:317] [preflight] Running pre-flight checks
I0921 22:02:37.021521 163433 kubeadm.go:317] [preflight] The system verification failed. Printing the output from the verification:
I0921 22:02:37.021617 163433 kubeadm.go:317] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
I0921 22:02:37.021677 163433 kubeadm.go:317] [0;37mOS[0m: [0;32mLinux[0m
I0921 22:02:37.021750 163433 kubeadm.go:317] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0921 22:02:37.021830 163433 kubeadm.go:317] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0921 22:02:37.021902 163433 kubeadm.go:317] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0921 22:02:37.021973 163433 kubeadm.go:317] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0921 22:02:37.022047 163433 kubeadm.go:317] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0921 22:02:37.022121 163433 kubeadm.go:317] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0921 22:02:37.022188 163433 kubeadm.go:317] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0921 22:02:37.022253 163433 kubeadm.go:317] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0921 22:02:37.022318 163433 kubeadm.go:317] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0921 22:02:37.022421 163433 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0921 22:02:37.022566 163433 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0921 22:02:37.022728 163433 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0921 22:02:37.022848 163433 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0921 22:02:37.025283 163433 out.go:204] - Generating certificates and keys ...
I0921 22:02:37.025385 163433 kubeadm.go:317] [certs] Using existing ca certificate authority
I0921 22:02:37.025475 163433 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0921 22:02:37.025582 163433 kubeadm.go:317] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0921 22:02:37.025662 163433 kubeadm.go:317] [certs] Using existing front-proxy-ca certificate authority
I0921 22:02:37.025742 163433 kubeadm.go:317] [certs] Using existing front-proxy-client certificate and key on disk
I0921 22:02:37.025826 163433 kubeadm.go:317] [certs] Using existing etcd/ca certificate authority
I0921 22:02:37.025908 163433 kubeadm.go:317] [certs] Using existing etcd/server certificate and key on disk
I0921 22:02:37.025970 163433 kubeadm.go:317] [certs] Using existing etcd/peer certificate and key on disk
I0921 22:02:37.026043 163433 kubeadm.go:317] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0921 22:02:37.026115 163433 kubeadm.go:317] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0921 22:02:37.026150 163433 kubeadm.go:317] [certs] Using the existing "sa" key
I0921 22:02:37.026236 163433 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0921 22:02:37.026314 163433 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0921 22:02:37.026388 163433 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0921 22:02:37.026472 163433 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0921 22:02:37.026545 163433 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0921 22:02:37.026684 163433 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0921 22:02:37.026792 163433 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0921 22:02:37.026839 163433 kubeadm.go:317] [kubelet-start] Starting the kubelet
I0921 22:02:37.026895 163433 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0921 22:02:37.028542 163433 out.go:204] - Booting up control plane ...
I0921 22:02:37.028642 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0921 22:02:37.028731 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0921 22:02:37.028833 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0921 22:02:37.028952 163433 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0921 22:02:37.029138 163433 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0921 22:02:37.029208 163433 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0921 22:02:37.029298 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.029557 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.029641 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.029848 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.029934 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030147 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030218 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030380 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030466 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030646 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030656 163433 kubeadm.go:317]
I0921 22:02:37.030710 163433 kubeadm.go:317] Unfortunately, an error has occurred:
I0921 22:02:37.030783 163433 kubeadm.go:317] timed out waiting for the condition
I0921 22:02:37.030793 163433 kubeadm.go:317]
I0921 22:02:37.030849 163433 kubeadm.go:317] This error is likely caused by:
I0921 22:02:37.030900 163433 kubeadm.go:317] - The kubelet is not running
I0921 22:02:37.031052 163433 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0921 22:02:37.031070 163433 kubeadm.go:317]
I0921 22:02:37.031191 163433 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0921 22:02:37.031243 163433 kubeadm.go:317] - 'systemctl status kubelet'
I0921 22:02:37.031297 163433 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0921 22:02:37.031308 163433 kubeadm.go:317]
I0921 22:02:37.031422 163433 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0921 22:02:37.031534 163433 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0921 22:02:37.031667 163433 kubeadm.go:317] Here is one example how you may list all running Kubernetes containers by using crictl:
I0921 22:02:37.031810 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
I0921 22:02:37.031929 163433 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0921 22:02:37.032051 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
W0921 22:02:37.032372 163433 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:00:40.923939 8214 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:00:40.923939 8214 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0921 22:02:37.032427 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0921 22:02:38.854856 163433 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force": (1.822402127s)
I0921 22:02:38.854917 163433 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0921 22:02:38.864392 163433 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0921 22:02:38.864449 163433 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0921 22:02:38.871143 163433 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0921 22:02:38.871179 163433 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0921 22:02:38.909279 163433 kubeadm.go:317] [init] Using Kubernetes version: v1.25.2
I0921 22:02:38.909365 163433 kubeadm.go:317] [preflight] Running pre-flight checks
I0921 22:02:38.936932 163433 kubeadm.go:317] [preflight] The system verification failed. Printing the output from the verification:
I0921 22:02:38.937031 163433 kubeadm.go:317] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
I0921 22:02:38.937063 163433 kubeadm.go:317] [0;37mOS[0m: [0;32mLinux[0m
I0921 22:02:38.937103 163433 kubeadm.go:317] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0921 22:02:38.937150 163433 kubeadm.go:317] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0921 22:02:38.937239 163433 kubeadm.go:317] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0921 22:02:38.937318 163433 kubeadm.go:317] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0921 22:02:38.937390 163433 kubeadm.go:317] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0921 22:02:38.937481 163433 kubeadm.go:317] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0921 22:02:38.937544 163433 kubeadm.go:317] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0921 22:02:38.937595 163433 kubeadm.go:317] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0921 22:02:38.937662 163433 kubeadm.go:317] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0921 22:02:39.000511 163433 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0921 22:02:39.000640 163433 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0921 22:02:39.000796 163433 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0921 22:02:39.116829 163433 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0921 22:02:39.119686 163433 out.go:204] - Generating certificates and keys ...
I0921 22:02:39.119846 163433 kubeadm.go:317] [certs] Using existing ca certificate authority
I0921 22:02:39.119930 163433 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0921 22:02:39.120020 163433 kubeadm.go:317] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0921 22:02:39.120098 163433 kubeadm.go:317] [certs] Using existing front-proxy-ca certificate authority
I0921 22:02:39.120195 163433 kubeadm.go:317] [certs] Using existing front-proxy-client certificate and key on disk
I0921 22:02:39.120267 163433 kubeadm.go:317] [certs] Using existing etcd/ca certificate authority
I0921 22:02:39.120352 163433 kubeadm.go:317] [certs] Using existing etcd/server certificate and key on disk
I0921 22:02:39.120437 163433 kubeadm.go:317] [certs] Using existing etcd/peer certificate and key on disk
I0921 22:02:39.120536 163433 kubeadm.go:317] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0921 22:02:39.120627 163433 kubeadm.go:317] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0921 22:02:39.120686 163433 kubeadm.go:317] [certs] Using the existing "sa" key
I0921 22:02:39.120764 163433 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0921 22:02:39.214860 163433 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0921 22:02:39.365850 163433 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0921 22:02:39.700601 163433 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0921 22:02:40.053396 163433 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0921 22:02:40.065917 163433 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0921 22:02:40.066776 163433 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0921 22:02:40.066842 163433 kubeadm.go:317] [kubelet-start] Starting the kubelet
I0921 22:02:40.151802 163433 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0921 22:02:40.154174 163433 out.go:204] - Booting up control plane ...
I0921 22:02:40.154312 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0921 22:02:40.154475 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0921 22:02:40.155428 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0921 22:02:40.156385 163433 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0921 22:02:40.159229 163433 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0921 22:03:20.159558 163433 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0921 22:03:20.159897 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:20.160149 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:25.160511 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:25.160744 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:35.160837 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:35.161107 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:55.162112 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:55.162307 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:04:35.163184 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:04:35.163483 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:04:35.163518 163433 kubeadm.go:317]
I0921 22:04:35.163584 163433 kubeadm.go:317] Unfortunately, an error has occurred:
I0921 22:04:35.163648 163433 kubeadm.go:317] timed out waiting for the condition
I0921 22:04:35.163661 163433 kubeadm.go:317]
I0921 22:04:35.163710 163433 kubeadm.go:317] This error is likely caused by:
I0921 22:04:35.163796 163433 kubeadm.go:317] - The kubelet is not running
I0921 22:04:35.163928 163433 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0921 22:04:35.163937 163433 kubeadm.go:317]
I0921 22:04:35.164060 163433 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0921 22:04:35.164118 163433 kubeadm.go:317] - 'systemctl status kubelet'
I0921 22:04:35.164157 163433 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0921 22:04:35.164173 163433 kubeadm.go:317]
I0921 22:04:35.164320 163433 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0921 22:04:35.164436 163433 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0921 22:04:35.164551 163433 kubeadm.go:317] Here is one example how you may list all running Kubernetes containers by using crictl:
I0921 22:04:35.164687 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
I0921 22:04:35.164801 163433 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0921 22:04:35.164918 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
I0921 22:04:35.166104 163433 kubeadm.go:317] W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
I0921 22:04:35.166318 163433 kubeadm.go:317] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
I0921 22:04:35.166415 163433 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0921 22:04:35.166492 163433 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0921 22:04:35.166592 163433 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0921 22:04:35.166696 163433 kubeadm.go:398] StartCluster complete in 7m57.461301325s
I0921 22:04:35.166738 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 22:04:35.166788 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 22:04:35.191228 163433 cri.go:87] found id: ""
I0921 22:04:35.191254 163433 logs.go:274] 0 containers: []
W0921 22:04:35.191261 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 22:04:35.191272 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 22:04:35.191332 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 22:04:35.214715 163433 cri.go:87] found id: ""
I0921 22:04:35.214744 163433 logs.go:274] 0 containers: []
W0921 22:04:35.214750 163433 logs.go:276] No container was found matching "etcd"
I0921 22:04:35.214756 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 22:04:35.214804 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 22:04:35.239162 163433 cri.go:87] found id: ""
I0921 22:04:35.239186 163433 logs.go:274] 0 containers: []
W0921 22:04:35.239192 163433 logs.go:276] No container was found matching "coredns"
I0921 22:04:35.239197 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 22:04:35.239252 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 22:04:35.263333 163433 cri.go:87] found id: ""
I0921 22:04:35.263357 163433 logs.go:274] 0 containers: []
W0921 22:04:35.263363 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 22:04:35.263368 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 22:04:35.263407 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 22:04:35.286321 163433 cri.go:87] found id: ""
I0921 22:04:35.286347 163433 logs.go:274] 0 containers: []
W0921 22:04:35.286355 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 22:04:35.286364 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 22:04:35.286426 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 22:04:35.314686 163433 cri.go:87] found id: ""
I0921 22:04:35.314714 163433 logs.go:274] 0 containers: []
W0921 22:04:35.314722 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 22:04:35.314730 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 22:04:35.314783 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 22:04:35.338110 163433 cri.go:87] found id: ""
I0921 22:04:35.338142 163433 logs.go:274] 0 containers: []
W0921 22:04:35.338151 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 22:04:35.338160 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 22:04:35.338240 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 22:04:35.363135 163433 cri.go:87] found id: ""
I0921 22:04:35.363167 163433 logs.go:274] 0 containers: []
W0921 22:04:35.363176 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 22:04:35.363188 163433 logs.go:123] Gathering logs for kubelet ...
I0921 22:04:35.363200 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 22:04:35.379811 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:45 kubernetes-upgrade-20220921215522-10174 kubelet[12183]: E0921 22:03:45.374524 12183 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.380296 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12194]: E0921 22:03:46.123389 12194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.380707 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12205]: E0921 22:03:46.873433 12205 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381121 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:47 kubernetes-upgrade-20220921215522-10174 kubelet[12216]: E0921 22:03:47.622295 12216 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381546 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:48 kubernetes-upgrade-20220921215522-10174 kubelet[12227]: E0921 22:03:48.372753 12227 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381922 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:49 kubernetes-upgrade-20220921215522-10174 kubelet[12238]: E0921 22:03:49.121733 12238 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.382304 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:49 kubernetes-upgrade-20220921215522-10174 kubelet[12249]: E0921 22:03:49.874411 12249 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.382688 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:50 kubernetes-upgrade-20220921215522-10174 kubelet[12260]: E0921 22:03:50.621954 12260 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383079 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:51 kubernetes-upgrade-20220921215522-10174 kubelet[12271]: E0921 22:03:51.373297 12271 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383463 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:52 kubernetes-upgrade-20220921215522-10174 kubelet[12282]: E0921 22:03:52.124808 12282 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383909 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:52 kubernetes-upgrade-20220921215522-10174 kubelet[12293]: E0921 22:03:52.872464 12293 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.384310 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:53 kubernetes-upgrade-20220921215522-10174 kubelet[12304]: E0921 22:03:53.621146 12304 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.384694 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:54 kubernetes-upgrade-20220921215522-10174 kubelet[12315]: E0921 22:03:54.372737 12315 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385094 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:55 kubernetes-upgrade-20220921215522-10174 kubelet[12326]: E0921 22:03:55.122043 12326 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385483 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:55 kubernetes-upgrade-20220921215522-10174 kubelet[12337]: E0921 22:03:55.873129 12337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385896 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:56 kubernetes-upgrade-20220921215522-10174 kubelet[12349]: E0921 22:03:56.622479 12349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.386331 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:57 kubernetes-upgrade-20220921215522-10174 kubelet[12360]: E0921 22:03:57.373797 12360 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.386778 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:58 kubernetes-upgrade-20220921215522-10174 kubelet[12371]: E0921 22:03:58.123173 12371 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387176 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:58 kubernetes-upgrade-20220921215522-10174 kubelet[12383]: E0921 22:03:58.872655 12383 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387549 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:59 kubernetes-upgrade-20220921215522-10174 kubelet[12394]: E0921 22:03:59.624915 12394 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387984 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:00 kubernetes-upgrade-20220921215522-10174 kubelet[12405]: E0921 22:04:00.373541 12405 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.388383 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:01 kubernetes-upgrade-20220921215522-10174 kubelet[12416]: E0921 22:04:01.125035 12416 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.388766 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:01 kubernetes-upgrade-20220921215522-10174 kubelet[12428]: E0921 22:04:01.874353 12428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.389209 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:02 kubernetes-upgrade-20220921215522-10174 kubelet[12439]: E0921 22:04:02.624609 12439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.389617 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:03 kubernetes-upgrade-20220921215522-10174 kubelet[12450]: E0921 22:04:03.374384 12450 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390085 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:04 kubernetes-upgrade-20220921215522-10174 kubelet[12461]: E0921 22:04:04.121121 12461 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390489 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:04 kubernetes-upgrade-20220921215522-10174 kubelet[12471]: E0921 22:04:04.874129 12471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390875 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:05 kubernetes-upgrade-20220921215522-10174 kubelet[12482]: E0921 22:04:05.622119 12482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.391301 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:06 kubernetes-upgrade-20220921215522-10174 kubelet[12494]: E0921 22:04:06.373783 12494 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.391702 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:07 kubernetes-upgrade-20220921215522-10174 kubelet[12504]: E0921 22:04:07.122518 12504 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.392128 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:07 kubernetes-upgrade-20220921215522-10174 kubelet[12516]: E0921 22:04:07.873670 12516 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.392780 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:08 kubernetes-upgrade-20220921215522-10174 kubelet[12527]: E0921 22:04:08.622374 12527 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.393441 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:09 kubernetes-upgrade-20220921215522-10174 kubelet[12539]: E0921 22:04:09.380221 12539 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.393896 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:10 kubernetes-upgrade-20220921215522-10174 kubelet[12549]: E0921 22:04:10.123024 12549 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.394289 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:10 kubernetes-upgrade-20220921215522-10174 kubelet[12560]: E0921 22:04:10.871810 12560 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.394689 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:11 kubernetes-upgrade-20220921215522-10174 kubelet[12570]: E0921 22:04:11.623184 12570 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395075 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:12 kubernetes-upgrade-20220921215522-10174 kubelet[12581]: E0921 22:04:12.373639 12581 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395484 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:13 kubernetes-upgrade-20220921215522-10174 kubelet[12592]: E0921 22:04:13.122993 12592 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395916 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:13 kubernetes-upgrade-20220921215522-10174 kubelet[12602]: E0921 22:04:13.874559 12602 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.396306 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:14 kubernetes-upgrade-20220921215522-10174 kubelet[12612]: E0921 22:04:14.623058 12612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.396713 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:15 kubernetes-upgrade-20220921215522-10174 kubelet[12623]: E0921 22:04:15.372885 12623 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397101 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:16 kubernetes-upgrade-20220921215522-10174 kubelet[12633]: E0921 22:04:16.123144 12633 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397495 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:16 kubernetes-upgrade-20220921215522-10174 kubelet[12644]: E0921 22:04:16.872120 12644 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397886 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:17 kubernetes-upgrade-20220921215522-10174 kubelet[12655]: E0921 22:04:17.621957 12655 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.398268 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:18 kubernetes-upgrade-20220921215522-10174 kubelet[12666]: E0921 22:04:18.372467 12666 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.398694 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:19 kubernetes-upgrade-20220921215522-10174 kubelet[12676]: E0921 22:04:19.122753 12676 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399092 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:19 kubernetes-upgrade-20220921215522-10174 kubelet[12687]: E0921 22:04:19.876067 12687 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399483 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:20 kubernetes-upgrade-20220921215522-10174 kubelet[12698]: E0921 22:04:20.623954 12698 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399925 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:21 kubernetes-upgrade-20220921215522-10174 kubelet[12709]: E0921 22:04:21.373994 12709 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.400440 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:22 kubernetes-upgrade-20220921215522-10174 kubelet[12720]: E0921 22:04:22.124039 12720 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.400894 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:22 kubernetes-upgrade-20220921215522-10174 kubelet[12730]: E0921 22:04:22.872700 12730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.401275 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:23 kubernetes-upgrade-20220921215522-10174 kubelet[12741]: E0921 22:04:23.620191 12741 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.401664 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:24 kubernetes-upgrade-20220921215522-10174 kubelet[12752]: E0921 22:04:24.373902 12752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402051 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:25 kubernetes-upgrade-20220921215522-10174 kubelet[12763]: E0921 22:04:25.123958 12763 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402430 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:25 kubernetes-upgrade-20220921215522-10174 kubelet[12773]: E0921 22:04:25.874249 12773 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402818 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:26 kubernetes-upgrade-20220921215522-10174 kubelet[12786]: E0921 22:04:26.623403 12786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403198 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:27 kubernetes-upgrade-20220921215522-10174 kubelet[12797]: E0921 22:04:27.375008 12797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403573 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:28 kubernetes-upgrade-20220921215522-10174 kubelet[12808]: E0921 22:04:28.122905 12808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403988 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:28 kubernetes-upgrade-20220921215522-10174 kubelet[12819]: E0921 22:04:28.871801 12819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.404363 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:29 kubernetes-upgrade-20220921215522-10174 kubelet[12830]: E0921 22:04:29.622004 12830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.404759 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:30 kubernetes-upgrade-20220921215522-10174 kubelet[12842]: E0921 22:04:30.373730 12842 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405157 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:31 kubernetes-upgrade-20220921215522-10174 kubelet[12854]: E0921 22:04:31.121895 12854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405540 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:31 kubernetes-upgrade-20220921215522-10174 kubelet[12865]: E0921 22:04:31.872824 12865 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405921 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:32 kubernetes-upgrade-20220921215522-10174 kubelet[12876]: E0921 22:04:32.622571 12876 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.406306 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:33 kubernetes-upgrade-20220921215522-10174 kubelet[12887]: E0921 22:04:33.373474 12887 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.406697 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12898]: E0921 22:04:34.121999 12898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.407098 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12909]: E0921 22:04:34.878168 12909 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.407226 163433 logs.go:123] Gathering logs for dmesg ...
I0921 22:04:35.407241 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 22:04:35.428361 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 22:04:35.428401 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 22:04:35.484445 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 22:04:35.484475 163433 logs.go:123] Gathering logs for containerd ...
I0921 22:04:35.484488 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 22:04:35.545769 163433 logs.go:123] Gathering logs for container status ...
I0921 22:04:35.545802 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0921 22:04:35.571634 163433 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.571680 163433 out.go:239] *
*
W0921 22:04:35.571931 163433 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.571967 163433 out.go:239] *
*
W0921 22:04:35.572762 163433 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0921 22:04:35.575608 163433 out.go:177] X Problems detected in kubelet:
I0921 22:04:35.576957 163433 out.go:177] Sep 21 22:03:45 kubernetes-upgrade-20220921215522-10174 kubelet[12183]: E0921 22:03:45.374524 12183 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.578291 163433 out.go:177] Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12194]: E0921 22:03:46.123389 12194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.579626 163433 out.go:177] Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12205]: E0921 22:03:46.873433 12205 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.583058 163433 out.go:177]
W0921 22:04:35.584539 163433 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.584658 163433 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0921 22:04:35.584719 163433 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0921 22:04:35.586989 163433 out.go:177]
** /stderr **
version_upgrade_test.go:252: failed to upgrade with newest k8s version. args: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220921215522-10174 --memory=2200 --kubernetes-version=v1.25.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd : exit status 109
version_upgrade_test.go:255: (dbg) Run: kubectl --context kubernetes-upgrade-20220921215522-10174 version --output=json
version_upgrade_test.go:255: (dbg) Non-zero exit: kubectl --context kubernetes-upgrade-20220921215522-10174 version --output=json: exit status 1 (49.824513ms)
-- stdout --
{
"clientVersion": {
"major": "1",
"minor": "25",
"gitVersion": "v1.25.2",
"gitCommit": "5835544ca568b757a8ecae5c153f317e5736700e",
"gitTreeState": "clean",
"buildDate": "2022-09-21T14:33:49Z",
"goVersion": "go1.19.1",
"compiler": "gc",
"platform": "linux/amd64"
},
"kustomizeVersion": "v4.5.7"
}
-- /stdout --
** stderr **
The connection to the server 192.168.67.2:8443 was refused - did you specify the right host or port?
** /stderr **
version_upgrade_test.go:257: error running kubectl: exit status 1
panic.go:522: *** TestKubernetesUpgrade FAILED at 2022-09-21 22:04:36.011661338 +0000 UTC m=+2244.150113346
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestKubernetesUpgrade]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect kubernetes-upgrade-20220921215522-10174
helpers_test.go:235: (dbg) docker inspect kubernetes-upgrade-20220921215522-10174:
-- stdout --
[
{
"Id": "78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623",
"Created": "2022-09-21T21:55:31.510769348Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 163812,
"ExitCode": 0,
"Error": "",
"StartedAt": "2022-09-21T21:56:11.026038243Z",
"FinishedAt": "2022-09-21T21:56:09.280743853Z"
},
"Image": "sha256:5f58fddaff4349397c9f51a6b73926a9b118af22b4ccb4492e84c74d0b59dcd4",
"ResolvConfPath": "/var/lib/docker/containers/78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623/hostname",
"HostsPath": "/var/lib/docker/containers/78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623/hosts",
"LogPath": "/var/lib/docker/containers/78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623/78e6429f54d6491f63320f4836b927743b052e04b60ab2936b115556d1f08623-json.log",
"Name": "/kubernetes-upgrade-20220921215522-10174",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"kubernetes-upgrade-20220921215522-10174:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "kubernetes-upgrade-20220921215522-10174",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/996ea718c980edd62bcd4bc973111203322953381af43c6088f7819e8d6b15b2-init/diff:/var/lib/docker/overlay2/e464f9a636cb97832b8aa5685163afb26b14d037782877c3831e0f261b6fb12b/diff:/var/lib/docker/overlay2/6f2f230003f0711dfcff3390931e14f9abd39be1cd2b674079cb6a0fc99dab7f/diff:/var/lib/docker/overlay2/a539506727a1471a83b57694b6923a137699d9d47fe00601ffa27c34d63d17c2/diff:/var/lib/docker/overlay2/7dc48fda616494e9959c05191521a1f5bdf6be0555a35fc1d7ce85b87a0522ad/diff:/var/lib/docker/overlay2/6a1c06da85f8c03c9712693ca8a75d804e7b9f135a6b92e3929385cc791b0b3a/diff:/var/lib/docker/overlay2/d178ee6cf0c027c7943422dc71b38eb93ee131daede423f6cf283424eaebc517/diff:/var/lib/docker/overlay2/f4c8d81a89934a29db99d9ed4b4542a8882465ab2b6419ba4a3fe09214d25d9e/diff:/var/lib/docker/overlay2/4411575e37772974371f716b7c4abb3053138ddcaf53d883eb3116b4c3a37f78/diff:/var/lib/docker/overlay2/a3f8db0c0d790efcede2f41e470d293e280b022ff0bbb46c8bc25f190e3146fc/diff:/var/lib/docker/overlay2/a5bfd3
703378d6d5b2622598a13b4b2eb9203659c1ffff9aa93944ef8d20d22c/diff:/var/lib/docker/overlay2/3c24c9c8a37c1a488d12209df991b3857ccbc448a3329e46a8d48fc28ef81b83/diff:/var/lib/docker/overlay2/199771963b33bc809e912a6d428c556897f0ba04db2268e08695cb7ce0ee3bad/diff:/var/lib/docker/overlay2/4bc13570e456a5d261fbaf68140f2e5b2ea10c6683623858e16ee3ed1b117ef7/diff:/var/lib/docker/overlay2/9831fdfd44eff7e3f4780d10f5480f090b388735bb188e7a1e93251b7769d8f3/diff:/var/lib/docker/overlay2/28126fe31ddfbf99d4588c5ac750503b9b6cee8f2e00781941cff5f4323d1c76/diff:/var/lib/docker/overlay2/173221e8ff5f6ec22870429dc8bb01272ade638fe47275d1daa1ce07952c8c8c/diff:/var/lib/docker/overlay2/53bc2e4fa25c7c694765cf4d57e59552b26eed732ba59b8d74b221ea0ada7479/diff:/var/lib/docker/overlay2/0175a1cb7e1c92247b6cbac270da49d811d3508e193c1c2e47bef8cb769a47bf/diff:/var/lib/docker/overlay2/1651afeb98cf83af553f3e0a4dcb564af84cb440147702fddc0133cdae08e879/diff:/var/lib/docker/overlay2/64c0563be8f8902fe0e9d207f839be98da0e23d6d839403a7e5498f0468e6b75/diff:/var/lib/d
ocker/overlay2/c9ce1f7c22e681e0e279ce7656144b4c75e2d24f7297acad69e621f2756b75e8/diff:/var/lib/docker/overlay2/469e6b8bbc078383af3dc64c254906774ab56a833c4de1dd39b46bfa27fee3b0/diff:/var/lib/docker/overlay2/797536b923ada4261904843a51188063c401089eae5fec971f1dfb3c21d000a8/diff:/var/lib/docker/overlay2/9d5cbab97d75347795fc5814806362514e12ffc998b48411ecf1d23f980badf6/diff:/var/lib/docker/overlay2/8681f41e3df379cfdc8fd52b2e5837512b8e36a64c87fb159548a876d16e691d/diff:/var/lib/docker/overlay2/78ea1917a0aca968f65d796cbc2ee95d7d190a700496b9e47b29884fe13b1bec/diff:/var/lib/docker/overlay2/c3c231a74b7992f1fdc04b623c10d7791eab4fd97c803ed2610d595097c682c2/diff:/var/lib/docker/overlay2/f7b38a2a87d3958318f3a4b244e33d8096cd001a8fcb916eec022b0679382900/diff:/var/lib/docker/overlay2/1107be3397c1dfc460decaf307982d427cc431beb7938fb0e78f4f7cbc0afd3b/diff:/var/lib/docker/overlay2/59319d0c4cc381deff6e51ba1cc718b7cfd4fc557e74b8e83bd228afca501c8c/diff:/var/lib/docker/overlay2/9d031408a1ab9825246b7bba81db2596cb92e70fbedc70fba9be1d25320
8529f/diff:/var/lib/docker/overlay2/5800b717c814431baf3cc17520b099e7e011915cdeb46ed6360ec2f831a926ec/diff:/var/lib/docker/overlay2/6660d8dfc6dbb7f41f871ff7ec7e0fdfdb2ca3480141cdd4cce92869cb5382bf/diff:/var/lib/docker/overlay2/525a566b79110ff18e78880f2652b8d9cfcfbdbf3272fedb650933fcbbf869bd/diff:/var/lib/docker/overlay2/a0730aa8e0952fc661048d3a7f986ff246a5b2a29ad4e8195970bb407e3cecfc/diff:/var/lib/docker/overlay2/f9d25765ae914f5f013b5f46292f03c39d82f675d2102904f5fabecf07189337/diff:/var/lib/docker/overlay2/b7b34d126964ff60bc0fb625da7343e53e8bb4e77d5e72e33257c28b3d395ca3/diff:/var/lib/docker/overlay2/8514d6cc3775ebdd86f683a78503419bc97346c0059ac0d48563d2edec9727f0/diff:/var/lib/docker/overlay2/dd6f7bead5718d42040b4ee90ce09fa15720d04f3c58e2964b1d5b241bf5b7ed/diff:/var/lib/docker/overlay2/1a8adeb0355a42e9284e241b527930f4f6b7108d459c10550d5d0c4451f9e924/diff:/var/lib/docker/overlay2/703991449e0070d93945e4435da56144ce54e461e877e0b084144663d46b10f6/diff:/var/lib/docker/overlay2/65855711cfa445d374537e6268fd1bea2f62ea
bf2f590842933254b4755050dd/diff",
"MergedDir": "/var/lib/docker/overlay2/996ea718c980edd62bcd4bc973111203322953381af43c6088f7819e8d6b15b2/merged",
"UpperDir": "/var/lib/docker/overlay2/996ea718c980edd62bcd4bc973111203322953381af43c6088f7819e8d6b15b2/diff",
"WorkDir": "/var/lib/docker/overlay2/996ea718c980edd62bcd4bc973111203322953381af43c6088f7819e8d6b15b2/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "kubernetes-upgrade-20220921215522-10174",
"Source": "/var/lib/docker/volumes/kubernetes-upgrade-20220921215522-10174/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "kubernetes-upgrade-20220921215522-10174",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "kubernetes-upgrade-20220921215522-10174",
"name.minikube.sigs.k8s.io": "kubernetes-upgrade-20220921215522-10174",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "920af20b7ee11cedb3f2be56164b424fcbd75f33ad214de4ee14caa52403027b",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49344"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49343"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49340"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49342"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49341"
}
]
},
"SandboxKey": "/var/run/docker/netns/920af20b7ee1",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"kubernetes-upgrade-20220921215522-10174": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"78e6429f54d6",
"kubernetes-upgrade-20220921215522-10174"
],
"NetworkID": "454b31bb712a5040405e0ca56980290573f47fd969e65a7a2ee9d7f640b1457f",
"EndpointID": "764a8970c8218b00af2b0dad6965e745ec88b4a1dc8504ec47f82dfb39c1cbe9",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p kubernetes-upgrade-20220921215522-10174 -n kubernetes-upgrade-20220921215522-10174
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p kubernetes-upgrade-20220921215522-10174 -n kubernetes-upgrade-20220921215522-10174: exit status 2 (363.359067ms)
-- stdout --
Running
-- /stdout --
helpers_test.go:239: status error: exit status 2 (may be ok)
helpers_test.go:244: <<< TestKubernetesUpgrade FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestKubernetesUpgrade]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-amd64 -p kubernetes-upgrade-20220921215522-10174 logs -n 25
helpers_test.go:252: TestKubernetesUpgrade logs:
-- stdout --
*
* ==> Audit <==
* |---------|-----------------------------------------|-----------------------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|-----------------------------------------|-----------------------------------------|---------|---------|---------------------|---------------------|
| ssh | cert-options-20220921215754-10174 | cert-options-20220921215754-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | ssh openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p | cert-options-20220921215754-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | cert-options-20220921215754-10174 | | | | | |
| | -- sudo cat | | | | | |
| | /etc/kubernetes/admin.conf | | | | | |
| delete | -p | cert-options-20220921215754-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | cert-options-20220921215754-10174 | | | | | |
| start | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | --alsologtostderr | | | | | |
| | -v=1 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| start | -p auto-20220921215523-10174 | auto-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:59 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| pause | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | --alsologtostderr -v=5 | | | | | |
| unpause | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | --alsologtostderr -v=5 | | | | | |
| pause | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | --alsologtostderr -v=5 | | | | | |
| delete | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| | --alsologtostderr -v=5 | | | | | |
| profile | list --output json | minikube | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| delete | -p pause-20220921215721-10174 | pause-20220921215721-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:58 UTC |
| start | -p | kindnet-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:59 UTC |
| | kindnet-20220921215523-10174 | | | | | |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=kindnet --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| start | -p | cert-expiration-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 21:58 UTC | 21 Sep 22 21:59 UTC |
| | cert-expiration-20220921215524-10174 | | | | | |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p | cert-expiration-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 21:59 UTC |
| | cert-expiration-20220921215524-10174 | | | | | |
| start | -p cilium-20220921215524-10174 | cilium-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 22:01 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=cilium --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p auto-20220921215523-10174 | auto-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 21:59 UTC |
| | pgrep -a kubelet | | | | | |
| delete | -p auto-20220921215523-10174 | auto-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 21:59 UTC |
| start | -p calico-20220921215524-10174 | calico-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=calico --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p | kindnet-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 21:59 UTC |
| | kindnet-20220921215523-10174 | | | | | |
| | pgrep -a kubelet | | | | | |
| delete | -p | kindnet-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | 21 Sep 22 21:59 UTC |
| | kindnet-20220921215523-10174 | | | | | |
| start | -p | enable-default-cni-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 21:59 UTC | |
| | enable-default-cni-20220921215523-10174 | | | | | |
| | --memory=2048 --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --enable-default-cni=true | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p cilium-20220921215524-10174 | cilium-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 22:01 UTC | 21 Sep 22 22:01 UTC |
| | pgrep -a kubelet | | | | | |
| delete | -p cilium-20220921215524-10174 | cilium-20220921215524-10174 | jenkins | v1.27.0 | 21 Sep 22 22:01 UTC | 21 Sep 22 22:01 UTC |
| start | -p bridge-20220921215523-10174 | bridge-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 22:01 UTC | 21 Sep 22 22:01 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=bridge --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p bridge-20220921215523-10174 | bridge-20220921215523-10174 | jenkins | v1.27.0 | 21 Sep 22 22:01 UTC | 21 Sep 22 22:01 UTC |
| | pgrep -a kubelet | | | | | |
|---------|-----------------------------------------|-----------------------------------------|---------|---------|---------------------|---------------------|
*
* ==> Last Start <==
* Log file created at: 2022/09/21 22:01:20
Running on machine: ubuntu-20-agent-6
Binary: Built with gc go1.19.1 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0921 22:01:20.399992 215122 out.go:296] Setting OutFile to fd 1 ...
I0921 22:01:20.400128 215122 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:01:20.400143 215122 out.go:309] Setting ErrFile to fd 2...
I0921 22:01:20.400151 215122 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0921 22:01:20.400261 215122 root.go:333] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/bin
I0921 22:01:20.400815 215122 out.go:303] Setting JSON to false
I0921 22:01:20.402724 215122 start.go:115] hostinfo: {"hostname":"ubuntu-20-agent-6","uptime":2632,"bootTime":1663795049,"procs":1067,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1017-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0921 22:01:20.402792 215122 start.go:125] virtualization: kvm guest
I0921 22:01:20.405589 215122 out.go:177] * [bridge-20220921215523-10174] minikube v1.27.0 on Ubuntu 20.04 (kvm/amd64)
I0921 22:01:20.406830 215122 notify.go:214] Checking for updates...
I0921 22:01:20.408353 215122 out.go:177] - MINIKUBE_LOCATION=14995
I0921 22:01:20.409956 215122 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0921 22:01:20.411346 215122 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig
I0921 22:01:20.412867 215122 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube
I0921 22:01:20.414274 215122 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0921 22:01:20.415999 215122 config.go:180] Loaded profile config "calico-20220921215524-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 22:01:20.416089 215122 config.go:180] Loaded profile config "enable-default-cni-20220921215523-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 22:01:20.416168 215122 config.go:180] Loaded profile config "kubernetes-upgrade-20220921215522-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 22:01:20.416216 215122 driver.go:365] Setting default libvirt URI to qemu:///system
I0921 22:01:20.447353 215122 docker.go:137] docker version: linux-20.10.18
I0921 22:01:20.447449 215122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0921 22:01:20.543184 215122 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:56 OomKillDisable:true NGoroutines:57 SystemTime:2022-09-21 22:01:20.468243338 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1017-gcp OperatingSystem:Ubuntu 20.04.5 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33660669952 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:20.10.18 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 Expected:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.10.2] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0921 22:01:20.543395 215122 docker.go:254] overlay module found
I0921 22:01:20.545758 215122 out.go:177] * Using the docker driver based on user configuration
I0921 22:01:20.547135 215122 start.go:284] selected driver: docker
I0921 22:01:20.547158 215122 start.go:808] validating driver "docker" against <nil>
I0921 22:01:20.547185 215122 start.go:819] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0921 22:01:20.548120 215122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0921 22:01:20.642575 215122 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:56 OomKillDisable:true NGoroutines:57 SystemTime:2022-09-21 22:01:20.569196544 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1017-gcp OperatingSystem:Ubuntu 20.04.5 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33660669952 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-6 Labels:[] ExperimentalBuild:false ServerVersion:20.10.18 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 Expected:9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6} RuncCommit:{ID:v1.1.4-0-g5fd4c4d Expected:v1.1.4-0-g5fd4c4d} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.9.1-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.10.2] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0921 22:01:20.642729 215122 start_flags.go:302] no existing cluster config was found, will generate one from the flags
I0921 22:01:20.642899 215122 start_flags.go:867] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0921 22:01:20.644995 215122 out.go:177] * Using Docker driver with root privileges
I0921 22:01:20.646356 215122 cni.go:95] Creating CNI manager for "bridge"
I0921 22:01:20.646377 215122 start_flags.go:311] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0921 22:01:20.646385 215122 start_flags.go:316] config:
{Name:bridge-20220921215523-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.25.2 ClusterName:bridge-20220921215523-10174 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:cont
ainerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet}
I0921 22:01:20.647870 215122 out.go:177] * Starting control plane node bridge-20220921215523-10174 in cluster bridge-20220921215523-10174
I0921 22:01:20.649143 215122 cache.go:120] Beginning downloading kic base image for docker with containerd
I0921 22:01:20.650480 215122 out.go:177] * Pulling base image ...
I0921 22:01:20.651764 215122 preload.go:132] Checking if preload exists for k8s version v1.25.2 and runtime containerd
I0921 22:01:20.651804 215122 preload.go:148] Found local preload: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4
I0921 22:01:20.651815 215122 cache.go:57] Caching tarball of preloaded images
I0921 22:01:20.651860 215122 image.go:75] Checking for gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c in local docker daemon
I0921 22:01:20.652066 215122 preload.go:174] Found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4 in cache, skipping download
I0921 22:01:20.652088 215122 cache.go:60] Finished verifying existence of preloaded tar for v1.25.2 on containerd
I0921 22:01:20.652199 215122 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/config.json ...
I0921 22:01:20.652225 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/config.json: {Name:mk27c183536830420bb2d6132b593804695deea4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:20.677363 215122 image.go:79] Found gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c in local docker daemon, skipping pull
I0921 22:01:20.677390 215122 cache.go:142] gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c exists in daemon, skipping load
I0921 22:01:20.677404 215122 cache.go:208] Successfully downloaded all kic artifacts
I0921 22:01:20.677444 215122 start.go:364] acquiring machines lock for bridge-20220921215523-10174: {Name:mkc981b7e955f66b2bf99465d3282ba0dad6f5bb Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0921 22:01:20.677563 215122 start.go:368] acquired machines lock for "bridge-20220921215523-10174" in 99.083µs
I0921 22:01:20.677587 215122 start.go:93] Provisioning new machine with config: &{Name:bridge-20220921215523-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.25.2 ClusterName:bridge-20220921215523-10174 Namespace:default APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.25.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOpti
mizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet} &{Name: IP: Port:8443 KubernetesVersion:v1.25.2 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0921 22:01:20.677667 215122 start.go:125] createHost starting for "" (driver="docker")
I0921 22:01:20.796308 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:22.796445 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:21.788431 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:24.282795 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:20.680580 215122 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...
I0921 22:01:20.680792 215122 start.go:159] libmachine.API.Create for "bridge-20220921215523-10174" (driver="docker")
I0921 22:01:20.680826 215122 client.go:168] LocalClient.Create starting
I0921 22:01:20.680895 215122 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem
I0921 22:01:20.680928 215122 main.go:134] libmachine: Decoding PEM data...
I0921 22:01:20.680945 215122 main.go:134] libmachine: Parsing certificate...
I0921 22:01:20.681015 215122 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem
I0921 22:01:20.681038 215122 main.go:134] libmachine: Decoding PEM data...
I0921 22:01:20.681052 215122 main.go:134] libmachine: Parsing certificate...
I0921 22:01:20.681458 215122 cli_runner.go:164] Run: docker network inspect bridge-20220921215523-10174 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0921 22:01:20.704178 215122 cli_runner.go:211] docker network inspect bridge-20220921215523-10174 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0921 22:01:20.704263 215122 network_create.go:272] running [docker network inspect bridge-20220921215523-10174] to gather additional debugging logs...
I0921 22:01:20.704302 215122 cli_runner.go:164] Run: docker network inspect bridge-20220921215523-10174
W0921 22:01:20.729823 215122 cli_runner.go:211] docker network inspect bridge-20220921215523-10174 returned with exit code 1
I0921 22:01:20.729859 215122 network_create.go:275] error running [docker network inspect bridge-20220921215523-10174]: docker network inspect bridge-20220921215523-10174: exit status 1
stdout:
[]
stderr:
Error: No such network: bridge-20220921215523-10174
I0921 22:01:20.729877 215122 network_create.go:277] output of [docker network inspect bridge-20220921215523-10174]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: bridge-20220921215523-10174
** /stderr **
I0921 22:01:20.729920 215122 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0921 22:01:20.755762 215122 network.go:241] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName:br-b7c23e57d062 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:a3:39:9d:03}}
I0921 22:01:20.756906 215122 network.go:241] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 Interface:{IfaceName:br-bfa8cb3d5f9b IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:8c:39:36:0c}}
I0921 22:01:20.757830 215122 network.go:241] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 Interface:{IfaceName:br-454b31bb712a IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:f9:25:10:f3}}
I0921 22:01:20.758735 215122 network.go:290] reserving subnet 192.168.76.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.76.0:0xc0004f8098] misses:0}
I0921 22:01:20.758771 215122 network.go:236] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0921 22:01:20.758784 215122 network_create.go:115] attempt to create docker network bridge-20220921215523-10174 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0921 22:01:20.758842 215122 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=bridge-20220921215523-10174 bridge-20220921215523-10174
I0921 22:01:20.827257 215122 network_create.go:99] docker network bridge-20220921215523-10174 192.168.76.0/24 created
I0921 22:01:20.827297 215122 kic.go:106] calculated static IP "192.168.76.2" for the "bridge-20220921215523-10174" container
I0921 22:01:20.827361 215122 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0921 22:01:20.854549 215122 cli_runner.go:164] Run: docker volume create bridge-20220921215523-10174 --label name.minikube.sigs.k8s.io=bridge-20220921215523-10174 --label created_by.minikube.sigs.k8s.io=true
I0921 22:01:20.879039 215122 oci.go:103] Successfully created a docker volume bridge-20220921215523-10174
I0921 22:01:20.879114 215122 cli_runner.go:164] Run: docker run --rm --name bridge-20220921215523-10174-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=bridge-20220921215523-10174 --entrypoint /usr/bin/test -v bridge-20220921215523-10174:/var gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c -d /var/lib
I0921 22:01:21.444486 215122 oci.go:107] Successfully prepared a docker volume bridge-20220921215523-10174
I0921 22:01:21.444562 215122 preload.go:132] Checking if preload exists for k8s version v1.25.2 and runtime containerd
I0921 22:01:21.444582 215122 kic.go:179] Starting extracting preloaded images to volume ...
I0921 22:01:21.444637 215122 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v bridge-20220921215523-10174:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c -I lz4 -xf /preloaded.tar -C /extractDir
I0921 22:01:25.295077 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:27.296268 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:26.283052 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:28.284721 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:27.967769 215122 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.25.2-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v bridge-20220921215523-10174:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c -I lz4 -xf /preloaded.tar -C /extractDir: (6.523042715s)
I0921 22:01:27.967801 215122 kic.go:188] duration metric: took 6.523216 seconds to extract preloaded images to volume
W0921 22:01:27.967925 215122 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0921 22:01:27.968011 215122 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0921 22:01:28.061462 215122 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname bridge-20220921215523-10174 --name bridge-20220921215523-10174 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=bridge-20220921215523-10174 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=bridge-20220921215523-10174 --network bridge-20220921215523-10174 --ip 192.168.76.2 --volume bridge-20220921215523-10174:/var --security-opt apparmor=unconfined --memory=2048mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c
I0921 22:01:28.458187 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Running}}
I0921 22:01:28.485826 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:28.511954 215122 cli_runner.go:164] Run: docker exec bridge-20220921215523-10174 stat /var/lib/dpkg/alternatives/iptables
I0921 22:01:28.572618 215122 oci.go:144] the created container "bridge-20220921215523-10174" has a running status.
I0921 22:01:28.572655 215122 kic.go:210] Creating ssh key for kic: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa...
I0921 22:01:28.687866 215122 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0921 22:01:28.760716 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:28.800826 215122 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0921 22:01:28.800848 215122 kic_runner.go:114] Args: [docker exec --privileged bridge-20220921215523-10174 chown docker:docker /home/docker/.ssh/authorized_keys]
I0921 22:01:28.883124 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:28.913807 215122 machine.go:88] provisioning docker machine ...
I0921 22:01:28.913853 215122 ubuntu.go:169] provisioning hostname "bridge-20220921215523-10174"
I0921 22:01:28.913911 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:28.944411 215122 main.go:134] libmachine: Using SSH client type: native
I0921 22:01:28.944625 215122 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7ec520] 0x7ef6a0 <nil> [] 0s} 127.0.0.1 49393 <nil> <nil>}
I0921 22:01:28.944649 215122 main.go:134] libmachine: About to run SSH command:
sudo hostname bridge-20220921215523-10174 && echo "bridge-20220921215523-10174" | sudo tee /etc/hostname
I0921 22:01:29.104862 215122 main.go:134] libmachine: SSH cmd err, output: <nil>: bridge-20220921215523-10174
I0921 22:01:29.104937 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.131408 215122 main.go:134] libmachine: Using SSH client type: native
I0921 22:01:29.131557 215122 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7ec520] 0x7ef6a0 <nil> [] 0s} 127.0.0.1 49393 <nil> <nil>}
I0921 22:01:29.131579 215122 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\sbridge-20220921215523-10174' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 bridge-20220921215523-10174/g' /etc/hosts;
else
echo '127.0.1.1 bridge-20220921215523-10174' | sudo tee -a /etc/hosts;
fi
fi
I0921 22:01:29.259477 215122 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0921 22:01:29.259512 215122 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/c
erts/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube}
I0921 22:01:29.259536 215122 ubuntu.go:177] setting up certificates
I0921 22:01:29.259544 215122 provision.go:83] configureAuth start
I0921 22:01:29.259591 215122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220921215523-10174
I0921 22:01:29.283351 215122 provision.go:138] copyHostCerts
I0921 22:01:29.283422 215122 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem, removing ...
I0921 22:01:29.283434 215122 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem
I0921 22:01:29.283513 215122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.pem (1078 bytes)
I0921 22:01:29.283627 215122 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem, removing ...
I0921 22:01:29.283645 215122 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem
I0921 22:01:29.283684 215122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/cert.pem (1123 bytes)
I0921 22:01:29.283807 215122 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem, removing ...
I0921 22:01:29.283823 215122 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem
I0921 22:01:29.283858 215122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/key.pem (1675 bytes)
I0921 22:01:29.283925 215122 provision.go:112] generating server cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem org=jenkins.bridge-20220921215523-10174 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube bridge-20220921215523-10174]
I0921 22:01:29.347391 215122 provision.go:172] copyRemoteCerts
I0921 22:01:29.347446 215122 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0921 22:01:29.347478 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.373584 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:29.467228 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0921 22:01:29.484247 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0921 22:01:29.501261 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/server.pem --> /etc/docker/server.pem (1257 bytes)
I0921 22:01:29.518060 215122 provision.go:86] duration metric: configureAuth took 258.507279ms
I0921 22:01:29.518084 215122 ubuntu.go:193] setting minikube options for container-runtime
I0921 22:01:29.518230 215122 config.go:180] Loaded profile config "bridge-20220921215523-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 22:01:29.518241 215122 machine.go:91] provisioned docker machine in 604.41268ms
I0921 22:01:29.518246 215122 client.go:171] LocalClient.Create took 8.837412157s
I0921 22:01:29.518260 215122 start.go:167] duration metric: libmachine.API.Create for "bridge-20220921215523-10174" took 8.837469774s
I0921 22:01:29.518270 215122 start.go:300] post-start starting for "bridge-20220921215523-10174" (driver="docker")
I0921 22:01:29.518276 215122 start.go:328] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0921 22:01:29.518315 215122 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0921 22:01:29.518356 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.542683 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:29.635262 215122 ssh_runner.go:195] Run: cat /etc/os-release
I0921 22:01:29.638009 215122 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0921 22:01:29.638037 215122 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0921 22:01:29.638048 215122 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0921 22:01:29.638054 215122 info.go:137] Remote host: Ubuntu 20.04.5 LTS
I0921 22:01:29.638064 215122 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/addons for local assets ...
I0921 22:01:29.638117 215122 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files for local assets ...
I0921 22:01:29.638196 215122 filesync.go:149] local asset: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem -> 101742.pem in /etc/ssl/certs
I0921 22:01:29.638279 215122 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0921 22:01:29.644987 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem --> /etc/ssl/certs/101742.pem (1708 bytes)
I0921 22:01:29.662057 215122 start.go:303] post-start completed in 143.775945ms
I0921 22:01:29.662374 215122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220921215523-10174
I0921 22:01:29.686087 215122 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/config.json ...
I0921 22:01:29.686359 215122 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0921 22:01:29.686401 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.712755 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:29.800243 215122 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0921 22:01:29.804306 215122 start.go:128] duration metric: createHost completed in 9.126616648s
I0921 22:01:29.804329 215122 start.go:83] releasing machines lock for "bridge-20220921215523-10174", held for 9.126753984s
I0921 22:01:29.804398 215122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220921215523-10174
I0921 22:01:29.828357 215122 ssh_runner.go:195] Run: systemctl --version
I0921 22:01:29.828401 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.828440 215122 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0921 22:01:29.828524 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:29.853434 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:29.854844 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:29.971953 215122 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0921 22:01:29.981882 215122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0921 22:01:29.991059 215122 docker.go:188] disabling docker service ...
I0921 22:01:29.991132 215122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0921 22:01:30.007744 215122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0921 22:01:30.016621 215122 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0921 22:01:30.108331 215122 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0921 22:01:30.182311 215122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0921 22:01:30.191307 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0921 22:01:30.203641 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*sandbox_image = .*$|sandbox_image = "registry.k8s.io/pause:3.8"|' -i /etc/containerd/config.toml"
I0921 22:01:30.211170 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*restrict_oom_score_adj = .*$|restrict_oom_score_adj = false|' -i /etc/containerd/config.toml"
I0921 22:01:30.218912 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*SystemdCgroup = .*$|SystemdCgroup = false|' -i /etc/containerd/config.toml"
I0921 22:01:30.226199 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*conf_dir = .*$|conf_dir = "/etc/cni/net.d"|' -i /etc/containerd/config.toml"
I0921 22:01:30.233859 215122 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0921 22:01:30.240077 215122 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0921 22:01:30.246033 215122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0921 22:01:30.316300 215122 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0921 22:01:30.388357 215122 start.go:450] Will wait 60s for socket path /run/containerd/containerd.sock
I0921 22:01:30.388430 215122 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0921 22:01:30.391887 215122 start.go:471] Will wait 60s for crictl version
I0921 22:01:30.391949 215122 ssh_runner.go:195] Run: sudo crictl version
I0921 22:01:30.419762 215122 start.go:480] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.8
RuntimeApiVersion: v1alpha2
I0921 22:01:30.419824 215122 ssh_runner.go:195] Run: containerd --version
I0921 22:01:30.449680 215122 ssh_runner.go:195] Run: containerd --version
I0921 22:01:30.481291 215122 out.go:177] * Preparing Kubernetes v1.25.2 on containerd 1.6.8 ...
I0921 22:01:30.482875 215122 cli_runner.go:164] Run: docker network inspect bridge-20220921215523-10174 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0921 22:01:30.507780 215122 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0921 22:01:30.510930 215122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0921 22:01:30.520463 215122 preload.go:132] Checking if preload exists for k8s version v1.25.2 and runtime containerd
I0921 22:01:30.520525 215122 ssh_runner.go:195] Run: sudo crictl images --output json
I0921 22:01:30.544218 215122 containerd.go:553] all images are preloaded for containerd runtime.
I0921 22:01:30.544244 215122 containerd.go:467] Images already preloaded, skipping extraction
I0921 22:01:30.544298 215122 ssh_runner.go:195] Run: sudo crictl images --output json
I0921 22:01:30.567882 215122 containerd.go:553] all images are preloaded for containerd runtime.
I0921 22:01:30.567905 215122 cache_images.go:84] Images are preloaded, skipping loading
I0921 22:01:30.567945 215122 ssh_runner.go:195] Run: sudo crictl info
I0921 22:01:30.591632 215122 cni.go:95] Creating CNI manager for "bridge"
I0921 22:01:30.591658 215122 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0921 22:01:30.591672 215122 kubeadm.go:156] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.25.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:bridge-20220921215523-10174 NodeName:bridge-20220921215523-10174 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/cer
ts/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false}
I0921 22:01:30.591815 215122 kubeadm.go:161] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "bridge-20220921215523-10174"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.25.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0921 22:01:30.591902 215122 kubeadm.go:962] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.25.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=bridge-20220921215523-10174 --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2 --runtime-request-timeout=15m
[Install]
config:
{KubernetesVersion:v1.25.2 ClusterName:bridge-20220921215523-10174 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:}
I0921 22:01:30.591948 215122 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.25.2
I0921 22:01:30.599049 215122 binaries.go:44] Found k8s binaries, skipping transfer
I0921 22:01:30.599104 215122 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0921 22:01:30.605713 215122 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (520 bytes)
I0921 22:01:30.618251 215122 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0921 22:01:30.630673 215122 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2056 bytes)
I0921 22:01:30.642757 215122 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0921 22:01:30.645493 215122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0921 22:01:30.653938 215122 certs.go:54] Setting up /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174 for IP: 192.168.76.2
I0921 22:01:30.654035 215122 certs.go:182] skipping minikubeCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.key
I0921 22:01:30.654075 215122 certs.go:182] skipping proxyClientCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.key
I0921 22:01:30.654126 215122 certs.go:302] generating minikube-user signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.key
I0921 22:01:30.654140 215122 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.crt with IP's: []
I0921 22:01:30.717387 215122 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.crt ...
I0921 22:01:30.717416 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.crt: {Name:mk43aabc1da4bac55bcffe901401521f4aea0e91 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:30.717609 215122 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.key ...
I0921 22:01:30.717627 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/client.key: {Name:mke49682624e380c4693e416e37baf3a4bca8b82 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:30.717729 215122 certs.go:302] generating minikube signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key.31bdca25
I0921 22:01:30.717746 215122 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0921 22:01:30.903254 215122 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt.31bdca25 ...
I0921 22:01:30.903292 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt.31bdca25: {Name:mk6bf35dba38a8b20d945a06965afc655853b287 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:30.903523 215122 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key.31bdca25 ...
I0921 22:01:30.903551 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key.31bdca25: {Name:mke9ddfe2a0e862817321007cda24dba740f7c0e Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:30.903705 215122 certs.go:320] copying /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt
I0921 22:01:30.903836 215122 certs.go:324] copying /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key
I0921 22:01:30.903894 215122 certs.go:302] generating aggregator signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.key
I0921 22:01:30.903911 215122 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.crt with IP's: []
I0921 22:01:31.104376 215122 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.crt ...
I0921 22:01:31.104418 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.crt: {Name:mk3d1852a42d99d3174c0629c6f8a1b67ab3049f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:31.104623 215122 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.key ...
I0921 22:01:31.104638 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.key: {Name:mka948a198a0065642897187120a9ce1a1f48079 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:31.104811 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174.pem (1338 bytes)
W0921 22:01:31.104849 215122 certs.go:384] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174_empty.pem, impossibly tiny 0 bytes
I0921 22:01:31.104863 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca-key.pem (1679 bytes)
I0921 22:01:31.104887 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/ca.pem (1078 bytes)
I0921 22:01:31.104912 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/cert.pem (1123 bytes)
I0921 22:01:31.104934 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/key.pem (1675 bytes)
I0921 22:01:31.104971 215122 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem (1708 bytes)
I0921 22:01:31.105464 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0921 22:01:31.123583 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0921 22:01:31.140587 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0921 22:01:31.157520 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/bridge-20220921215523-10174/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0921 22:01:31.174427 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0921 22:01:31.191183 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0921 22:01:31.208210 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0921 22:01:31.224755 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0921 22:01:31.242006 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/files/etc/ssl/certs/101742.pem --> /usr/share/ca-certificates/101742.pem (1708 bytes)
I0921 22:01:31.259325 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0921 22:01:31.275913 215122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/certs/10174.pem --> /usr/share/ca-certificates/10174.pem (1338 bytes)
I0921 22:01:31.293352 215122 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0921 22:01:31.306447 215122 ssh_runner.go:195] Run: openssl version
I0921 22:01:31.311131 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0921 22:01:31.318135 215122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0921 22:01:31.321067 215122 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Sep 21 21:28 /usr/share/ca-certificates/minikubeCA.pem
I0921 22:01:31.321106 215122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0921 22:01:31.325758 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0921 22:01:31.332811 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/10174.pem && ln -fs /usr/share/ca-certificates/10174.pem /etc/ssl/certs/10174.pem"
I0921 22:01:31.340266 215122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/10174.pem
I0921 22:01:31.343153 215122 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Sep 21 21:32 /usr/share/ca-certificates/10174.pem
I0921 22:01:31.343208 215122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/10174.pem
I0921 22:01:31.347987 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/10174.pem /etc/ssl/certs/51391683.0"
I0921 22:01:31.354868 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/101742.pem && ln -fs /usr/share/ca-certificates/101742.pem /etc/ssl/certs/101742.pem"
I0921 22:01:31.362081 215122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/101742.pem
I0921 22:01:31.364960 215122 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Sep 21 21:32 /usr/share/ca-certificates/101742.pem
I0921 22:01:31.364997 215122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/101742.pem
I0921 22:01:31.369540 215122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/101742.pem /etc/ssl/certs/3ec20f2e.0"
I0921 22:01:31.376545 215122 kubeadm.go:396] StartCluster: {Name:bridge-20220921215523-10174 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.25.2 ClusterName:bridge-20220921215523-10174 Namespace:default APIServerName:minikubeCA APIServerNames:[] A
PIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.25.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:
false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath:/opt/socket_vmnet/bin/socket_vmnet_client SocketVMnetPath:/var/run/socket_vmnet}
I0921 22:01:31.376625 215122 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0921 22:01:31.376654 215122 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0921 22:01:31.400010 215122 cri.go:87] found id: ""
I0921 22:01:31.400057 215122 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0921 22:01:31.406974 215122 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0921 22:01:31.413564 215122 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0921 22:01:31.413604 215122 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0921 22:01:31.420012 215122 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0921 22:01:31.420051 215122 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0921 22:01:31.460421 215122 kubeadm.go:317] [init] Using Kubernetes version: v1.25.2
I0921 22:01:31.460512 215122 kubeadm.go:317] [preflight] Running pre-flight checks
I0921 22:01:31.487776 215122 kubeadm.go:317] [preflight] The system verification failed. Printing the output from the verification:
I0921 22:01:31.487888 215122 kubeadm.go:317] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
I0921 22:01:31.487954 215122 kubeadm.go:317] [0;37mOS[0m: [0;32mLinux[0m
I0921 22:01:31.488026 215122 kubeadm.go:317] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0921 22:01:31.488104 215122 kubeadm.go:317] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0921 22:01:31.488168 215122 kubeadm.go:317] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0921 22:01:31.488235 215122 kubeadm.go:317] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0921 22:01:31.488291 215122 kubeadm.go:317] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0921 22:01:31.488353 215122 kubeadm.go:317] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0921 22:01:31.488423 215122 kubeadm.go:317] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0921 22:01:31.488515 215122 kubeadm.go:317] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0921 22:01:31.488577 215122 kubeadm.go:317] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0921 22:01:31.552193 215122 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0921 22:01:31.552354 215122 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0921 22:01:31.552493 215122 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0921 22:01:31.673966 215122 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0921 22:01:31.677138 215122 out.go:204] - Generating certificates and keys ...
I0921 22:01:31.677275 215122 kubeadm.go:317] [certs] Using existing ca certificate authority
I0921 22:01:31.677362 215122 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0921 22:01:31.927942 215122 kubeadm.go:317] [certs] Generating "apiserver-kubelet-client" certificate and key
I0921 22:01:32.076422 215122 kubeadm.go:317] [certs] Generating "front-proxy-ca" certificate and key
I0921 22:01:32.186739 215122 kubeadm.go:317] [certs] Generating "front-proxy-client" certificate and key
I0921 22:01:32.514364 215122 kubeadm.go:317] [certs] Generating "etcd/ca" certificate and key
I0921 22:01:32.655578 215122 kubeadm.go:317] [certs] Generating "etcd/server" certificate and key
I0921 22:01:32.655808 215122 kubeadm.go:317] [certs] etcd/server serving cert is signed for DNS names [bridge-20220921215523-10174 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0921 22:01:32.759413 215122 kubeadm.go:317] [certs] Generating "etcd/peer" certificate and key
I0921 22:01:32.759668 215122 kubeadm.go:317] [certs] etcd/peer serving cert is signed for DNS names [bridge-20220921215523-10174 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0921 22:01:32.896993 215122 kubeadm.go:317] [certs] Generating "etcd/healthcheck-client" certificate and key
I0921 22:01:33.198251 215122 kubeadm.go:317] [certs] Generating "apiserver-etcd-client" certificate and key
I0921 22:01:33.422103 215122 kubeadm.go:317] [certs] Generating "sa" key and public key
I0921 22:01:33.422230 215122 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0921 22:01:33.656334 215122 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0921 22:01:33.773030 215122 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0921 22:01:33.908387 215122 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0921 22:01:34.027812 215122 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0921 22:01:34.039137 215122 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0921 22:01:34.040753 215122 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0921 22:01:34.040864 215122 kubeadm.go:317] [kubelet-start] Starting the kubelet
I0921 22:01:34.127129 215122 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0921 22:01:29.794678 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:31.795317 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:34.296071 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:30.783006 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:32.786124 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:34.130675 215122 out.go:204] - Booting up control plane ...
I0921 22:01:34.130824 215122 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0921 22:01:34.130931 215122 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0921 22:01:34.131032 215122 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0921 22:01:34.131566 215122 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0921 22:01:34.133233 215122 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0921 22:01:36.298594 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:38.800929 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:35.284259 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:37.286508 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:39.783829 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:40.135581 215122 kubeadm.go:317] [apiclient] All control plane components are healthy after 6.002264 seconds
I0921 22:01:40.135792 215122 kubeadm.go:317] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0921 22:01:40.143652 215122 kubeadm.go:317] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0921 22:01:40.658118 215122 kubeadm.go:317] [upload-certs] Skipping phase. Please see --upload-certs
I0921 22:01:40.658320 215122 kubeadm.go:317] [mark-control-plane] Marking the node bridge-20220921215523-10174 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0921 22:01:41.165185 215122 kubeadm.go:317] [bootstrap-token] Using token: aqku34.qf5z1n8skf8o3ddv
I0921 22:01:41.167890 215122 out.go:204] - Configuring RBAC rules ...
I0921 22:01:41.168031 215122 kubeadm.go:317] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0921 22:01:41.169792 215122 kubeadm.go:317] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0921 22:01:41.174042 215122 kubeadm.go:317] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0921 22:01:41.176023 215122 kubeadm.go:317] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0921 22:01:41.177813 215122 kubeadm.go:317] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0921 22:01:41.179505 215122 kubeadm.go:317] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0921 22:01:41.186227 215122 kubeadm.go:317] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0921 22:01:41.390399 215122 kubeadm.go:317] [addons] Applied essential addon: CoreDNS
I0921 22:01:41.581750 215122 kubeadm.go:317] [addons] Applied essential addon: kube-proxy
I0921 22:01:41.583080 215122 kubeadm.go:317]
I0921 22:01:41.583170 215122 kubeadm.go:317] Your Kubernetes control-plane has initialized successfully!
I0921 22:01:41.583183 215122 kubeadm.go:317]
I0921 22:01:41.583281 215122 kubeadm.go:317] To start using your cluster, you need to run the following as a regular user:
I0921 22:01:41.583311 215122 kubeadm.go:317]
I0921 22:01:41.583351 215122 kubeadm.go:317] mkdir -p $HOME/.kube
I0921 22:01:41.583432 215122 kubeadm.go:317] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0921 22:01:41.583512 215122 kubeadm.go:317] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0921 22:01:41.583545 215122 kubeadm.go:317]
I0921 22:01:41.583629 215122 kubeadm.go:317] Alternatively, if you are the root user, you can run:
I0921 22:01:41.583641 215122 kubeadm.go:317]
I0921 22:01:41.583697 215122 kubeadm.go:317] export KUBECONFIG=/etc/kubernetes/admin.conf
I0921 22:01:41.583707 215122 kubeadm.go:317]
I0921 22:01:41.583848 215122 kubeadm.go:317] You should now deploy a pod network to the cluster.
I0921 22:01:41.583951 215122 kubeadm.go:317] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0921 22:01:41.584038 215122 kubeadm.go:317] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0921 22:01:41.584048 215122 kubeadm.go:317]
I0921 22:01:41.584138 215122 kubeadm.go:317] You can now join any number of control-plane nodes by copying certificate authorities
I0921 22:01:41.584233 215122 kubeadm.go:317] and service account keys on each node and then running the following as root:
I0921 22:01:41.584244 215122 kubeadm.go:317]
I0921 22:01:41.584332 215122 kubeadm.go:317] kubeadm join control-plane.minikube.internal:8443 --token aqku34.qf5z1n8skf8o3ddv \
I0921 22:01:41.584460 215122 kubeadm.go:317] --discovery-token-ca-cert-hash sha256:b419e756666ce2e30bdac31039f21ad7242d26e2b9c03a55c5bc6fe8dcfddcf7 \
I0921 22:01:41.584500 215122 kubeadm.go:317] --control-plane
I0921 22:01:41.584514 215122 kubeadm.go:317]
I0921 22:01:41.584614 215122 kubeadm.go:317] Then you can join any number of worker nodes by running the following on each as root:
I0921 22:01:41.584624 215122 kubeadm.go:317]
I0921 22:01:41.584710 215122 kubeadm.go:317] kubeadm join control-plane.minikube.internal:8443 --token aqku34.qf5z1n8skf8o3ddv \
I0921 22:01:41.584824 215122 kubeadm.go:317] --discovery-token-ca-cert-hash sha256:b419e756666ce2e30bdac31039f21ad7242d26e2b9c03a55c5bc6fe8dcfddcf7
I0921 22:01:41.587861 215122 kubeadm.go:317] W0921 22:01:31.452977 737 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
I0921 22:01:41.588099 215122 kubeadm.go:317] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
I0921 22:01:41.588226 215122 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0921 22:01:41.588268 215122 cni.go:95] Creating CNI manager for "bridge"
I0921 22:01:41.590760 215122 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...
I0921 22:01:41.295390 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:43.796834 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:41.787421 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:44.283230 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:41.592344 215122 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d
I0921 22:01:41.601450 215122 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)
I0921 22:01:41.676889 215122 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0921 22:01:41.676983 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:41.676984 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl label nodes minikube.k8s.io/version=v1.27.0 minikube.k8s.io/commit=937c68716dfaac5b5ffa3b6655158d5d3472b8c4 minikube.k8s.io/name=bridge-20220921215523-10174 minikube.k8s.io/updated_at=2022_09_21T22_01_41_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:41.686757 215122 ops.go:34] apiserver oom_adj: -16
I0921 22:01:41.811777 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:42.436985 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:42.936471 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:43.437449 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:43.936528 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:44.436919 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:44.936555 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:45.796921 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:48.296174 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:46.785358 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:49.283231 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:45.437123 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:45.936578 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:46.436936 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:46.936921 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:47.436757 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:47.936741 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:48.436653 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:48.937368 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:49.436939 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:49.936924 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:50.296253 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:52.795419 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:51.784647 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:53.785717 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:50.437167 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:50.936941 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:51.436582 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:51.936930 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:52.436939 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:52.937410 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:53.436577 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:53.936955 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:54.436802 215122 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.25.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0921 22:01:54.509936 215122 kubeadm.go:1067] duration metric: took 12.833017338s to wait for elevateKubeSystemPrivileges.
I0921 22:01:54.509973 215122 kubeadm.go:398] StartCluster complete in 23.13343596s
I0921 22:01:54.509994 215122 settings.go:142] acquiring lock: {Name:mk2e017b0c75e33bad2b3a546d00214b38a21694 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:54.510181 215122 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig
I0921 22:01:54.512644 215122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/kubeconfig: {Name:mkdec5faf1bb182b50a3cd5458b352f38075dc15 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0921 22:01:55.033091 215122 kapi.go:244] deployment "coredns" in namespace "kube-system" and context "bridge-20220921215523-10174" rescaled to 1
I0921 22:01:55.033167 215122 start.go:211] Will wait 5m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.25.2 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0921 22:01:55.036011 215122 out.go:177] * Verifying Kubernetes components...
I0921 22:01:55.033220 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0921 22:01:55.033256 215122 addons.go:412] enableAddons start: toEnable=map[], additional=[]
I0921 22:01:55.033396 215122 config.go:180] Loaded profile config "bridge-20220921215523-10174": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.25.2
I0921 22:01:55.037672 215122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0921 22:01:55.037838 215122 addons.go:65] Setting storage-provisioner=true in profile "bridge-20220921215523-10174"
I0921 22:01:55.037857 215122 addons.go:153] Setting addon storage-provisioner=true in "bridge-20220921215523-10174"
W0921 22:01:55.037864 215122 addons.go:162] addon storage-provisioner should already be in state true
I0921 22:01:55.037932 215122 host.go:66] Checking if "bridge-20220921215523-10174" exists ...
I0921 22:01:55.038499 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:55.038568 215122 addons.go:65] Setting default-storageclass=true in profile "bridge-20220921215523-10174"
I0921 22:01:55.038594 215122 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "bridge-20220921215523-10174"
I0921 22:01:55.038875 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:55.068033 215122 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0921 22:01:55.069630 215122 addons.go:345] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0921 22:01:55.069649 215122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0921 22:01:55.069700 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:55.076739 215122 addons.go:153] Setting addon default-storageclass=true in "bridge-20220921215523-10174"
W0921 22:01:55.076769 215122 addons.go:162] addon default-storageclass should already be in state true
I0921 22:01:55.076798 215122 host.go:66] Checking if "bridge-20220921215523-10174" exists ...
I0921 22:01:55.077281 215122 cli_runner.go:164] Run: docker container inspect bridge-20220921215523-10174 --format={{.State.Status}}
I0921 22:01:55.111072 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:55.115635 215122 addons.go:345] installing /etc/kubernetes/addons/storageclass.yaml
I0921 22:01:55.115663 215122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0921 22:01:55.115836 215122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220921215523-10174
I0921 22:01:55.143012 215122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49393 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/machines/bridge-20220921215523-10174/id_rsa Username:docker}
I0921 22:01:55.206059 215122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.76.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.25.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0921 22:01:55.207286 215122 node_ready.go:35] waiting up to 5m0s for node "bridge-20220921215523-10174" to be "Ready" ...
I0921 22:01:55.210245 215122 node_ready.go:49] node "bridge-20220921215523-10174" has status "Ready":"True"
I0921 22:01:55.210270 215122 node_ready.go:38] duration metric: took 2.956209ms waiting for node "bridge-20220921215523-10174" to be "Ready" ...
I0921 22:01:55.210283 215122 pod_ready.go:35] extra waiting up to 5m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0921 22:01:55.217942 215122 pod_ready.go:78] waiting up to 5m0s for pod "coredns-565d847f94-bqxgb" in "kube-system" namespace to be "Ready" ...
I0921 22:01:55.295379 215122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.25.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0921 22:01:55.395147 215122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.25.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0921 22:01:56.598791 215122 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.76.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.25.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (1.392690906s)
I0921 22:01:56.598825 215122 start.go:810] {"host.minikube.internal": 192.168.76.1} host record injected into CoreDNS
I0921 22:01:56.703574 215122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.25.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.408152473s)
I0921 22:01:56.703625 215122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.25.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (1.308438079s)
I0921 22:01:56.705275 215122 out.go:177] * Enabled addons: storage-provisioner, default-storageclass
I0921 22:01:56.706487 215122 addons.go:414] enableAddons completed in 1.673269376s
I0921 22:01:57.225568 215122 pod_ready.go:97] error getting pod "coredns-565d847f94-bqxgb" in "kube-system" namespace (skipping!): pods "coredns-565d847f94-bqxgb" not found
I0921 22:01:57.225607 215122 pod_ready.go:81] duration metric: took 2.007638897s waiting for pod "coredns-565d847f94-bqxgb" in "kube-system" namespace to be "Ready" ...
E0921 22:01:57.225619 215122 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-565d847f94-bqxgb" in "kube-system" namespace (skipping!): pods "coredns-565d847f94-bqxgb" not found
I0921 22:01:57.225628 215122 pod_ready.go:78] waiting up to 5m0s for pod "coredns-565d847f94-fxxks" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.230414 215122 pod_ready.go:92] pod "coredns-565d847f94-fxxks" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.230440 215122 pod_ready.go:81] duration metric: took 4.802284ms waiting for pod "coredns-565d847f94-fxxks" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.230454 215122 pod_ready.go:78] waiting up to 5m0s for pod "etcd-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.234767 215122 pod_ready.go:92] pod "etcd-bridge-20220921215523-10174" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.234786 215122 pod_ready.go:81] duration metric: took 4.3247ms waiting for pod "etcd-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.234794 215122 pod_ready.go:78] waiting up to 5m0s for pod "kube-apiserver-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.238841 215122 pod_ready.go:92] pod "kube-apiserver-bridge-20220921215523-10174" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.238865 215122 pod_ready.go:81] duration metric: took 4.061603ms waiting for pod "kube-apiserver-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.238879 215122 pod_ready.go:78] waiting up to 5m0s for pod "kube-controller-manager-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.242961 215122 pod_ready.go:92] pod "kube-controller-manager-bridge-20220921215523-10174" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.242977 215122 pod_ready.go:81] duration metric: took 4.091538ms waiting for pod "kube-controller-manager-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.242988 215122 pod_ready.go:78] waiting up to 5m0s for pod "kube-proxy-l98hk" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.426825 215122 pod_ready.go:92] pod "kube-proxy-l98hk" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.426846 215122 pod_ready.go:81] duration metric: took 183.851206ms waiting for pod "kube-proxy-l98hk" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.426855 215122 pod_ready.go:78] waiting up to 5m0s for pod "kube-scheduler-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.826694 215122 pod_ready.go:92] pod "kube-scheduler-bridge-20220921215523-10174" in "kube-system" namespace has status "Ready":"True"
I0921 22:01:57.826717 215122 pod_ready.go:81] duration metric: took 399.856935ms waiting for pod "kube-scheduler-bridge-20220921215523-10174" in "kube-system" namespace to be "Ready" ...
I0921 22:01:57.826725 215122 pod_ready.go:38] duration metric: took 2.616430261s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0921 22:01:57.826742 215122 api_server.go:51] waiting for apiserver process to appear ...
I0921 22:01:57.826786 215122 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0921 22:01:57.837015 215122 api_server.go:71] duration metric: took 2.803807751s to wait for apiserver process to appear ...
I0921 22:01:57.837046 215122 api_server.go:87] waiting for apiserver healthz status ...
I0921 22:01:57.837056 215122 api_server.go:240] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0921 22:01:57.842917 215122 api_server.go:266] https://192.168.76.2:8443/healthz returned 200:
ok
I0921 22:01:57.843710 215122 api_server.go:140] control plane version: v1.25.2
I0921 22:01:57.843771 215122 api_server.go:130] duration metric: took 6.71763ms to wait for apiserver health ...
I0921 22:01:57.843781 215122 system_pods.go:43] waiting for kube-system pods to appear ...
I0921 22:01:58.028997 215122 system_pods.go:59] 7 kube-system pods found
I0921 22:01:58.029048 215122 system_pods.go:61] "coredns-565d847f94-fxxks" [d6f9c23e-6e19-4508-b035-add7e2e4254c] Running
I0921 22:01:58.029059 215122 system_pods.go:61] "etcd-bridge-20220921215523-10174" [32f2917b-da4f-412a-8102-0e356ea4cf2d] Running
I0921 22:01:58.029066 215122 system_pods.go:61] "kube-apiserver-bridge-20220921215523-10174" [b17f870a-7301-4606-afa6-ea69e5c53dde] Running
I0921 22:01:58.029073 215122 system_pods.go:61] "kube-controller-manager-bridge-20220921215523-10174" [ce2ffc4f-bf2a-4196-a9a6-8eecc8f17191] Running
I0921 22:01:58.029079 215122 system_pods.go:61] "kube-proxy-l98hk" [bd75f59e-4881-4ebf-ad5d-148ad1c5c538] Running
I0921 22:01:58.029097 215122 system_pods.go:61] "kube-scheduler-bridge-20220921215523-10174" [c6a289e9-7646-4bf5-a10d-76a8bbeb05ca] Running
I0921 22:01:58.029108 215122 system_pods.go:61] "storage-provisioner" [27990257-9062-4409-b438-527977853a49] Pending / Ready:ContainersNotReady (containers with unready status: [storage-provisioner]) / ContainersReady:ContainersNotReady (containers with unready status: [storage-provisioner])
I0921 22:01:58.029125 215122 system_pods.go:74] duration metric: took 185.331727ms to wait for pod list to return data ...
I0921 22:01:58.029140 215122 default_sa.go:34] waiting for default service account to be created ...
I0921 22:01:58.225893 215122 default_sa.go:45] found service account: "default"
I0921 22:01:58.225918 215122 default_sa.go:55] duration metric: took 196.771614ms for default service account to be created ...
I0921 22:01:58.225926 215122 system_pods.go:116] waiting for k8s-apps to be running ...
I0921 22:01:58.428204 215122 system_pods.go:86] 7 kube-system pods found
I0921 22:01:58.428234 215122 system_pods.go:89] "coredns-565d847f94-fxxks" [d6f9c23e-6e19-4508-b035-add7e2e4254c] Running
I0921 22:01:58.428240 215122 system_pods.go:89] "etcd-bridge-20220921215523-10174" [32f2917b-da4f-412a-8102-0e356ea4cf2d] Running
I0921 22:01:58.428245 215122 system_pods.go:89] "kube-apiserver-bridge-20220921215523-10174" [b17f870a-7301-4606-afa6-ea69e5c53dde] Running
I0921 22:01:58.428250 215122 system_pods.go:89] "kube-controller-manager-bridge-20220921215523-10174" [ce2ffc4f-bf2a-4196-a9a6-8eecc8f17191] Running
I0921 22:01:58.428254 215122 system_pods.go:89] "kube-proxy-l98hk" [bd75f59e-4881-4ebf-ad5d-148ad1c5c538] Running
I0921 22:01:58.428260 215122 system_pods.go:89] "kube-scheduler-bridge-20220921215523-10174" [c6a289e9-7646-4bf5-a10d-76a8bbeb05ca] Running
I0921 22:01:58.428264 215122 system_pods.go:89] "storage-provisioner" [27990257-9062-4409-b438-527977853a49] Running
I0921 22:01:58.428270 215122 system_pods.go:126] duration metric: took 202.339794ms to wait for k8s-apps to be running ...
I0921 22:01:58.428284 215122 system_svc.go:44] waiting for kubelet service to be running ....
I0921 22:01:58.428321 215122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0921 22:01:58.438155 215122 system_svc.go:56] duration metric: took 9.862878ms WaitForService to wait for kubelet.
I0921 22:01:58.438185 215122 kubeadm.go:573] duration metric: took 3.404981518s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...
I0921 22:01:58.438203 215122 node_conditions.go:102] verifying NodePressure condition ...
I0921 22:01:58.626287 215122 node_conditions.go:122] node storage ephemeral capacity is 304681132Ki
I0921 22:01:58.626314 215122 node_conditions.go:123] node cpu capacity is 8
I0921 22:01:58.626326 215122 node_conditions.go:105] duration metric: took 188.11801ms to run NodePressure ...
I0921 22:01:58.626336 215122 start.go:216] waiting for startup goroutines ...
I0921 22:01:58.669164 215122 start.go:506] kubectl: 1.25.2, cluster: 1.25.2 (minor skew: 0)
I0921 22:01:58.671666 215122 out.go:177] * Done! kubectl is now configured to use "bridge-20220921215523-10174" cluster and "default" namespace by default
I0921 22:01:54.795644 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:56.796027 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:59.295611 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:56.284041 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:01:58.783815 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:01.295691 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:03.295961 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:00.783923 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:03.283561 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:05.795975 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:07.796658 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:05.784837 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:07.785470 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:10.296146 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:12.795313 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:10.283603 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:12.785065 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:14.785360 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:14.795662 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:17.295701 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:19.296052 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:16.785760 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:18.786632 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:21.795289 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:23.795532 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:21.282907 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:23.283162 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:26.295643 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:28.795800 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:25.283561 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:27.284451 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:29.782771 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:31.295448 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:33.295517 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:31.785078 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:34.283276 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:37.018351 163433 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0921 22:02:37.018524 163433 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0921 22:02:37.021344 163433 kubeadm.go:317] [init] Using Kubernetes version: v1.25.2
I0921 22:02:37.021412 163433 kubeadm.go:317] [preflight] Running pre-flight checks
I0921 22:02:37.021521 163433 kubeadm.go:317] [preflight] The system verification failed. Printing the output from the verification:
I0921 22:02:37.021617 163433 kubeadm.go:317] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
I0921 22:02:37.021677 163433 kubeadm.go:317] [0;37mOS[0m: [0;32mLinux[0m
I0921 22:02:37.021750 163433 kubeadm.go:317] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0921 22:02:37.021830 163433 kubeadm.go:317] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0921 22:02:37.021902 163433 kubeadm.go:317] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0921 22:02:37.021973 163433 kubeadm.go:317] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0921 22:02:37.022047 163433 kubeadm.go:317] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0921 22:02:37.022121 163433 kubeadm.go:317] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0921 22:02:37.022188 163433 kubeadm.go:317] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0921 22:02:37.022253 163433 kubeadm.go:317] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0921 22:02:37.022318 163433 kubeadm.go:317] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0921 22:02:37.022421 163433 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0921 22:02:37.022566 163433 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0921 22:02:37.022728 163433 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0921 22:02:37.022848 163433 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0921 22:02:37.025283 163433 out.go:204] - Generating certificates and keys ...
I0921 22:02:37.025385 163433 kubeadm.go:317] [certs] Using existing ca certificate authority
I0921 22:02:37.025475 163433 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0921 22:02:37.025582 163433 kubeadm.go:317] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0921 22:02:37.025662 163433 kubeadm.go:317] [certs] Using existing front-proxy-ca certificate authority
I0921 22:02:37.025742 163433 kubeadm.go:317] [certs] Using existing front-proxy-client certificate and key on disk
I0921 22:02:37.025826 163433 kubeadm.go:317] [certs] Using existing etcd/ca certificate authority
I0921 22:02:37.025908 163433 kubeadm.go:317] [certs] Using existing etcd/server certificate and key on disk
I0921 22:02:37.025970 163433 kubeadm.go:317] [certs] Using existing etcd/peer certificate and key on disk
I0921 22:02:37.026043 163433 kubeadm.go:317] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0921 22:02:37.026115 163433 kubeadm.go:317] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0921 22:02:37.026150 163433 kubeadm.go:317] [certs] Using the existing "sa" key
I0921 22:02:37.026236 163433 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0921 22:02:37.026314 163433 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0921 22:02:37.026388 163433 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0921 22:02:37.026472 163433 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0921 22:02:37.026545 163433 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0921 22:02:37.026684 163433 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0921 22:02:37.026792 163433 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0921 22:02:37.026839 163433 kubeadm.go:317] [kubelet-start] Starting the kubelet
I0921 22:02:37.026895 163433 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0921 22:02:37.028542 163433 out.go:204] - Booting up control plane ...
I0921 22:02:37.028642 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0921 22:02:37.028731 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0921 22:02:37.028833 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0921 22:02:37.028952 163433 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0921 22:02:37.029138 163433 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0921 22:02:37.029208 163433 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0921 22:02:37.029298 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.029557 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.029641 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.029848 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.029934 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030147 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030218 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030380 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030466 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:02:37.030646 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:02:37.030656 163433 kubeadm.go:317]
I0921 22:02:37.030710 163433 kubeadm.go:317] Unfortunately, an error has occurred:
I0921 22:02:37.030783 163433 kubeadm.go:317] timed out waiting for the condition
I0921 22:02:37.030793 163433 kubeadm.go:317]
I0921 22:02:37.030849 163433 kubeadm.go:317] This error is likely caused by:
I0921 22:02:37.030900 163433 kubeadm.go:317] - The kubelet is not running
I0921 22:02:37.031052 163433 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0921 22:02:37.031070 163433 kubeadm.go:317]
I0921 22:02:37.031191 163433 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0921 22:02:37.031243 163433 kubeadm.go:317] - 'systemctl status kubelet'
I0921 22:02:37.031297 163433 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0921 22:02:37.031308 163433 kubeadm.go:317]
I0921 22:02:37.031422 163433 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0921 22:02:37.031534 163433 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0921 22:02:37.031667 163433 kubeadm.go:317] Here is one example how you may list all running Kubernetes containers by using crictl:
I0921 22:02:37.031810 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
I0921 22:02:37.031929 163433 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0921 22:02:37.032051 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
W0921 22:02:37.032372 163433 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:00:40.923939 8214 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0921 22:02:37.032427 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0921 22:02:38.854856 163433 ssh_runner.go:235] Completed: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force": (1.822402127s)
I0921 22:02:38.854917 163433 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0921 22:02:38.864392 163433 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0921 22:02:38.864449 163433 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0921 22:02:38.871143 163433 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0921 22:02:38.871179 163433 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0921 22:02:38.909279 163433 kubeadm.go:317] [init] Using Kubernetes version: v1.25.2
I0921 22:02:38.909365 163433 kubeadm.go:317] [preflight] Running pre-flight checks
I0921 22:02:38.936932 163433 kubeadm.go:317] [preflight] The system verification failed. Printing the output from the verification:
I0921 22:02:38.937031 163433 kubeadm.go:317] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
I0921 22:02:38.937063 163433 kubeadm.go:317] [0;37mOS[0m: [0;32mLinux[0m
I0921 22:02:38.937103 163433 kubeadm.go:317] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0921 22:02:38.937150 163433 kubeadm.go:317] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0921 22:02:38.937239 163433 kubeadm.go:317] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0921 22:02:38.937318 163433 kubeadm.go:317] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0921 22:02:38.937390 163433 kubeadm.go:317] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0921 22:02:38.937481 163433 kubeadm.go:317] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0921 22:02:38.937544 163433 kubeadm.go:317] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0921 22:02:38.937595 163433 kubeadm.go:317] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0921 22:02:38.937662 163433 kubeadm.go:317] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0921 22:02:39.000511 163433 kubeadm.go:317] [preflight] Pulling images required for setting up a Kubernetes cluster
I0921 22:02:39.000640 163433 kubeadm.go:317] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0921 22:02:39.000796 163433 kubeadm.go:317] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0921 22:02:39.116829 163433 kubeadm.go:317] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0921 22:02:35.295884 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:37.296675 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:36.784618 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:38.785789 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:39.119686 163433 out.go:204] - Generating certificates and keys ...
I0921 22:02:39.119846 163433 kubeadm.go:317] [certs] Using existing ca certificate authority
I0921 22:02:39.119930 163433 kubeadm.go:317] [certs] Using existing apiserver certificate and key on disk
I0921 22:02:39.120020 163433 kubeadm.go:317] [certs] Using existing apiserver-kubelet-client certificate and key on disk
I0921 22:02:39.120098 163433 kubeadm.go:317] [certs] Using existing front-proxy-ca certificate authority
I0921 22:02:39.120195 163433 kubeadm.go:317] [certs] Using existing front-proxy-client certificate and key on disk
I0921 22:02:39.120267 163433 kubeadm.go:317] [certs] Using existing etcd/ca certificate authority
I0921 22:02:39.120352 163433 kubeadm.go:317] [certs] Using existing etcd/server certificate and key on disk
I0921 22:02:39.120437 163433 kubeadm.go:317] [certs] Using existing etcd/peer certificate and key on disk
I0921 22:02:39.120536 163433 kubeadm.go:317] [certs] Using existing etcd/healthcheck-client certificate and key on disk
I0921 22:02:39.120627 163433 kubeadm.go:317] [certs] Using existing apiserver-etcd-client certificate and key on disk
I0921 22:02:39.120686 163433 kubeadm.go:317] [certs] Using the existing "sa" key
I0921 22:02:39.120764 163433 kubeadm.go:317] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0921 22:02:39.214860 163433 kubeadm.go:317] [kubeconfig] Writing "admin.conf" kubeconfig file
I0921 22:02:39.365850 163433 kubeadm.go:317] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0921 22:02:39.700601 163433 kubeadm.go:317] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0921 22:02:40.053396 163433 kubeadm.go:317] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0921 22:02:40.065917 163433 kubeadm.go:317] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0921 22:02:40.066776 163433 kubeadm.go:317] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0921 22:02:40.066842 163433 kubeadm.go:317] [kubelet-start] Starting the kubelet
I0921 22:02:40.151802 163433 kubeadm.go:317] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0921 22:02:40.154174 163433 out.go:204] - Booting up control plane ...
I0921 22:02:40.154312 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0921 22:02:40.154475 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0921 22:02:40.155428 163433 kubeadm.go:317] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0921 22:02:40.156385 163433 kubeadm.go:317] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0921 22:02:40.159229 163433 kubeadm.go:317] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
I0921 22:02:39.796175 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:41.796249 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:44.295204 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:41.282949 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:43.283158 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:46.295601 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:48.795781 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:45.783545 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:48.283437 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:51.295524 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:53.795526 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:50.283654 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:52.785091 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:54.785272 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:56.295265 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:58.296043 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:57.286226 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:02:59.784817 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:00.796016 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:03.295516 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:01.785748 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:04.283055 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:05.796132 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:08.295230 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:06.283163 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:08.785308 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:10.295605 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:12.794717 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:11.283557 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:13.784480 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:14.795958 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:17.295388 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:15.784531 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:17.785041 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:19.785311 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:20.159558 163433 kubeadm.go:317] [kubelet-check] Initial timeout of 40s passed.
I0921 22:03:20.159897 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:20.160149 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:19.796040 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:22.295383 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:22.282513 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:24.282757 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:25.160511 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:25.160744 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:24.795802 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:27.296018 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:26.282981 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:28.283100 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:29.795884 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:31.796035 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:34.295079 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:30.785240 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:33.283585 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:35.160837 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:35.161107 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:36.297275 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:38.795043 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:35.784442 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:37.784725 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:39.784865 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:41.295402 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:43.295511 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:42.282777 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:44.282942 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:45.795988 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:48.295486 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:46.783254 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:49.283243 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:50.295629 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:52.794913 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:51.783288 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:53.784868 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:55.162112 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:03:55.162307 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:03:54.796081 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:57.296420 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:55.786215 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:58.282766 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:03:59.795300 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:01.796087 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:04.295001 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:00.782996 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:02.784677 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:06.295897 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:08.795477 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:05.285308 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:07.783365 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:09.785992 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:11.295345 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:13.295670 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:12.282740 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:14.283416 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:15.795912 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:18.295246 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:16.785048 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:19.283886 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:20.794913 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:22.796143 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:21.783586 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:24.283127 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:25.296005 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:27.296540 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:26.283486 199263 pod_ready.go:102] pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:27.790035 199263 pod_ready.go:81] duration metric: took 4m0.046661281s waiting for pod "calico-kube-controllers-7df895d496-vk6vs" in "kube-system" namespace to be "Ready" ...
E0921 22:04:27.790062 199263 pod_ready.go:66] WaitExtra: waitPodCondition: timed out waiting for the condition
I0921 22:04:27.790073 199263 pod_ready.go:78] waiting up to 5m0s for pod "calico-node-lcxq5" in "kube-system" namespace to be "Ready" ...
I0921 22:04:29.800454 199263 pod_ready.go:102] pod "calico-node-lcxq5" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:29.795939 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:32.295828 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:34.296018 203160 pod_ready.go:102] pod "coredns-565d847f94-hsd8z" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:32.300614 199263 pod_ready.go:102] pod "calico-node-lcxq5" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:34.301269 199263 pod_ready.go:102] pod "calico-node-lcxq5" in "kube-system" namespace has status "Ready":"False"
I0921 22:04:35.163184 163433 kubeadm.go:317] [kubelet-check] It seems like the kubelet isn't running or healthy.
I0921 22:04:35.163483 163433 kubeadm.go:317] [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
I0921 22:04:35.163518 163433 kubeadm.go:317]
I0921 22:04:35.163584 163433 kubeadm.go:317] Unfortunately, an error has occurred:
I0921 22:04:35.163648 163433 kubeadm.go:317] timed out waiting for the condition
I0921 22:04:35.163661 163433 kubeadm.go:317]
I0921 22:04:35.163710 163433 kubeadm.go:317] This error is likely caused by:
I0921 22:04:35.163796 163433 kubeadm.go:317] - The kubelet is not running
I0921 22:04:35.163928 163433 kubeadm.go:317] - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
I0921 22:04:35.163937 163433 kubeadm.go:317]
I0921 22:04:35.164060 163433 kubeadm.go:317] If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
I0921 22:04:35.164118 163433 kubeadm.go:317] - 'systemctl status kubelet'
I0921 22:04:35.164157 163433 kubeadm.go:317] - 'journalctl -xeu kubelet'
I0921 22:04:35.164173 163433 kubeadm.go:317]
I0921 22:04:35.164320 163433 kubeadm.go:317] Additionally, a control plane component may have crashed or exited when started by the container runtime.
I0921 22:04:35.164436 163433 kubeadm.go:317] To troubleshoot, list all containers using your preferred container runtimes CLI.
I0921 22:04:35.164551 163433 kubeadm.go:317] Here is one example how you may list all running Kubernetes containers by using crictl:
I0921 22:04:35.164687 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
I0921 22:04:35.164801 163433 kubeadm.go:317] Once you have found the failing container, you can inspect its logs with:
I0921 22:04:35.164918 163433 kubeadm.go:317] - 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
I0921 22:04:35.166104 163433 kubeadm.go:317] W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
I0921 22:04:35.166318 163433 kubeadm.go:317] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
I0921 22:04:35.166415 163433 kubeadm.go:317] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0921 22:04:35.166492 163433 kubeadm.go:317] error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
I0921 22:04:35.166592 163433 kubeadm.go:317] To see the stack trace of this error execute with --v=5 or higher
I0921 22:04:35.166696 163433 kubeadm.go:398] StartCluster complete in 7m57.461301325s
I0921 22:04:35.166738 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0921 22:04:35.166788 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0921 22:04:35.191228 163433 cri.go:87] found id: ""
I0921 22:04:35.191254 163433 logs.go:274] 0 containers: []
W0921 22:04:35.191261 163433 logs.go:276] No container was found matching "kube-apiserver"
I0921 22:04:35.191272 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0921 22:04:35.191332 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0921 22:04:35.214715 163433 cri.go:87] found id: ""
I0921 22:04:35.214744 163433 logs.go:274] 0 containers: []
W0921 22:04:35.214750 163433 logs.go:276] No container was found matching "etcd"
I0921 22:04:35.214756 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0921 22:04:35.214804 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0921 22:04:35.239162 163433 cri.go:87] found id: ""
I0921 22:04:35.239186 163433 logs.go:274] 0 containers: []
W0921 22:04:35.239192 163433 logs.go:276] No container was found matching "coredns"
I0921 22:04:35.239197 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0921 22:04:35.239252 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0921 22:04:35.263333 163433 cri.go:87] found id: ""
I0921 22:04:35.263357 163433 logs.go:274] 0 containers: []
W0921 22:04:35.263363 163433 logs.go:276] No container was found matching "kube-scheduler"
I0921 22:04:35.263368 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0921 22:04:35.263407 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0921 22:04:35.286321 163433 cri.go:87] found id: ""
I0921 22:04:35.286347 163433 logs.go:274] 0 containers: []
W0921 22:04:35.286355 163433 logs.go:276] No container was found matching "kube-proxy"
I0921 22:04:35.286364 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0921 22:04:35.286426 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0921 22:04:35.314686 163433 cri.go:87] found id: ""
I0921 22:04:35.314714 163433 logs.go:274] 0 containers: []
W0921 22:04:35.314722 163433 logs.go:276] No container was found matching "kubernetes-dashboard"
I0921 22:04:35.314730 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0921 22:04:35.314783 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0921 22:04:35.338110 163433 cri.go:87] found id: ""
I0921 22:04:35.338142 163433 logs.go:274] 0 containers: []
W0921 22:04:35.338151 163433 logs.go:276] No container was found matching "storage-provisioner"
I0921 22:04:35.338160 163433 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0921 22:04:35.338240 163433 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0921 22:04:35.363135 163433 cri.go:87] found id: ""
I0921 22:04:35.363167 163433 logs.go:274] 0 containers: []
W0921 22:04:35.363176 163433 logs.go:276] No container was found matching "kube-controller-manager"
I0921 22:04:35.363188 163433 logs.go:123] Gathering logs for kubelet ...
I0921 22:04:35.363200 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0921 22:04:35.379811 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:45 kubernetes-upgrade-20220921215522-10174 kubelet[12183]: E0921 22:03:45.374524 12183 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.380296 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12194]: E0921 22:03:46.123389 12194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.380707 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12205]: E0921 22:03:46.873433 12205 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381121 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:47 kubernetes-upgrade-20220921215522-10174 kubelet[12216]: E0921 22:03:47.622295 12216 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381546 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:48 kubernetes-upgrade-20220921215522-10174 kubelet[12227]: E0921 22:03:48.372753 12227 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.381922 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:49 kubernetes-upgrade-20220921215522-10174 kubelet[12238]: E0921 22:03:49.121733 12238 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.382304 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:49 kubernetes-upgrade-20220921215522-10174 kubelet[12249]: E0921 22:03:49.874411 12249 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.382688 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:50 kubernetes-upgrade-20220921215522-10174 kubelet[12260]: E0921 22:03:50.621954 12260 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383079 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:51 kubernetes-upgrade-20220921215522-10174 kubelet[12271]: E0921 22:03:51.373297 12271 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383463 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:52 kubernetes-upgrade-20220921215522-10174 kubelet[12282]: E0921 22:03:52.124808 12282 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.383909 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:52 kubernetes-upgrade-20220921215522-10174 kubelet[12293]: E0921 22:03:52.872464 12293 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.384310 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:53 kubernetes-upgrade-20220921215522-10174 kubelet[12304]: E0921 22:03:53.621146 12304 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.384694 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:54 kubernetes-upgrade-20220921215522-10174 kubelet[12315]: E0921 22:03:54.372737 12315 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385094 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:55 kubernetes-upgrade-20220921215522-10174 kubelet[12326]: E0921 22:03:55.122043 12326 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385483 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:55 kubernetes-upgrade-20220921215522-10174 kubelet[12337]: E0921 22:03:55.873129 12337 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.385896 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:56 kubernetes-upgrade-20220921215522-10174 kubelet[12349]: E0921 22:03:56.622479 12349 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.386331 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:57 kubernetes-upgrade-20220921215522-10174 kubelet[12360]: E0921 22:03:57.373797 12360 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.386778 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:58 kubernetes-upgrade-20220921215522-10174 kubelet[12371]: E0921 22:03:58.123173 12371 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387176 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:58 kubernetes-upgrade-20220921215522-10174 kubelet[12383]: E0921 22:03:58.872655 12383 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387549 163433 logs.go:138] Found kubelet problem: Sep 21 22:03:59 kubernetes-upgrade-20220921215522-10174 kubelet[12394]: E0921 22:03:59.624915 12394 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.387984 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:00 kubernetes-upgrade-20220921215522-10174 kubelet[12405]: E0921 22:04:00.373541 12405 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.388383 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:01 kubernetes-upgrade-20220921215522-10174 kubelet[12416]: E0921 22:04:01.125035 12416 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.388766 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:01 kubernetes-upgrade-20220921215522-10174 kubelet[12428]: E0921 22:04:01.874353 12428 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.389209 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:02 kubernetes-upgrade-20220921215522-10174 kubelet[12439]: E0921 22:04:02.624609 12439 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.389617 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:03 kubernetes-upgrade-20220921215522-10174 kubelet[12450]: E0921 22:04:03.374384 12450 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390085 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:04 kubernetes-upgrade-20220921215522-10174 kubelet[12461]: E0921 22:04:04.121121 12461 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390489 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:04 kubernetes-upgrade-20220921215522-10174 kubelet[12471]: E0921 22:04:04.874129 12471 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.390875 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:05 kubernetes-upgrade-20220921215522-10174 kubelet[12482]: E0921 22:04:05.622119 12482 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.391301 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:06 kubernetes-upgrade-20220921215522-10174 kubelet[12494]: E0921 22:04:06.373783 12494 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.391702 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:07 kubernetes-upgrade-20220921215522-10174 kubelet[12504]: E0921 22:04:07.122518 12504 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.392128 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:07 kubernetes-upgrade-20220921215522-10174 kubelet[12516]: E0921 22:04:07.873670 12516 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.392780 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:08 kubernetes-upgrade-20220921215522-10174 kubelet[12527]: E0921 22:04:08.622374 12527 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.393441 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:09 kubernetes-upgrade-20220921215522-10174 kubelet[12539]: E0921 22:04:09.380221 12539 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.393896 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:10 kubernetes-upgrade-20220921215522-10174 kubelet[12549]: E0921 22:04:10.123024 12549 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.394289 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:10 kubernetes-upgrade-20220921215522-10174 kubelet[12560]: E0921 22:04:10.871810 12560 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.394689 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:11 kubernetes-upgrade-20220921215522-10174 kubelet[12570]: E0921 22:04:11.623184 12570 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395075 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:12 kubernetes-upgrade-20220921215522-10174 kubelet[12581]: E0921 22:04:12.373639 12581 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395484 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:13 kubernetes-upgrade-20220921215522-10174 kubelet[12592]: E0921 22:04:13.122993 12592 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.395916 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:13 kubernetes-upgrade-20220921215522-10174 kubelet[12602]: E0921 22:04:13.874559 12602 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.396306 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:14 kubernetes-upgrade-20220921215522-10174 kubelet[12612]: E0921 22:04:14.623058 12612 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.396713 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:15 kubernetes-upgrade-20220921215522-10174 kubelet[12623]: E0921 22:04:15.372885 12623 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397101 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:16 kubernetes-upgrade-20220921215522-10174 kubelet[12633]: E0921 22:04:16.123144 12633 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397495 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:16 kubernetes-upgrade-20220921215522-10174 kubelet[12644]: E0921 22:04:16.872120 12644 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.397886 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:17 kubernetes-upgrade-20220921215522-10174 kubelet[12655]: E0921 22:04:17.621957 12655 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.398268 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:18 kubernetes-upgrade-20220921215522-10174 kubelet[12666]: E0921 22:04:18.372467 12666 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.398694 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:19 kubernetes-upgrade-20220921215522-10174 kubelet[12676]: E0921 22:04:19.122753 12676 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399092 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:19 kubernetes-upgrade-20220921215522-10174 kubelet[12687]: E0921 22:04:19.876067 12687 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399483 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:20 kubernetes-upgrade-20220921215522-10174 kubelet[12698]: E0921 22:04:20.623954 12698 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.399925 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:21 kubernetes-upgrade-20220921215522-10174 kubelet[12709]: E0921 22:04:21.373994 12709 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.400440 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:22 kubernetes-upgrade-20220921215522-10174 kubelet[12720]: E0921 22:04:22.124039 12720 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.400894 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:22 kubernetes-upgrade-20220921215522-10174 kubelet[12730]: E0921 22:04:22.872700 12730 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.401275 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:23 kubernetes-upgrade-20220921215522-10174 kubelet[12741]: E0921 22:04:23.620191 12741 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.401664 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:24 kubernetes-upgrade-20220921215522-10174 kubelet[12752]: E0921 22:04:24.373902 12752 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402051 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:25 kubernetes-upgrade-20220921215522-10174 kubelet[12763]: E0921 22:04:25.123958 12763 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402430 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:25 kubernetes-upgrade-20220921215522-10174 kubelet[12773]: E0921 22:04:25.874249 12773 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.402818 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:26 kubernetes-upgrade-20220921215522-10174 kubelet[12786]: E0921 22:04:26.623403 12786 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403198 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:27 kubernetes-upgrade-20220921215522-10174 kubelet[12797]: E0921 22:04:27.375008 12797 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403573 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:28 kubernetes-upgrade-20220921215522-10174 kubelet[12808]: E0921 22:04:28.122905 12808 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.403988 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:28 kubernetes-upgrade-20220921215522-10174 kubelet[12819]: E0921 22:04:28.871801 12819 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.404363 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:29 kubernetes-upgrade-20220921215522-10174 kubelet[12830]: E0921 22:04:29.622004 12830 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.404759 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:30 kubernetes-upgrade-20220921215522-10174 kubelet[12842]: E0921 22:04:30.373730 12842 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405157 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:31 kubernetes-upgrade-20220921215522-10174 kubelet[12854]: E0921 22:04:31.121895 12854 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405540 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:31 kubernetes-upgrade-20220921215522-10174 kubelet[12865]: E0921 22:04:31.872824 12865 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.405921 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:32 kubernetes-upgrade-20220921215522-10174 kubelet[12876]: E0921 22:04:32.622571 12876 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.406306 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:33 kubernetes-upgrade-20220921215522-10174 kubelet[12887]: E0921 22:04:33.373474 12887 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.406697 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12898]: E0921 22:04:34.121999 12898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
W0921 22:04:35.407098 163433 logs.go:138] Found kubelet problem: Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12909]: E0921 22:04:34.878168 12909 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.407226 163433 logs.go:123] Gathering logs for dmesg ...
I0921 22:04:35.407241 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0921 22:04:35.428361 163433 logs.go:123] Gathering logs for describe nodes ...
I0921 22:04:35.428401 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0921 22:04:35.484445 163433 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0921 22:04:35.484475 163433 logs.go:123] Gathering logs for containerd ...
I0921 22:04:35.484488 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0921 22:04:35.545769 163433 logs.go:123] Gathering logs for container status ...
I0921 22:04:35.545802 163433 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0921 22:04:35.571634 163433 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.571680 163433 out.go:239] *
W0921 22:04:35.571931 163433 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.571967 163433 out.go:239] *
W0921 22:04:35.572762 163433 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0921 22:04:35.575608 163433 out.go:177] X Problems detected in kubelet:
I0921 22:04:35.576957 163433 out.go:177] Sep 21 22:03:45 kubernetes-upgrade-20220921215522-10174 kubelet[12183]: E0921 22:03:45.374524 12183 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.578291 163433 out.go:177] Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12194]: E0921 22:03:46.123389 12194 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.579626 163433 out.go:177] Sep 21 22:03:46 kubernetes-upgrade-20220921215522-10174 kubelet[12205]: E0921 22:03:46.873433 12205 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
I0921 22:04:35.583058 163433 out.go:177]
W0921 22:04:35.584539 163433 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.25.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.25.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1017-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0921 22:02:38.904023 11075 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1017-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0921 22:04:35.584658 163433 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0921 22:04:35.584719 163433 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
I0921 22:04:35.586989 163433 out.go:177]
*
* ==> container status <==
* CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
*
* ==> containerd <==
* -- Logs begin at Wed 2022-09-21 21:56:11 UTC, end at Wed 2022-09-21 22:04:36 UTC. --
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.646790703Z" level=error msg="StopPodSandbox for \"format\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"format\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.664755311Z" level=info msg="StopPodSandbox for \"format\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.664806600Z" level=error msg="StopPodSandbox for \"format\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"format\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.682342993Z" level=info msg="StopPodSandbox for \"format\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.682393627Z" level=error msg="StopPodSandbox for \"format\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"format\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.699043301Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.699124093Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.716535656Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.716594187Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.732898902Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.732955687Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.749905496Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.749963734Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.765653033Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.765713465Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.782759775Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.782823951Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.800951175Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.801013780Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.816954723Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.817008244Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.833786120Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.833839985Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.849942783Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Sep 21 22:02:38 kubernetes-upgrade-20220921215522-10174 containerd[485]: time="2022-09-21T22:02:38.849991644Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
*
* ==> describe nodes <==
*
* ==> dmesg <==
* [ +0.000006] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +2.955867] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000005] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.015869] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000005] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.019929] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000005] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +12.977079] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.009970] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.023922] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000005] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +2.963873] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.035863] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.019908] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +2.943857] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.027872] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
[ +1.019949] IPv4: martian source 10.244.0.40 from 10.244.0.4, on dev eth0
[ +0.000008] ll header: 00000000: ff ff ff ff ff ff 42 92 2a 6c 80 87 08 06
*
* ==> kernel <==
* 22:04:37 up 47 min, 0 users, load average: 1.85, 2.67, 2.24
Linux kubernetes-upgrade-20220921215522-10174 5.15.0-1017-gcp #23~20.04.2-Ubuntu SMP Wed Aug 17 02:46:40 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 20.04.5 LTS"
*
* ==> kubelet <==
* -- Logs begin at Wed 2022-09-21 21:56:11 UTC, end at Wed 2022-09-21 22:04:37 UTC. --
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: Started kubelet: The Kubernetes Node Agent.
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12898]: E0921 22:04:34.121999 12898 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 153.
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: Started kubelet: The Kubernetes Node Agent.
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 kubelet[12909]: E0921 22:04:34.878168 12909 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 22:04:34 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 154.
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 systemd[1]: Started kubelet: The Kubernetes Node Agent.
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 kubelet[13056]: E0921 22:04:35.635120 13056 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 22:04:35 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 155.
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 systemd[1]: Started kubelet: The Kubernetes Node Agent.
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 kubelet[13076]: E0921 22:04:36.340866 13076 run.go:74] "command failed" err="failed to parse kubelet flag: unknown flag: --cni-conf-dir"
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Sep 21 22:04:36 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Failed with result 'exit-code'.
Sep 21 22:04:37 kubernetes-upgrade-20220921215522-10174 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 156.
Sep 21 22:04:37 kubernetes-upgrade-20220921215522-10174 systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Sep 21 22:04:37 kubernetes-upgrade-20220921215522-10174 systemd[1]: Started kubelet: The Kubernetes Node Agent.
-- /stdout --
** stderr **
E0921 22:04:37.066160 227518 logs.go:192] command /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" failed with error: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.25.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output: "\n** stderr ** \nThe connection to the server localhost:8443 was refused - did you specify the right host or port?\n\n** /stderr **"
! unable to fetch logs for: describe nodes
** /stderr **
helpers_test.go:254: (dbg) Run: out/minikube-linux-amd64 status --format={{.APIServer}} -p kubernetes-upgrade-20220921215522-10174 -n kubernetes-upgrade-20220921215522-10174
E0921 22:04:37.249434 10174 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14995-3622-411d4579fd248fd57a4259437564c3e08f354535/.minikube/profiles/auto-20220921215523-10174/client.crt: no such file or directory
helpers_test.go:254: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.APIServer}} -p kubernetes-upgrade-20220921215522-10174 -n kubernetes-upgrade-20220921215522-10174: exit status 2 (375.582999ms)
-- stdout --
Stopped
-- /stdout --
helpers_test.go:254: status error: exit status 2 (may be ok)
helpers_test.go:256: "kubernetes-upgrade-20220921215522-10174" apiserver is not running, skipping kubectl commands (state="Stopped")
helpers_test.go:175: Cleaning up "kubernetes-upgrade-20220921215522-10174" profile ...
helpers_test.go:178: (dbg) Run: out/minikube-linux-amd64 delete -p kubernetes-upgrade-20220921215522-10174
=== CONT TestKubernetesUpgrade
helpers_test.go:178: (dbg) Done: out/minikube-linux-amd64 delete -p kubernetes-upgrade-20220921215522-10174: (2.046861985s)
--- FAIL: TestKubernetesUpgrade (556.86s)