=== RUN TestKubernetesUpgrade
=== PAUSE TestKubernetesUpgrade
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:229: (dbg) Run: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220630205032-9950 --memory=2200 --kubernetes-version=v1.16.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
E0630 20:50:47.269257 9950 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/ingress-addon-legacy-20220630202923-9950/client.crt: no such file or directory
version_upgrade_test.go:229: (dbg) Done: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220630205032-9950 --memory=2200 --kubernetes-version=v1.16.0 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: (52.20861152s)
version_upgrade_test.go:234: (dbg) Run: out/minikube-linux-amd64 stop -p kubernetes-upgrade-20220630205032-9950
version_upgrade_test.go:234: (dbg) Done: out/minikube-linux-amd64 stop -p kubernetes-upgrade-20220630205032-9950: (1.370478709s)
version_upgrade_test.go:239: (dbg) Run: out/minikube-linux-amd64 -p kubernetes-upgrade-20220630205032-9950 status --format={{.Host}}
version_upgrade_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 -p kubernetes-upgrade-20220630205032-9950 status --format={{.Host}}: exit status 7 (136.289372ms)
-- stdout --
Stopped
-- /stdout --
version_upgrade_test.go:241: status error: exit status 7 (may be ok)
version_upgrade_test.go:250: (dbg) Run: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220630205032-9950 --memory=2200 --kubernetes-version=v1.24.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd
=== CONT TestKubernetesUpgrade
version_upgrade_test.go:250: (dbg) Non-zero exit: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220630205032-9950 --memory=2200 --kubernetes-version=v1.24.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd: exit status 109 (8m34.487237284s)
-- stdout --
* [kubernetes-upgrade-20220630205032-9950] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
- MINIKUBE_LOCATION=14420
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
- MINIKUBE_BIN=out/minikube-linux-amd64
* Using the docker driver based on existing profile
* Starting control plane node kubernetes-upgrade-20220630205032-9950 in cluster kubernetes-upgrade-20220630205032-9950
* Pulling base image ...
* Restarting existing docker container for "kubernetes-upgrade-20220630205032-9950" ...
* Preparing Kubernetes v1.24.2 on containerd 1.6.6 ...
- kubelet.cni-conf-dir=/etc/cni/net.mk
- Generating certificates and keys ...
- Booting up control plane ...
- Generating certificates and keys ...
- Booting up control plane ...
X Problems detected in kubelet:
Jun 30 21:00:00 kubernetes-upgrade-20220630205032-9950 kubelet[11593]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
-- /stdout --
** stderr **
I0630 20:51:26.586252 166730 out.go:296] Setting OutFile to fd 1 ...
I0630 20:51:26.586359 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:51:26.586368 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:51:26.586373 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:51:26.586840 166730 root.go:329] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/bin
I0630 20:51:26.587077 166730 out.go:303] Setting JSON to false
I0630 20:51:26.588945 166730 start.go:115] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":2039,"bootTime":1656620248,"procs":982,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1012-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0630 20:51:26.589009 166730 start.go:125] virtualization: kvm guest
I0630 20:51:26.591984 166730 out.go:177] * [kubernetes-upgrade-20220630205032-9950] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
I0630 20:51:26.593600 166730 notify.go:193] Checking for updates...
I0630 20:51:26.593666 166730 preload.go:306] deleting older generation preload /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.0-containerd-overlay2-amd64.tar.lz4
I0630 20:51:26.593628 166730 out.go:177] - MINIKUBE_LOCATION=14420
I0630 20:51:26.595320 166730 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0630 20:51:26.597266 166730 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 20:51:26.598973 166730 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
I0630 20:51:26.600457 166730 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0630 20:51:26.602410 166730 config.go:178] Loaded profile config "kubernetes-upgrade-20220630205032-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.16.0
I0630 20:51:26.603015 166730 driver.go:360] Setting default libvirt URI to qemu:///system
I0630 20:51:26.656019 166730 docker.go:137] docker version: linux-20.10.17
I0630 20:51:26.656127 166730 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 20:51:26.792023 166730 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:3 ContainersPaused:0 ContainersStopped:1 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:90 OomKillDisable:true NGoroutines:89 SystemTime:2022-06-30 20:51:26.692586748 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1012-gcp OperatingSystem:Ubuntu 20.04.4 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33662451712 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-8 Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.0] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 20:51:26.792170 166730 docker.go:254] overlay module found
I0630 20:51:26.795114 166730 out.go:177] * Using the docker driver based on existing profile
I0630 20:51:26.796421 166730 start.go:284] selected driver: docker
I0630 20:51:26.796446 166730 start.go:808] validating driver "docker" against &{Name:kubernetes-upgrade-20220630205032-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.16.0 ClusterName:kubernetes-upgrade-20220630205032-9
950 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:26214
4 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 20:51:26.796575 166730 start.go:819] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0630 20:51:26.797439 166730 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 20:51:26.941945 166730 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:4 ContainersRunning:3 ContainersPaused:0 ContainersStopped:1 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:90 OomKillDisable:true NGoroutines:89 SystemTime:2022-06-30 20:51:26.836163709 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1012-gcp OperatingSystem:Ubuntu 20.04.4 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33662451712 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-8 Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.0] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 20:51:26.942276 166730 cni.go:95] Creating CNI manager for ""
I0630 20:51:26.942296 166730 cni.go:162] "docker" driver + containerd runtime found, recommending kindnet
I0630 20:51:26.942310 166730 start_flags.go:310] config:
{Name:kubernetes-upgrade-20220630205032-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:kubernetes-upgrade-20220630205032-9950 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[]
DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.16.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: Disable
Optimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 20:51:26.945459 166730 out.go:177] * Starting control plane node kubernetes-upgrade-20220630205032-9950 in cluster kubernetes-upgrade-20220630205032-9950
I0630 20:51:26.947103 166730 cache.go:120] Beginning downloading kic base image for docker with containerd
I0630 20:51:26.948526 166730 out.go:177] * Pulling base image ...
I0630 20:51:26.949998 166730 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime containerd
I0630 20:51:26.950056 166730 preload.go:148] Found local preload: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4
I0630 20:51:26.950083 166730 cache.go:57] Caching tarball of preloaded images
I0630 20:51:26.950092 166730 image.go:75] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon
I0630 20:51:26.950355 166730 preload.go:174] Found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4 in cache, skipping download
I0630 20:51:26.950374 166730 cache.go:60] Finished verifying existence of preloaded tar for v1.24.2 on containerd
I0630 20:51:26.950521 166730 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/config.json ...
I0630 20:51:26.987537 166730 image.go:79] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon, skipping pull
I0630 20:51:26.987580 166730 cache.go:142] gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e exists in daemon, skipping load
I0630 20:51:26.987594 166730 cache.go:208] Successfully downloaded all kic artifacts
I0630 20:51:26.987685 166730 start.go:352] acquiring machines lock for kubernetes-upgrade-20220630205032-9950: {Name:mk3ce5577a5981c634683234969065cedd5d124e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0630 20:51:26.987805 166730 start.go:356] acquired machines lock for "kubernetes-upgrade-20220630205032-9950" in 93.45µs
I0630 20:51:26.987834 166730 start.go:94] Skipping create...Using existing machine configuration
I0630 20:51:26.987845 166730 fix.go:55] fixHost starting:
I0630 20:51:26.988107 166730 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-20220630205032-9950 --format={{.State.Status}}
I0630 20:51:27.021932 166730 fix.go:103] recreateIfNeeded on kubernetes-upgrade-20220630205032-9950: state=Stopped err=<nil>
W0630 20:51:27.021973 166730 fix.go:129] unexpected machine state, will restart: <nil>
I0630 20:51:27.024690 166730 out.go:177] * Restarting existing docker container for "kubernetes-upgrade-20220630205032-9950" ...
I0630 20:51:27.026337 166730 cli_runner.go:164] Run: docker start kubernetes-upgrade-20220630205032-9950
I0630 20:51:27.580949 166730 cli_runner.go:164] Run: docker container inspect kubernetes-upgrade-20220630205032-9950 --format={{.State.Status}}
I0630 20:51:27.622320 166730 kic.go:416] container "kubernetes-upgrade-20220630205032-9950" state is running.
I0630 20:51:27.622828 166730 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220630205032-9950
I0630 20:51:27.667608 166730 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/config.json ...
I0630 20:51:27.667858 166730 machine.go:88] provisioning docker machine ...
I0630 20:51:27.667881 166730 ubuntu.go:169] provisioning hostname "kubernetes-upgrade-20220630205032-9950"
I0630 20:51:27.667929 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:27.706610 166730 main.go:134] libmachine: Using SSH client type: native
I0630 20:51:27.706822 166730 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil> [] 0s} 127.0.0.1 49344 <nil> <nil>}
I0630 20:51:27.706848 166730 main.go:134] libmachine: About to run SSH command:
sudo hostname kubernetes-upgrade-20220630205032-9950 && echo "kubernetes-upgrade-20220630205032-9950" | sudo tee /etc/hostname
I0630 20:51:27.707572 166730 main.go:134] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:55004->127.0.0.1:49344: read: connection reset by peer
I0630 20:51:30.852653 166730 main.go:134] libmachine: SSH cmd err, output: <nil>: kubernetes-upgrade-20220630205032-9950
I0630 20:51:30.852738 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:30.894688 166730 main.go:134] libmachine: Using SSH client type: native
I0630 20:51:30.894880 166730 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil> [] 0s} 127.0.0.1 49344 <nil> <nil>}
I0630 20:51:30.894912 166730 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\skubernetes-upgrade-20220630205032-9950' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 kubernetes-upgrade-20220630205032-9950/g' /etc/hosts;
else
echo '127.0.1.1 kubernetes-upgrade-20220630205032-9950' | sudo tee -a /etc/hosts;
fi
fi
I0630 20:51:31.026245 166730 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0630 20:51:31.026281 166730 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/c
erts/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube}
I0630 20:51:31.026321 166730 ubuntu.go:177] setting up certificates
I0630 20:51:31.026334 166730 provision.go:83] configureAuth start
I0630 20:51:31.026401 166730 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.068746 166730 provision.go:138] copyHostCerts
I0630 20:51:31.068804 166730 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem, removing ...
I0630 20:51:31.068819 166730 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem
I0630 20:51:31.068873 166730 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem (1082 bytes)
I0630 20:51:31.068967 166730 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem, removing ...
I0630 20:51:31.068984 166730 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem
I0630 20:51:31.069024 166730 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem (1123 bytes)
I0630 20:51:31.069154 166730 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem, removing ...
I0630 20:51:31.069170 166730 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem
I0630 20:51:31.069207 166730 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem (1679 bytes)
I0630 20:51:31.069255 166730 provision.go:112] generating server cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem org=jenkins.kubernetes-upgrade-20220630205032-9950 san=[192.168.67.2 127.0.0.1 localhost 127.0.0.1 minikube kubernetes-upgrade-20220630205032-9950]
I0630 20:51:31.128341 166730 provision.go:172] copyRemoteCerts
I0630 20:51:31.128404 166730 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0630 20:51:31.128457 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.162442 166730 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/kubernetes-upgrade-20220630205032-9950/id_rsa Username:docker}
I0630 20:51:31.254557 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0630 20:51:31.272454 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem --> /etc/docker/server.pem (1285 bytes)
I0630 20:51:31.288797 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0630 20:51:31.305252 166730 provision.go:86] duration metric: configureAuth took 278.89858ms
I0630 20:51:31.305278 166730 ubuntu.go:193] setting minikube options for container-runtime
I0630 20:51:31.305428 166730 config.go:178] Loaded profile config "kubernetes-upgrade-20220630205032-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:51:31.305449 166730 machine.go:91] provisioned docker machine in 3.63756739s
I0630 20:51:31.305455 166730 start.go:306] post-start starting for "kubernetes-upgrade-20220630205032-9950" (driver="docker")
I0630 20:51:31.305461 166730 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0630 20:51:31.305497 166730 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0630 20:51:31.305531 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.340269 166730 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/kubernetes-upgrade-20220630205032-9950/id_rsa Username:docker}
I0630 20:51:31.424271 166730 ssh_runner.go:195] Run: cat /etc/os-release
I0630 20:51:31.426813 166730 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0630 20:51:31.426834 166730 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0630 20:51:31.426846 166730 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0630 20:51:31.426854 166730 info.go:137] Remote host: Ubuntu 20.04.4 LTS
I0630 20:51:31.426863 166730 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/addons for local assets ...
I0630 20:51:31.426912 166730 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files for local assets ...
I0630 20:51:31.426990 166730 filesync.go:149] local asset: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem -> 99502.pem in /etc/ssl/certs
I0630 20:51:31.427081 166730 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0630 20:51:31.433879 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem --> /etc/ssl/certs/99502.pem (1708 bytes)
I0630 20:51:31.450663 166730 start.go:309] post-start completed in 145.193312ms
I0630 20:51:31.450735 166730 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0630 20:51:31.450774 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.484811 166730 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/kubernetes-upgrade-20220630205032-9950/id_rsa Username:docker}
I0630 20:51:31.569337 166730 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0630 20:51:31.573174 166730 fix.go:57] fixHost completed within 4.585324282s
I0630 20:51:31.573203 166730 start.go:81] releasing machines lock for "kubernetes-upgrade-20220630205032-9950", held for 4.585380278s
I0630 20:51:31.573303 166730 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.605506 166730 ssh_runner.go:195] Run: systemctl --version
I0630 20:51:31.605568 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.605572 166730 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0630 20:51:31.605626 166730 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" kubernetes-upgrade-20220630205032-9950
I0630 20:51:31.641192 166730 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/kubernetes-upgrade-20220630205032-9950/id_rsa Username:docker}
I0630 20:51:31.641502 166730 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49344 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/kubernetes-upgrade-20220630205032-9950/id_rsa Username:docker}
I0630 20:51:31.754899 166730 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0630 20:51:31.769747 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0630 20:51:31.781864 166730 docker.go:179] disabling docker service ...
I0630 20:51:31.781909 166730 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0630 20:51:31.793418 166730 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0630 20:51:31.805271 166730 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0630 20:51:31.894517 166730 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0630 20:51:31.982759 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0630 20:51:31.994518 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0630 20:51:32.010132 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*sandbox_image = .*$|sandbox_image = "k8s.gcr.io/pause:3.7"|' -i /etc/containerd/config.toml"
I0630 20:51:32.017921 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*restrict_oom_score_adj = .*$|restrict_oom_score_adj = false|' -i /etc/containerd/config.toml"
I0630 20:51:32.025982 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*SystemdCgroup = .*$|SystemdCgroup = false|' -i /etc/containerd/config.toml"
I0630 20:51:32.036272 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*conf_dir = .*$|conf_dir = "/etc/cni/net.mk"|' -i /etc/containerd/config.toml"
I0630 20:51:32.045346 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^# imports|imports = ["/etc/containerd/containerd.conf.d/02-containerd.conf"]|' -i /etc/containerd/config.toml"
I0630 20:51:32.055128 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc/containerd/containerd.conf.d && printf %s "dmVyc2lvbiA9IDIK" | base64 -d | sudo tee /etc/containerd/containerd.conf.d/02-containerd.conf"
I0630 20:51:32.070535 166730 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0630 20:51:32.078660 166730 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0630 20:51:32.086631 166730 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0630 20:51:32.177917 166730 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0630 20:51:32.251946 166730 start.go:447] Will wait 60s for socket path /run/containerd/containerd.sock
I0630 20:51:32.252020 166730 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0630 20:51:32.256063 166730 start.go:468] Will wait 60s for crictl version
I0630 20:51:32.256137 166730 ssh_runner.go:195] Run: sudo crictl version
I0630 20:51:32.293501 166730 retry.go:31] will retry after 11.04660288s: Temporary Error: sudo crictl version: Process exited with status 1
stdout:
stderr:
time="2022-06-30T20:51:32Z" level=fatal msg="getting the runtime version: rpc error: code = Unknown desc = server is not initialized yet"
I0630 20:51:43.341147 166730 ssh_runner.go:195] Run: sudo crictl version
I0630 20:51:43.363404 166730 start.go:477] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.6
RuntimeApiVersion: v1alpha2
I0630 20:51:43.363474 166730 ssh_runner.go:195] Run: containerd --version
I0630 20:51:43.391400 166730 ssh_runner.go:195] Run: containerd --version
I0630 20:51:43.554073 166730 out.go:177] * Preparing Kubernetes v1.24.2 on containerd 1.6.6 ...
I0630 20:51:43.602563 166730 cli_runner.go:164] Run: docker network inspect kubernetes-upgrade-20220630205032-9950 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0630 20:51:43.637519 166730 ssh_runner.go:195] Run: grep 192.168.67.1 host.minikube.internal$ /etc/hosts
I0630 20:51:43.640999 166730 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.67.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 20:51:43.653224 166730 out.go:177] - kubelet.cni-conf-dir=/etc/cni/net.mk
I0630 20:51:43.654680 166730 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime containerd
I0630 20:51:43.655032 166730 ssh_runner.go:195] Run: sudo crictl images --output json
I0630 20:51:43.681485 166730 containerd.go:543] couldn't find preloaded image for "k8s.gcr.io/kube-apiserver:v1.24.2". assuming images are not preloaded.
I0630 20:51:43.681542 166730 ssh_runner.go:195] Run: which lz4
I0630 20:51:43.684514 166730 ssh_runner.go:195] Run: stat -c "%s %y" /preloaded.tar.lz4
I0630 20:51:43.687461 166730 ssh_runner.go:352] existence check for /preloaded.tar.lz4: stat -c "%s %y" /preloaded.tar.lz4: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/preloaded.tar.lz4': No such file or directory
I0630 20:51:43.687492 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4 --> /preloaded.tar.lz4 (447741112 bytes)
I0630 20:51:44.463680 166730 containerd.go:490] Took 0.779203 seconds to copy over tarball
I0630 20:51:44.463749 166730 ssh_runner.go:195] Run: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4
I0630 20:51:46.766499 166730 ssh_runner.go:235] Completed: sudo tar -I lz4 -C /var -xf /preloaded.tar.lz4: (2.302722694s)
I0630 20:51:46.861012 166730 containerd.go:497] Took 2.397296 seconds t extract the tarball
I0630 20:51:46.861028 166730 ssh_runner.go:146] rm: /preloaded.tar.lz4
I0630 20:51:46.942844 166730 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0630 20:51:47.021797 166730 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0630 20:51:47.105437 166730 ssh_runner.go:195] Run: sudo crictl images --output json
I0630 20:51:47.132274 166730 cache_images.go:88] LoadImages start: [k8s.gcr.io/kube-apiserver:v1.24.2 k8s.gcr.io/kube-controller-manager:v1.24.2 k8s.gcr.io/kube-scheduler:v1.24.2 k8s.gcr.io/kube-proxy:v1.24.2 k8s.gcr.io/pause:3.7 k8s.gcr.io/etcd:3.5.3-0 k8s.gcr.io/coredns/coredns:v1.8.6 gcr.io/k8s-minikube/storage-provisioner:v5]
I0630 20:51:47.132363 166730 image.go:134] retrieving image: gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:51:47.132382 166730 image.go:134] retrieving image: k8s.gcr.io/kube-proxy:v1.24.2
I0630 20:51:47.132403 166730 image.go:134] retrieving image: k8s.gcr.io/kube-controller-manager:v1.24.2
I0630 20:51:47.132410 166730 image.go:134] retrieving image: k8s.gcr.io/kube-scheduler:v1.24.2
I0630 20:51:47.132456 166730 image.go:134] retrieving image: k8s.gcr.io/kube-apiserver:v1.24.2
I0630 20:51:47.132466 166730 image.go:134] retrieving image: k8s.gcr.io/etcd:3.5.3-0
I0630 20:51:47.132471 166730 image.go:134] retrieving image: k8s.gcr.io/pause:3.7
I0630 20:51:47.132527 166730 image.go:134] retrieving image: k8s.gcr.io/coredns/coredns:v1.8.6
I0630 20:51:47.133673 166730 image.go:177] daemon lookup for k8s.gcr.io/etcd:3.5.3-0: Error: No such image: k8s.gcr.io/etcd:3.5.3-0
I0630 20:51:47.133701 166730 image.go:177] daemon lookup for k8s.gcr.io/kube-controller-manager:v1.24.2: Error: No such image: k8s.gcr.io/kube-controller-manager:v1.24.2
I0630 20:51:47.133732 166730 image.go:177] daemon lookup for k8s.gcr.io/kube-apiserver:v1.24.2: Error: No such image: k8s.gcr.io/kube-apiserver:v1.24.2
I0630 20:51:47.133678 166730 image.go:177] daemon lookup for k8s.gcr.io/kube-scheduler:v1.24.2: Error: No such image: k8s.gcr.io/kube-scheduler:v1.24.2
I0630 20:51:47.133784 166730 image.go:177] daemon lookup for gcr.io/k8s-minikube/storage-provisioner:v5: Error: No such image: gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:51:47.133675 166730 image.go:177] daemon lookup for k8s.gcr.io/coredns/coredns:v1.8.6: Error: No such image: k8s.gcr.io/coredns/coredns:v1.8.6
I0630 20:51:47.133680 166730 image.go:177] daemon lookup for k8s.gcr.io/pause:3.7: Error: No such image: k8s.gcr.io/pause:3.7
I0630 20:51:47.133843 166730 image.go:177] daemon lookup for k8s.gcr.io/kube-proxy:v1.24.2: Error: No such image: k8s.gcr.io/kube-proxy:v1.24.2
I0630 20:51:47.587260 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/coredns/coredns:v1.8.6"
I0630 20:51:47.593540 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/kube-proxy:v1.24.2"
I0630 20:51:47.593544 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/pause:3.7"
I0630 20:51:47.600688 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/kube-controller-manager:v1.24.2"
I0630 20:51:47.604698 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/etcd:3.5.3-0"
I0630 20:51:47.610061 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/kube-scheduler:v1.24.2"
I0630 20:51:47.669038 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep k8s.gcr.io/kube-apiserver:v1.24.2"
I0630 20:51:47.952030 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo ctr -n=k8s.io images check | grep gcr.io/k8s-minikube/storage-provisioner:v5"
I0630 20:51:48.342274 166730 cache_images.go:116] "k8s.gcr.io/coredns/coredns:v1.8.6" needs transfer: "k8s.gcr.io/coredns/coredns:v1.8.6" does not exist at hash "a4ca41631cc7ac19ce1be3ebf0314ac5f47af7c711f17066006db82ee3b75b03" in container runtime
I0630 20:51:48.342327 166730 cri.go:216] Removing image: k8s.gcr.io/coredns/coredns:v1.8.6
I0630 20:51:48.342367 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.357096 166730 cache_images.go:116] "k8s.gcr.io/kube-proxy:v1.24.2" needs transfer: "k8s.gcr.io/kube-proxy:v1.24.2" does not exist at hash "a634548d10b032c2a1d704ef9a2ab04c12b0574afe67ee192b196a7f12da9536" in container runtime
I0630 20:51:48.357140 166730 cri.go:216] Removing image: k8s.gcr.io/kube-proxy:v1.24.2
I0630 20:51:48.357176 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.439467 166730 cache_images.go:116] "k8s.gcr.io/pause:3.7" needs transfer: "k8s.gcr.io/pause:3.7" does not exist at hash "221177c6082a88ea4f6240ab2450d540955ac6f4d5454f0e15751b653ebda165" in container runtime
I0630 20:51:48.439514 166730 cri.go:216] Removing image: k8s.gcr.io/pause:3.7
I0630 20:51:48.439557 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.448014 166730 cache_images.go:116] "k8s.gcr.io/kube-controller-manager:v1.24.2" needs transfer: "k8s.gcr.io/kube-controller-manager:v1.24.2" does not exist at hash "34cdf99b1bb3b3a62c5b4226c3bc0983ab1f13e776269d1872092091b07203df" in container runtime
I0630 20:51:48.448062 166730 cri.go:216] Removing image: k8s.gcr.io/kube-controller-manager:v1.24.2
I0630 20:51:48.448103 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.448119 166730 cache_images.go:116] "k8s.gcr.io/etcd:3.5.3-0" needs transfer: "k8s.gcr.io/etcd:3.5.3-0" does not exist at hash "aebe758cef4cd05b9f8cee39758227714d02f42ef3088023c1e3cd454f927a2b" in container runtime
I0630 20:51:48.448158 166730 cri.go:216] Removing image: k8s.gcr.io/etcd:3.5.3-0
I0630 20:51:48.448197 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.450162 166730 cache_images.go:116] "k8s.gcr.io/kube-scheduler:v1.24.2" needs transfer: "k8s.gcr.io/kube-scheduler:v1.24.2" does not exist at hash "5d725196c1f47e72d2bc7069776d5928b1fb1e4adf09c18997733099aa3663ac" in container runtime
I0630 20:51:48.450226 166730 cri.go:216] Removing image: k8s.gcr.io/kube-scheduler:v1.24.2
I0630 20:51:48.450263 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.541939 166730 cache_images.go:116] "k8s.gcr.io/kube-apiserver:v1.24.2" needs transfer: "k8s.gcr.io/kube-apiserver:v1.24.2" does not exist at hash "d3377ffb7177cc4becce8a534d8547aca9530cb30fac9ebe479b31102f1ba503" in container runtime
I0630 20:51:48.541989 166730 cri.go:216] Removing image: k8s.gcr.io/kube-apiserver:v1.24.2
I0630 20:51:48.542042 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.630143 166730 cache_images.go:116] "gcr.io/k8s-minikube/storage-provisioner:v5" needs transfer: "gcr.io/k8s-minikube/storage-provisioner:v5" does not exist at hash "6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562" in container runtime
I0630 20:51:48.630182 166730 cri.go:216] Removing image: gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:51:48.630228 166730 ssh_runner.go:195] Run: which crictl
I0630 20:51:48.630245 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/coredns/coredns:v1.8.6
I0630 20:51:48.630304 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-proxy:v1.24.2
I0630 20:51:48.630341 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/pause:3.7
I0630 20:51:48.630427 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-controller-manager:v1.24.2
I0630 20:51:48.630438 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/etcd:3.5.3-0
I0630 20:51:48.630500 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-scheduler:v1.24.2
I0630 20:51:48.630518 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-apiserver:v1.24.2
I0630 20:51:51.462351 166730 ssh_runner.go:235] Completed: which crictl: (2.832098591s)
I0630 20:51:51.462395 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/coredns/coredns:v1.8.6: (2.83212097s)
I0630 20:51:51.462407 166730 ssh_runner.go:195] Run: sudo /usr/bin/crictl rmi gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:51:51.462415 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/coredns/coredns_v1.8.6
I0630 20:51:51.462463 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-proxy:v1.24.2: (2.832131931s)
I0630 20:51:51.462485 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-proxy_v1.24.2
I0630 20:51:51.462488 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/coredns_v1.8.6
I0630 20:51:51.462500 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/pause:3.7: (2.832140439s)
I0630 20:51:51.462516 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/pause_3.7
I0630 20:51:51.462556 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-proxy_v1.24.2
I0630 20:51:51.462569 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/etcd:3.5.3-0: (2.832115956s)
I0630 20:51:51.462572 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/pause_3.7
I0630 20:51:51.462578 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/etcd_3.5.3-0
I0630 20:51:51.462619 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-controller-manager:v1.24.2: (2.832172571s)
I0630 20:51:51.462626 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-controller-manager_v1.24.2
I0630 20:51:51.462632 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/etcd_3.5.3-0
I0630 20:51:51.462662 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-scheduler:v1.24.2: (2.832143134s)
I0630 20:51:51.462667 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-controller-manager_v1.24.2
I0630 20:51:51.462672 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-scheduler_v1.24.2
I0630 20:51:51.462731 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-scheduler_v1.24.2
I0630 20:51:51.498315 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5
I0630 20:51:51.498409 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/storage-provisioner_v5
I0630 20:51:51.498429 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-proxy_v1.24.2: stat -c "%s %y" /var/lib/minikube/images/kube-proxy_v1.24.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-proxy_v1.24.2': No such file or directory
I0630 20:51:51.498332 166730 ssh_runner.go:235] Completed: sudo /usr/bin/crictl rmi k8s.gcr.io/kube-apiserver:v1.24.2: (2.867794285s)
I0630 20:51:51.498461 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-proxy_v1.24.2 --> /var/lib/minikube/images/kube-proxy_v1.24.2 (39518208 bytes)
I0630 20:51:51.498466 166730 cache_images.go:286] Loading image from: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-apiserver_v1.24.2
I0630 20:51:51.498504 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/coredns_v1.8.6: stat -c "%s %y" /var/lib/minikube/images/coredns_v1.8.6: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/coredns_v1.8.6': No such file or directory
I0630 20:51:51.498526 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/coredns/coredns_v1.8.6 --> /var/lib/minikube/images/coredns_v1.8.6 (13586432 bytes)
I0630 20:51:51.498536 166730 ssh_runner.go:195] Run: stat -c "%s %y" /var/lib/minikube/images/kube-apiserver_v1.24.2
I0630 20:51:51.498618 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/pause_3.7: stat -c "%s %y" /var/lib/minikube/images/pause_3.7: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/pause_3.7': No such file or directory
I0630 20:51:51.498640 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/pause_3.7 --> /var/lib/minikube/images/pause_3.7 (311296 bytes)
I0630 20:51:51.498686 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-controller-manager_v1.24.2: stat -c "%s %y" /var/lib/minikube/images/kube-controller-manager_v1.24.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-controller-manager_v1.24.2': No such file or directory
I0630 20:51:51.498710 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-controller-manager_v1.24.2 --> /var/lib/minikube/images/kube-controller-manager_v1.24.2 (31037952 bytes)
I0630 20:51:51.498755 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-scheduler_v1.24.2: stat -c "%s %y" /var/lib/minikube/images/kube-scheduler_v1.24.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-scheduler_v1.24.2': No such file or directory
I0630 20:51:51.498777 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-scheduler_v1.24.2 --> /var/lib/minikube/images/kube-scheduler_v1.24.2 (15491584 bytes)
I0630 20:51:51.498758 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/etcd_3.5.3-0: stat -c "%s %y" /var/lib/minikube/images/etcd_3.5.3-0: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/etcd_3.5.3-0': No such file or directory
I0630 20:51:51.498804 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/etcd_3.5.3-0 --> /var/lib/minikube/images/etcd_3.5.3-0 (102146048 bytes)
I0630 20:51:51.509709 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/storage-provisioner_v5: stat -c "%s %y" /var/lib/minikube/images/storage-provisioner_v5: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/storage-provisioner_v5': No such file or directory
I0630 20:51:51.509749 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 --> /var/lib/minikube/images/storage-provisioner_v5 (9060352 bytes)
I0630 20:51:51.509808 166730 ssh_runner.go:352] existence check for /var/lib/minikube/images/kube-apiserver_v1.24.2: stat -c "%s %y" /var/lib/minikube/images/kube-apiserver_v1.24.2: Process exited with status 1
stdout:
stderr:
stat: cannot stat '/var/lib/minikube/images/kube-apiserver_v1.24.2': No such file or directory
I0630 20:51:51.509833 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-apiserver_v1.24.2 --> /var/lib/minikube/images/kube-apiserver_v1.24.2 (33798144 bytes)
I0630 20:51:51.542946 166730 containerd.go:227] Loading image: /var/lib/minikube/images/pause_3.7
I0630 20:51:51.543017 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/pause_3.7
I0630 20:51:51.829369 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/pause_3.7 from cache
I0630 20:51:51.829437 166730 containerd.go:227] Loading image: /var/lib/minikube/images/storage-provisioner_v5
I0630 20:51:51.829511 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/storage-provisioner_v5
I0630 20:51:52.555734 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/gcr.io/k8s-minikube/storage-provisioner_v5 from cache
I0630 20:51:52.555787 166730 containerd.go:227] Loading image: /var/lib/minikube/images/coredns_v1.8.6
I0630 20:51:52.555854 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/coredns_v1.8.6
I0630 20:51:53.203567 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/coredns/coredns_v1.8.6 from cache
I0630 20:51:53.203601 166730 containerd.go:227] Loading image: /var/lib/minikube/images/kube-scheduler_v1.24.2
I0630 20:51:53.203636 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-scheduler_v1.24.2
I0630 20:51:54.025760 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-scheduler_v1.24.2 from cache
I0630 20:51:54.025801 166730 containerd.go:227] Loading image: /var/lib/minikube/images/kube-controller-manager_v1.24.2
I0630 20:51:54.025839 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-controller-manager_v1.24.2
I0630 20:51:55.298907 166730 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-controller-manager_v1.24.2: (1.273037745s)
I0630 20:51:55.298940 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-controller-manager_v1.24.2 from cache
I0630 20:51:55.298970 166730 containerd.go:227] Loading image: /var/lib/minikube/images/kube-apiserver_v1.24.2
I0630 20:51:55.299016 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-apiserver_v1.24.2
I0630 20:51:56.693542 166730 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-apiserver_v1.24.2: (1.394496936s)
I0630 20:51:56.693574 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-apiserver_v1.24.2 from cache
I0630 20:51:56.693601 166730 containerd.go:227] Loading image: /var/lib/minikube/images/kube-proxy_v1.24.2
I0630 20:51:56.693659 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/kube-proxy_v1.24.2
I0630 20:51:57.496211 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/kube-proxy_v1.24.2 from cache
I0630 20:51:57.496260 166730 containerd.go:227] Loading image: /var/lib/minikube/images/etcd_3.5.3-0
I0630 20:51:57.496303 166730 ssh_runner.go:195] Run: sudo ctr -n=k8s.io images import /var/lib/minikube/images/etcd_3.5.3-0
I0630 20:52:01.352409 166730 ssh_runner.go:235] Completed: sudo ctr -n=k8s.io images import /var/lib/minikube/images/etcd_3.5.3-0: (3.856075384s)
I0630 20:52:01.352443 166730 cache_images.go:315] Transferred and loaded /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/images/amd64/k8s.gcr.io/etcd_3.5.3-0 from cache
I0630 20:52:01.352477 166730 cache_images.go:123] Successfully loaded all cached images
I0630 20:52:01.352483 166730 cache_images.go:92] LoadImages completed in 14.220178423s
I0630 20:52:01.352541 166730 ssh_runner.go:195] Run: sudo crictl info
I0630 20:52:01.380791 166730 cni.go:95] Creating CNI manager for ""
I0630 20:52:01.380813 166730 cni.go:162] "docker" driver + containerd runtime found, recommending kindnet
I0630 20:52:01.380824 166730 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0630 20:52:01.380836 166730 kubeadm.go:158] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.67.2 APIServerPort:8443 KubernetesVersion:v1.24.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:kubernetes-upgrade-20220630205032-9950 NodeName:kubernetes-upgrade-20220630205032-9950 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.67.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.67.2 CgroupDriver:cg
roupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0630 20:52:01.380989 166730 kubeadm.go:162] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "kubernetes-upgrade-20220630205032-9950"
kubeletExtraArgs:
node-ip: 192.168.67.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.24.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0630 20:52:01.381121 166730 kubeadm.go:961] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.24.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cni-conf-dir=/etc/cni/net.mk --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=kubernetes-upgrade-20220630205032-9950 --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.67.2 --runtime-request-timeout=15m
[Install]
config:
{KubernetesVersion:v1.24.2 ClusterName:kubernetes-upgrade-20220630205032-9950 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:}
I0630 20:52:01.381186 166730 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.24.2
I0630 20:52:01.388900 166730 binaries.go:44] Found k8s binaries, skipping transfer
I0630 20:52:01.388975 166730 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0630 20:52:01.396955 166730 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (562 bytes)
I0630 20:52:01.410556 166730 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0630 20:52:01.424249 166730 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2067 bytes)
I0630 20:52:01.437849 166730 ssh_runner.go:195] Run: grep 192.168.67.2 control-plane.minikube.internal$ /etc/hosts
I0630 20:52:01.441016 166730 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.67.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 20:52:01.451256 166730 certs.go:54] Setting up /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950 for IP: 192.168.67.2
I0630 20:52:01.451384 166730 certs.go:182] skipping minikubeCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.key
I0630 20:52:01.451427 166730 certs.go:182] skipping proxyClientCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.key
I0630 20:52:01.451516 166730 certs.go:298] skipping minikube-user signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/client.key
I0630 20:52:01.451589 166730 certs.go:298] skipping minikube signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/apiserver.key.c7fa3a9e
I0630 20:52:01.451641 166730 certs.go:298] skipping aggregator signed cert generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/proxy-client.key
I0630 20:52:01.451785 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950.pem (1338 bytes)
W0630 20:52:01.451824 166730 certs.go:384] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950_empty.pem, impossibly tiny 0 bytes
I0630 20:52:01.451853 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem (1679 bytes)
I0630 20:52:01.451900 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem (1082 bytes)
I0630 20:52:01.451948 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem (1123 bytes)
I0630 20:52:01.451981 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/key.pem (1679 bytes)
I0630 20:52:01.452033 166730 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem (1708 bytes)
I0630 20:52:01.452704 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0630 20:52:01.471509 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0630 20:52:01.489340 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0630 20:52:01.507508 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0630 20:52:01.525863 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0630 20:52:01.544238 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0630 20:52:01.563954 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0630 20:52:01.583225 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0630 20:52:01.604725 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem --> /usr/share/ca-certificates/99502.pem (1708 bytes)
I0630 20:52:01.625613 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0630 20:52:01.644354 166730 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950.pem --> /usr/share/ca-certificates/9950.pem (1338 bytes)
I0630 20:52:01.662674 166730 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0630 20:52:01.676058 166730 ssh_runner.go:195] Run: openssl version
I0630 20:52:01.682146 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/99502.pem && ln -fs /usr/share/ca-certificates/99502.pem /etc/ssl/certs/99502.pem"
I0630 20:52:01.690320 166730 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/99502.pem
I0630 20:52:01.693950 166730 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Jun 30 20:26 /usr/share/ca-certificates/99502.pem
I0630 20:52:01.694042 166730 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/99502.pem
I0630 20:52:01.699253 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/99502.pem /etc/ssl/certs/3ec20f2e.0"
I0630 20:52:01.707280 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0630 20:52:01.716191 166730 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0630 20:52:01.719579 166730 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Jun 30 20:22 /usr/share/ca-certificates/minikubeCA.pem
I0630 20:52:01.719639 166730 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0630 20:52:01.724961 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0630 20:52:01.733025 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/9950.pem && ln -fs /usr/share/ca-certificates/9950.pem /etc/ssl/certs/9950.pem"
I0630 20:52:01.741314 166730 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/9950.pem
I0630 20:52:01.744800 166730 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Jun 30 20:26 /usr/share/ca-certificates/9950.pem
I0630 20:52:01.744867 166730 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/9950.pem
I0630 20:52:01.749927 166730 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/9950.pem /etc/ssl/certs/51391683.0"
I0630 20:52:01.757508 166730 kubeadm.go:395] StartCluster: {Name:kubernetes-upgrade-20220630205032-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2200 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:kubernetes-upgrade-20220630205032-9950 Namespace:defaul
t APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[{Component:kubelet Key:cni-conf-dir Value:/etc/cni/net.mk}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.67.2 Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] Mo
untPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 20:52:01.757641 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0630 20:52:01.757695 166730 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0630 20:52:01.786766 166730 cri.go:87] found id: ""
I0630 20:52:01.786847 166730 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0630 20:52:01.794672 166730 kubeadm.go:410] found existing configuration files, will attempt cluster restart
I0630 20:52:01.794697 166730 kubeadm.go:626] restartCluster start
I0630 20:52:01.794749 166730 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0630 20:52:01.802317 166730 kubeadm.go:127] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0630 20:52:01.802881 166730 kubeconfig.go:116] verify returned: extract IP: "kubernetes-upgrade-20220630205032-9950" does not appear in /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 20:52:01.803156 166730 kubeconfig.go:127] "kubernetes-upgrade-20220630205032-9950" context is missing from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig - will repair!
I0630 20:52:01.803774 166730 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig: {Name:mk34e4a4e7b04f624b9adbda4f77ee20b38308ba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:52:01.805311 166730 kapi.go:59] client config for kubernetes-upgrade-20220630205032-9950: &rest.Config{Host:"https://192.168.67.2:8443", APIPath:"", ContentConfig:rest.ContentConfig{AcceptContentTypes:"", ContentType:"", GroupVersion:(*schema.GroupVersion)(nil), NegotiatedSerializer:runtime.NegotiatedSerializer(nil)}, Username:"", Password:"", BearerToken:"", BearerTokenFile:"", Impersonate:rest.ImpersonationConfig{UserName:"", UID:"", Groups:[]string(nil), Extra:map[string][]string(nil)}, AuthProvider:<nil>, AuthConfigPersister:rest.AuthProviderConfigPersister(nil), ExecProvider:<nil>, TLSClientConfig:rest.sanitizedTLSClientConfig{Insecure:false, ServerName:"", CertFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kubernetes-upgrade-20220630205032-9950/client.crt", KeyFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/
profiles/kubernetes-upgrade-20220630205032-9950/client.key", CAFile:"/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.crt", CertData:[]uint8(nil), KeyData:[]uint8(nil), CAData:[]uint8(nil), NextProtos:[]string(nil)}, UserAgent:"", DisableCompression:false, Transport:http.RoundTripper(nil), WrapTransport:(transport.WrapperFunc)(0x173d420), QPS:0, Burst:0, RateLimiter:flowcontrol.RateLimiter(nil), WarningHandler:rest.WarningHandler(nil), Timeout:0, Dial:(func(context.Context, string, string) (net.Conn, error))(nil), Proxy:(func(*http.Request) (*url.URL, error))(nil)}
I0630 20:52:01.806033 166730 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0630 20:52:01.813898 166730 kubeadm.go:593] needs reconfigure: configs differ:
-- stdout --
--- /var/tmp/minikube/kubeadm.yaml 2022-06-30 20:50:52.985425599 +0000
+++ /var/tmp/minikube/kubeadm.yaml.new 2022-06-30 20:52:01.431872633 +0000
@@ -1,4 +1,4 @@
-apiVersion: kubeadm.k8s.io/v1beta1
+apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.67.2
@@ -17,7 +17,7 @@
node-ip: 192.168.67.2
taints: []
---
-apiVersion: kubeadm.k8s.io/v1beta1
+apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.67.2"]
@@ -31,16 +31,14 @@
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
-clusterName: kubernetes-upgrade-20220630205032-9950
+clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
-dns:
- type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
- listen-metrics-urls: http://127.0.0.1:2381,http://192.168.67.2:2381
-kubernetesVersion: v1.16.0
+ proxy-refresh-interval: "70000"
+kubernetesVersion: v1.24.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
-- /stdout --
I0630 20:52:01.813927 166730 kubeadm.go:1092] stopping kube-system containers ...
I0630 20:52:01.813943 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name: Namespaces:[kube-system]}
I0630 20:52:01.813998 166730 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0630 20:52:01.840698 166730 cri.go:87] found id: ""
I0630 20:52:01.840770 166730 ssh_runner.go:195] Run: sudo systemctl stop kubelet
I0630 20:52:01.851159 166730 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:52:01.859638 166730 kubeadm.go:155] found existing configuration files:
-rw------- 1 root root 5755 Jun 30 20:50 /etc/kubernetes/admin.conf
-rw------- 1 root root 5795 Jun 30 20:50 /etc/kubernetes/controller-manager.conf
-rw------- 1 root root 5955 Jun 30 20:50 /etc/kubernetes/kubelet.conf
-rw------- 1 root root 5739 Jun 30 20:50 /etc/kubernetes/scheduler.conf
I0630 20:52:01.859713 166730 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0630 20:52:01.867622 166730 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0630 20:52:01.875482 166730 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0630 20:52:01.883550 166730 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0630 20:52:01.891479 166730 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0630 20:52:01.899553 166730 kubeadm.go:703] reconfiguring cluster from /var/tmp/minikube/kubeadm.yaml
I0630 20:52:01.899583 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init phase certs all --config /var/tmp/minikube/kubeadm.yaml"
I0630 20:52:01.945951 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init phase kubeconfig all --config /var/tmp/minikube/kubeadm.yaml"
I0630 20:52:02.438142 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init phase kubelet-start --config /var/tmp/minikube/kubeadm.yaml"
I0630 20:52:02.642118 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init phase control-plane all --config /var/tmp/minikube/kubeadm.yaml"
I0630 20:52:02.697519 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init phase etcd local --config /var/tmp/minikube/kubeadm.yaml"
I0630 20:52:02.772854 166730 api_server.go:51] waiting for apiserver process to appear ...
I0630 20:52:02.772934 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:03.282155 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:03.782123 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:04.282248 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:04.782262 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:05.282402 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:05.782265 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:06.281862 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:06.782165 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:07.282074 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:07.782263 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:08.282314 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:08.782266 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:09.281607 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:09.782157 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:10.281762 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:10.782503 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:11.281768 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:11.781936 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:12.281667 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:12.782289 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:13.281813 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:13.782293 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:14.281683 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:14.782273 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:15.282263 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:15.781955 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:16.282016 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:16.781836 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:17.282239 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:17.782241 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:18.281825 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:18.781760 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:19.281711 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:19.782305 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:20.282076 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:20.781956 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:21.282322 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:21.782517 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:22.281725 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:22.782567 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:23.281648 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:23.782283 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:24.281749 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:24.781688 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:25.282286 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:25.782284 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:26.281770 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:26.781637 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:27.282264 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:27.781697 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:28.282261 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:28.782254 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:29.282275 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:29.781883 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:30.282267 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:30.782344 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:31.282256 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:31.782495 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:32.282260 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:32.782178 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:33.281639 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:33.782271 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:34.282242 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:34.782250 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:35.281624 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:35.782258 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:36.281917 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:36.781626 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:37.282264 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:37.781620 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:38.282219 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:38.781620 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:39.282263 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:39.782271 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:40.282226 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:40.781977 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:41.282255 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:41.781704 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:42.281765 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:42.782253 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:43.282553 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:43.782308 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:44.281968 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:44.781622 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:45.282257 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:45.781599 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:46.281948 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:46.782135 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:47.282277 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:47.782047 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:48.282249 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:48.782251 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:49.282256 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:49.781780 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:50.281875 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:50.782101 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:51.282294 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:51.782354 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:52.282249 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:52.782172 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:53.282495 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:53.782263 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:54.282292 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:54.782252 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:55.281769 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:55.782263 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:56.282249 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:56.781747 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:57.281598 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:57.781790 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:58.281825 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:58.781632 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:59.281641 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:52:59.781876 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:00.282315 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:00.782248 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:01.282260 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:01.782278 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:02.281839 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:02.782253 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:02.782329 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:02.810918 166730 cri.go:87] found id: ""
I0630 20:53:02.810951 166730 logs.go:274] 0 containers: []
W0630 20:53:02.810960 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:02.810967 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:02.811021 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:02.843332 166730 cri.go:87] found id: ""
I0630 20:53:02.843356 166730 logs.go:274] 0 containers: []
W0630 20:53:02.843364 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:02.843371 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:02.843418 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:02.875498 166730 cri.go:87] found id: ""
I0630 20:53:02.875521 166730 logs.go:274] 0 containers: []
W0630 20:53:02.875528 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:02.875533 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:02.875587 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:02.898982 166730 cri.go:87] found id: ""
I0630 20:53:02.899016 166730 logs.go:274] 0 containers: []
W0630 20:53:02.899025 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:02.899032 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:02.899074 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:02.921725 166730 cri.go:87] found id: ""
I0630 20:53:02.921752 166730 logs.go:274] 0 containers: []
W0630 20:53:02.921758 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:02.921764 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:02.921803 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:02.948725 166730 cri.go:87] found id: ""
I0630 20:53:02.948752 166730 logs.go:274] 0 containers: []
W0630 20:53:02.948760 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:02.948768 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:02.948825 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:02.973923 166730 cri.go:87] found id: ""
I0630 20:53:02.973950 166730 logs.go:274] 0 containers: []
W0630 20:53:02.973958 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:02.973966 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:02.974032 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:02.999773 166730 cri.go:87] found id: ""
I0630 20:53:02.999796 166730 logs.go:274] 0 containers: []
W0630 20:53:02.999803 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:02.999811 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:02.999824 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:03.052560 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:02 kubernetes-upgrade-20220630205032-9950 kubelet[2339]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:03.107990 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:03.108021 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:03.123360 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:03.123400 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:03.188388 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:03.188416 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:03.188430 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:03.226926 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:03.226966 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:03.266833 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:03.266857 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:03.266968 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:03.266987 166730 out.go:239] Jun 30 20:53:02 kubernetes-upgrade-20220630205032-9950 kubelet[2339]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:02 kubernetes-upgrade-20220630205032-9950 kubelet[2339]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:03.267052 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:03.267064 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:53:13.268087 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:13.281926 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:13.282001 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:13.308407 166730 cri.go:87] found id: ""
I0630 20:53:13.308436 166730 logs.go:274] 0 containers: []
W0630 20:53:13.308446 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:13.308453 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:13.308503 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:13.331806 166730 cri.go:87] found id: ""
I0630 20:53:13.331838 166730 logs.go:274] 0 containers: []
W0630 20:53:13.331848 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:13.331856 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:13.331908 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:13.359136 166730 cri.go:87] found id: ""
I0630 20:53:13.359161 166730 logs.go:274] 0 containers: []
W0630 20:53:13.359168 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:13.359173 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:13.359216 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:13.382834 166730 cri.go:87] found id: ""
I0630 20:53:13.382864 166730 logs.go:274] 0 containers: []
W0630 20:53:13.382873 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:13.382881 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:13.382924 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:13.405788 166730 cri.go:87] found id: ""
I0630 20:53:13.405816 166730 logs.go:274] 0 containers: []
W0630 20:53:13.405824 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:13.405831 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:13.405887 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:13.428499 166730 cri.go:87] found id: ""
I0630 20:53:13.428523 166730 logs.go:274] 0 containers: []
W0630 20:53:13.428532 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:13.428540 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:13.428593 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:13.452095 166730 cri.go:87] found id: ""
I0630 20:53:13.452126 166730 logs.go:274] 0 containers: []
W0630 20:53:13.452136 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:13.452144 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:13.452186 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:13.476167 166730 cri.go:87] found id: ""
I0630 20:53:13.476191 166730 logs.go:274] 0 containers: []
W0630 20:53:13.476197 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:13.476204 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:13.476214 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:13.525853 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:13.525875 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:13.525885 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:13.563912 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:13.563950 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:13.591009 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:13.591034 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:13.635289 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:13 kubernetes-upgrade-20220630205032-9950 kubelet[2636]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:13.680767 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:13.680803 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:13.695643 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:13.695670 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:13.695768 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:13.695781 166730 out.go:239] Jun 30 20:53:13 kubernetes-upgrade-20220630205032-9950 kubelet[2636]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:13 kubernetes-upgrade-20220630205032-9950 kubelet[2636]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:13.695785 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:13.695790 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:53:23.696367 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:23.782318 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:23.782418 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:23.808045 166730 cri.go:87] found id: ""
I0630 20:53:23.808068 166730 logs.go:274] 0 containers: []
W0630 20:53:23.808074 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:23.808080 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:23.808121 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:23.831977 166730 cri.go:87] found id: ""
I0630 20:53:23.832003 166730 logs.go:274] 0 containers: []
W0630 20:53:23.832009 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:23.832014 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:23.832065 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:23.856621 166730 cri.go:87] found id: ""
I0630 20:53:23.856647 166730 logs.go:274] 0 containers: []
W0630 20:53:23.856656 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:23.856664 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:23.856722 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:23.881889 166730 cri.go:87] found id: ""
I0630 20:53:23.881924 166730 logs.go:274] 0 containers: []
W0630 20:53:23.881934 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:23.881942 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:23.882013 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:23.904979 166730 cri.go:87] found id: ""
I0630 20:53:23.905018 166730 logs.go:274] 0 containers: []
W0630 20:53:23.905027 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:23.905034 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:23.905189 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:23.929127 166730 cri.go:87] found id: ""
I0630 20:53:23.929153 166730 logs.go:274] 0 containers: []
W0630 20:53:23.929172 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:23.929182 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:23.929243 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:23.953309 166730 cri.go:87] found id: ""
I0630 20:53:23.953332 166730 logs.go:274] 0 containers: []
W0630 20:53:23.953339 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:23.953345 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:23.953386 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:23.976665 166730 cri.go:87] found id: ""
I0630 20:53:23.976691 166730 logs.go:274] 0 containers: []
W0630 20:53:23.976697 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:23.976706 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:23.976720 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:24.016339 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:24.016374 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:24.044069 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:24.044095 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:24.091787 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:23 kubernetes-upgrade-20220630205032-9950 kubelet[2933]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:24.142983 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:24.143019 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:24.158258 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:24.158292 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:24.206195 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:24.206222 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:24.206234 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:24.206342 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:24.206359 166730 out.go:239] Jun 30 20:53:23 kubernetes-upgrade-20220630205032-9950 kubelet[2933]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:23 kubernetes-upgrade-20220630205032-9950 kubelet[2933]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:24.206365 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:24.206373 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:53:34.207830 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:34.282109 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:34.282186 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:34.316905 166730 cri.go:87] found id: ""
I0630 20:53:34.316933 166730 logs.go:274] 0 containers: []
W0630 20:53:34.316945 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:34.316953 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:34.317011 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:34.348275 166730 cri.go:87] found id: ""
I0630 20:53:34.348315 166730 logs.go:274] 0 containers: []
W0630 20:53:34.348328 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:34.348337 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:34.348383 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:34.380346 166730 cri.go:87] found id: ""
I0630 20:53:34.380375 166730 logs.go:274] 0 containers: []
W0630 20:53:34.380385 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:34.380392 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:34.380447 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:34.406348 166730 cri.go:87] found id: ""
I0630 20:53:34.406370 166730 logs.go:274] 0 containers: []
W0630 20:53:34.406378 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:34.406384 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:34.406450 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:34.436137 166730 cri.go:87] found id: ""
I0630 20:53:34.436163 166730 logs.go:274] 0 containers: []
W0630 20:53:34.436196 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:34.436208 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:34.436277 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:34.519770 166730 cri.go:87] found id: ""
I0630 20:53:34.519799 166730 logs.go:274] 0 containers: []
W0630 20:53:34.519809 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:34.519816 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:34.519870 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:34.578241 166730 cri.go:87] found id: ""
I0630 20:53:34.578270 166730 logs.go:274] 0 containers: []
W0630 20:53:34.578281 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:34.578289 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:34.578339 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:34.605842 166730 cri.go:87] found id: ""
I0630 20:53:34.605865 166730 logs.go:274] 0 containers: []
W0630 20:53:34.605870 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:34.605879 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:34.605891 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:34.680022 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:34.680066 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:34.680083 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:34.721435 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:34.721471 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:34.763829 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:34.763862 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:34.816984 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:34 kubernetes-upgrade-20220630205032-9950 kubelet[3229]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:34.887762 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:34.887803 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:34.903901 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:34.903928 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:34.904028 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:34.904043 166730 out.go:239] Jun 30 20:53:34 kubernetes-upgrade-20220630205032-9950 kubelet[3229]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:34 kubernetes-upgrade-20220630205032-9950 kubelet[3229]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:34.904047 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:34.904052 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:53:44.905002 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:45.281704 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:45.281784 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:45.309453 166730 cri.go:87] found id: ""
I0630 20:53:45.309478 166730 logs.go:274] 0 containers: []
W0630 20:53:45.309486 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:45.309494 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:45.309558 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:45.334824 166730 cri.go:87] found id: ""
I0630 20:53:45.334851 166730 logs.go:274] 0 containers: []
W0630 20:53:45.334859 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:45.334866 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:45.334921 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:45.362287 166730 cri.go:87] found id: ""
I0630 20:53:45.362316 166730 logs.go:274] 0 containers: []
W0630 20:53:45.362325 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:45.362333 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:45.362384 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:45.390494 166730 cri.go:87] found id: ""
I0630 20:53:45.390526 166730 logs.go:274] 0 containers: []
W0630 20:53:45.390535 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:45.390543 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:45.390597 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:45.417289 166730 cri.go:87] found id: ""
I0630 20:53:45.417314 166730 logs.go:274] 0 containers: []
W0630 20:53:45.417321 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:45.417326 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:45.417371 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:45.442817 166730 cri.go:87] found id: ""
I0630 20:53:45.442843 166730 logs.go:274] 0 containers: []
W0630 20:53:45.442852 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:45.442860 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:45.442920 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:45.468875 166730 cri.go:87] found id: ""
I0630 20:53:45.468906 166730 logs.go:274] 0 containers: []
W0630 20:53:45.468915 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:45.468923 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:45.468980 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:45.498382 166730 cri.go:87] found id: ""
I0630 20:53:45.498411 166730 logs.go:274] 0 containers: []
W0630 20:53:45.498420 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:45.498432 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:45.498446 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:45.557423 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:44 kubernetes-upgrade-20220630205032-9950 kubelet[3513]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:45.630364 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:45.630410 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:45.649242 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:45.649277 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:45.710775 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:45.710804 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:45.710819 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:45.764987 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:45.765029 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:45.802778 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:45.802814 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:45.802952 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:45.802973 166730 out.go:239] Jun 30 20:53:44 kubernetes-upgrade-20220630205032-9950 kubelet[3513]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:44 kubernetes-upgrade-20220630205032-9950 kubelet[3513]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:45.802981 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:45.802989 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:53:55.804125 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:53:56.281673 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:53:56.281763 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:53:56.306847 166730 cri.go:87] found id: ""
I0630 20:53:56.306876 166730 logs.go:274] 0 containers: []
W0630 20:53:56.306885 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:53:56.306899 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:53:56.306957 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:53:56.331358 166730 cri.go:87] found id: ""
I0630 20:53:56.331381 166730 logs.go:274] 0 containers: []
W0630 20:53:56.331388 166730 logs.go:276] No container was found matching "etcd"
I0630 20:53:56.331393 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:53:56.331436 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:53:56.355333 166730 cri.go:87] found id: ""
I0630 20:53:56.355355 166730 logs.go:274] 0 containers: []
W0630 20:53:56.355361 166730 logs.go:276] No container was found matching "coredns"
I0630 20:53:56.355367 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:53:56.355419 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:53:56.379449 166730 cri.go:87] found id: ""
I0630 20:53:56.379481 166730 logs.go:274] 0 containers: []
W0630 20:53:56.379492 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:53:56.379501 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:53:56.379558 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:53:56.402948 166730 cri.go:87] found id: ""
I0630 20:53:56.402972 166730 logs.go:274] 0 containers: []
W0630 20:53:56.402978 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:53:56.402983 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:53:56.403028 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:53:56.431564 166730 cri.go:87] found id: ""
I0630 20:53:56.431586 166730 logs.go:274] 0 containers: []
W0630 20:53:56.431594 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:53:56.431601 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:53:56.431663 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:53:56.461809 166730 cri.go:87] found id: ""
I0630 20:53:56.461837 166730 logs.go:274] 0 containers: []
W0630 20:53:56.461847 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:53:56.461854 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:53:56.461904 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:53:56.484441 166730 cri.go:87] found id: ""
I0630 20:53:56.484472 166730 logs.go:274] 0 containers: []
W0630 20:53:56.484481 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:53:56.484492 166730 logs.go:123] Gathering logs for container status ...
I0630 20:53:56.484513 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:53:56.510538 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:53:56.510566 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:53:56.559943 166730 logs.go:138] Found kubelet problem: Jun 30 20:53:56 kubernetes-upgrade-20220630205032-9950 kubelet[3808]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:56.605021 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:53:56.605070 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:53:56.620652 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:53:56.620680 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:53:56.679129 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:53:56.679153 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:53:56.679163 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:53:56.714310 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:56.714342 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:53:56.714464 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:53:56.714481 166730 out.go:239] Jun 30 20:53:56 kubernetes-upgrade-20220630205032-9950 kubelet[3808]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:53:56 kubernetes-upgrade-20220630205032-9950 kubelet[3808]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:53:56.714490 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:53:56.714499 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:54:06.715516 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:54:06.782139 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:54:06.782231 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:54:06.812026 166730 cri.go:87] found id: ""
I0630 20:54:06.812054 166730 logs.go:274] 0 containers: []
W0630 20:54:06.812062 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:54:06.812069 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:54:06.812121 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:54:06.843344 166730 cri.go:87] found id: ""
I0630 20:54:06.843368 166730 logs.go:274] 0 containers: []
W0630 20:54:06.843376 166730 logs.go:276] No container was found matching "etcd"
I0630 20:54:06.843383 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:54:06.843432 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:54:06.880385 166730 cri.go:87] found id: ""
I0630 20:54:06.880412 166730 logs.go:274] 0 containers: []
W0630 20:54:06.880420 166730 logs.go:276] No container was found matching "coredns"
I0630 20:54:06.880427 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:54:06.880477 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:54:06.911728 166730 cri.go:87] found id: ""
I0630 20:54:06.911753 166730 logs.go:274] 0 containers: []
W0630 20:54:06.911762 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:54:06.911769 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:54:06.911826 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:54:06.942194 166730 cri.go:87] found id: ""
I0630 20:54:06.942222 166730 logs.go:274] 0 containers: []
W0630 20:54:06.942231 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:54:06.942238 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:54:06.942287 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:54:06.983763 166730 cri.go:87] found id: ""
I0630 20:54:06.983786 166730 logs.go:274] 0 containers: []
W0630 20:54:06.983794 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:54:06.983801 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:54:06.983854 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:54:07.010756 166730 cri.go:87] found id: ""
I0630 20:54:07.010781 166730 logs.go:274] 0 containers: []
W0630 20:54:07.010788 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:54:07.010794 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:54:07.010855 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:54:07.040477 166730 cri.go:87] found id: ""
I0630 20:54:07.040507 166730 logs.go:274] 0 containers: []
W0630 20:54:07.040515 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:54:07.040527 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:54:07.040543 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:54:07.098608 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:54:07.098631 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:54:07.098642 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:54:07.135890 166730 logs.go:123] Gathering logs for container status ...
I0630 20:54:07.135926 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:54:07.167691 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:54:07.167718 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:54:07.226083 166730 logs.go:138] Found kubelet problem: Jun 30 20:54:06 kubernetes-upgrade-20220630205032-9950 kubelet[4101]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:07.309453 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:54:07.309500 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:54:07.339545 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:07.339642 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:54:07.339849 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:54:07.339906 166730 out.go:239] Jun 30 20:54:06 kubernetes-upgrade-20220630205032-9950 kubelet[4101]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:54:06 kubernetes-upgrade-20220630205032-9950 kubelet[4101]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:07.339915 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:07.339927 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:54:17.341440 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:54:17.781864 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:54:17.781991 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:54:17.814133 166730 cri.go:87] found id: ""
I0630 20:54:17.814163 166730 logs.go:274] 0 containers: []
W0630 20:54:17.814174 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:54:17.814183 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:54:17.814252 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:54:17.839529 166730 cri.go:87] found id: ""
I0630 20:54:17.839563 166730 logs.go:274] 0 containers: []
W0630 20:54:17.839576 166730 logs.go:276] No container was found matching "etcd"
I0630 20:54:17.839584 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:54:17.839636 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:54:17.864351 166730 cri.go:87] found id: ""
I0630 20:54:17.864376 166730 logs.go:274] 0 containers: []
W0630 20:54:17.864383 166730 logs.go:276] No container was found matching "coredns"
I0630 20:54:17.864388 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:54:17.864427 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:54:17.889669 166730 cri.go:87] found id: ""
I0630 20:54:17.889696 166730 logs.go:274] 0 containers: []
W0630 20:54:17.889702 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:54:17.889713 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:54:17.889760 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:54:17.915289 166730 cri.go:87] found id: ""
I0630 20:54:17.915319 166730 logs.go:274] 0 containers: []
W0630 20:54:17.915331 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:54:17.915339 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:54:17.915397 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:54:17.946132 166730 cri.go:87] found id: ""
I0630 20:54:17.946156 166730 logs.go:274] 0 containers: []
W0630 20:54:17.946163 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:54:17.946169 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:54:17.946216 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:54:17.975572 166730 cri.go:87] found id: ""
I0630 20:54:17.975594 166730 logs.go:274] 0 containers: []
W0630 20:54:17.975606 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:54:17.975614 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:54:17.975667 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:54:18.003487 166730 cri.go:87] found id: ""
I0630 20:54:18.003518 166730 logs.go:274] 0 containers: []
W0630 20:54:18.003527 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:54:18.003539 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:54:18.003554 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:54:18.063488 166730 logs.go:138] Found kubelet problem: Jun 30 20:54:17 kubernetes-upgrade-20220630205032-9950 kubelet[4406]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:18.115402 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:54:18.115442 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:54:18.135700 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:54:18.135773 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:54:18.200170 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:54:18.200202 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:54:18.200216 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:54:18.242490 166730 logs.go:123] Gathering logs for container status ...
I0630 20:54:18.242534 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:54:18.288564 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:18.288601 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:54:18.288755 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:54:18.288777 166730 out.go:239] Jun 30 20:54:17 kubernetes-upgrade-20220630205032-9950 kubelet[4406]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:54:17 kubernetes-upgrade-20220630205032-9950 kubelet[4406]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:18.288786 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:18.288795 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:54:28.290269 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:54:28.781730 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:54:28.781809 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:54:28.809443 166730 cri.go:87] found id: ""
I0630 20:54:28.809473 166730 logs.go:274] 0 containers: []
W0630 20:54:28.809483 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:54:28.809493 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:54:28.809550 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:54:28.837363 166730 cri.go:87] found id: ""
I0630 20:54:28.837391 166730 logs.go:274] 0 containers: []
W0630 20:54:28.837398 166730 logs.go:276] No container was found matching "etcd"
I0630 20:54:28.837403 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:54:28.837444 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:54:28.864155 166730 cri.go:87] found id: ""
I0630 20:54:28.864181 166730 logs.go:274] 0 containers: []
W0630 20:54:28.864188 166730 logs.go:276] No container was found matching "coredns"
I0630 20:54:28.864198 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:54:28.864247 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:54:28.890863 166730 cri.go:87] found id: ""
I0630 20:54:28.890887 166730 logs.go:274] 0 containers: []
W0630 20:54:28.890894 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:54:28.890900 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:54:28.890952 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:54:28.916072 166730 cri.go:87] found id: ""
I0630 20:54:28.916103 166730 logs.go:274] 0 containers: []
W0630 20:54:28.916112 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:54:28.916120 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:54:28.916172 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:54:28.940077 166730 cri.go:87] found id: ""
I0630 20:54:28.940106 166730 logs.go:274] 0 containers: []
W0630 20:54:28.940117 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:54:28.940125 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:54:28.940176 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:54:28.964182 166730 cri.go:87] found id: ""
I0630 20:54:28.964218 166730 logs.go:274] 0 containers: []
W0630 20:54:28.964224 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:54:28.964230 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:54:28.964280 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:54:28.988555 166730 cri.go:87] found id: ""
I0630 20:54:28.988583 166730 logs.go:274] 0 containers: []
W0630 20:54:28.988592 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:54:28.988603 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:54:28.988618 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:54:29.037202 166730 logs.go:138] Found kubelet problem: Jun 30 20:54:28 kubernetes-upgrade-20220630205032-9950 kubelet[4692]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:29.085796 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:54:29.085829 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:54:29.102294 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:54:29.102325 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:54:29.153169 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:54:29.153197 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:54:29.153211 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:54:29.205711 166730 logs.go:123] Gathering logs for container status ...
I0630 20:54:29.205757 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:54:29.233456 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:29.233486 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:54:29.233607 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:54:29.233624 166730 out.go:239] Jun 30 20:54:28 kubernetes-upgrade-20220630205032-9950 kubelet[4692]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:54:28 kubernetes-upgrade-20220630205032-9950 kubelet[4692]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:29.233630 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:29.233637 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:54:39.234859 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:54:39.282124 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:54:39.282226 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:54:39.311602 166730 cri.go:87] found id: ""
I0630 20:54:39.311627 166730 logs.go:274] 0 containers: []
W0630 20:54:39.311636 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:54:39.311645 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:54:39.311691 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:54:39.340505 166730 cri.go:87] found id: ""
I0630 20:54:39.340536 166730 logs.go:274] 0 containers: []
W0630 20:54:39.340548 166730 logs.go:276] No container was found matching "etcd"
I0630 20:54:39.340556 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:54:39.340613 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:54:39.369361 166730 cri.go:87] found id: ""
I0630 20:54:39.369391 166730 logs.go:274] 0 containers: []
W0630 20:54:39.369400 166730 logs.go:276] No container was found matching "coredns"
I0630 20:54:39.369408 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:54:39.369468 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:54:39.401447 166730 cri.go:87] found id: ""
I0630 20:54:39.401480 166730 logs.go:274] 0 containers: []
W0630 20:54:39.401490 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:54:39.401499 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:54:39.401558 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:54:39.431105 166730 cri.go:87] found id: ""
I0630 20:54:39.431134 166730 logs.go:274] 0 containers: []
W0630 20:54:39.431142 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:54:39.431150 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:54:39.431205 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:54:39.460705 166730 cri.go:87] found id: ""
I0630 20:54:39.460732 166730 logs.go:274] 0 containers: []
W0630 20:54:39.460739 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:54:39.460746 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:54:39.460794 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:54:39.486054 166730 cri.go:87] found id: ""
I0630 20:54:39.486084 166730 logs.go:274] 0 containers: []
W0630 20:54:39.486093 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:54:39.486101 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:54:39.486158 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:54:39.510776 166730 cri.go:87] found id: ""
I0630 20:54:39.510820 166730 logs.go:274] 0 containers: []
W0630 20:54:39.510829 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:54:39.510841 166730 logs.go:123] Gathering logs for container status ...
I0630 20:54:39.510861 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:54:39.539623 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:54:39.539656 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:54:39.610137 166730 logs.go:138] Found kubelet problem: Jun 30 20:54:38 kubernetes-upgrade-20220630205032-9950 kubelet[4982]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:39.659643 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:54:39.659698 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:54:39.676595 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:54:39.676625 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:54:39.729872 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:54:39.729894 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:54:39.729905 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:54:39.767919 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:39.767948 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:54:39.768071 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:54:39.768090 166730 out.go:239] Jun 30 20:54:38 kubernetes-upgrade-20220630205032-9950 kubelet[4982]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:54:38 kubernetes-upgrade-20220630205032-9950 kubelet[4982]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:39.768097 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:39.768105 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:54:49.769191 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:54:50.282553 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:54:50.282631 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:54:50.322527 166730 cri.go:87] found id: ""
I0630 20:54:50.322552 166730 logs.go:274] 0 containers: []
W0630 20:54:50.322558 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:54:50.322564 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:54:50.322606 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:54:50.355188 166730 cri.go:87] found id: ""
I0630 20:54:50.355220 166730 logs.go:274] 0 containers: []
W0630 20:54:50.355229 166730 logs.go:276] No container was found matching "etcd"
I0630 20:54:50.355237 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:54:50.355295 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:54:50.382332 166730 cri.go:87] found id: ""
I0630 20:54:50.382358 166730 logs.go:274] 0 containers: []
W0630 20:54:50.382365 166730 logs.go:276] No container was found matching "coredns"
I0630 20:54:50.382373 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:54:50.382426 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:54:50.407725 166730 cri.go:87] found id: ""
I0630 20:54:50.407754 166730 logs.go:274] 0 containers: []
W0630 20:54:50.407762 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:54:50.407771 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:54:50.407824 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:54:50.436457 166730 cri.go:87] found id: ""
I0630 20:54:50.436488 166730 logs.go:274] 0 containers: []
W0630 20:54:50.436514 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:54:50.436523 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:54:50.436589 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:54:50.470441 166730 cri.go:87] found id: ""
I0630 20:54:50.470468 166730 logs.go:274] 0 containers: []
W0630 20:54:50.470478 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:54:50.470491 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:54:50.470549 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:54:50.502908 166730 cri.go:87] found id: ""
I0630 20:54:50.502935 166730 logs.go:274] 0 containers: []
W0630 20:54:50.502943 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:54:50.502951 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:54:50.503004 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:54:50.533341 166730 cri.go:87] found id: ""
I0630 20:54:50.533370 166730 logs.go:274] 0 containers: []
W0630 20:54:50.533378 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:54:50.533390 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:54:50.533405 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:54:50.588222 166730 logs.go:138] Found kubelet problem: Jun 30 20:54:50 kubernetes-upgrade-20220630205032-9950 kubelet[5288]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:50.638567 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:54:50.638605 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:54:50.658087 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:54:50.658120 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:54:50.719357 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:54:50.719390 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:54:50.719404 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:54:50.768516 166730 logs.go:123] Gathering logs for container status ...
I0630 20:54:50.768565 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:54:50.811770 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:50.811799 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:54:50.811942 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:54:50.811967 166730 out.go:239] Jun 30 20:54:50 kubernetes-upgrade-20220630205032-9950 kubelet[5288]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:54:50 kubernetes-upgrade-20220630205032-9950 kubelet[5288]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:54:50.811979 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:54:50.811993 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:00.813470 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:01.282167 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:01.282237 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:01.316418 166730 cri.go:87] found id: ""
I0630 20:55:01.316451 166730 logs.go:274] 0 containers: []
W0630 20:55:01.316460 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:01.316468 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:01.316523 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:01.350741 166730 cri.go:87] found id: ""
I0630 20:55:01.350771 166730 logs.go:274] 0 containers: []
W0630 20:55:01.350780 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:01.350787 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:01.350838 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:01.389800 166730 cri.go:87] found id: ""
I0630 20:55:01.389823 166730 logs.go:274] 0 containers: []
W0630 20:55:01.389832 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:01.389840 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:01.389889 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:01.418650 166730 cri.go:87] found id: ""
I0630 20:55:01.418682 166730 logs.go:274] 0 containers: []
W0630 20:55:01.418691 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:01.418699 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:01.418770 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:01.449532 166730 cri.go:87] found id: ""
I0630 20:55:01.449559 166730 logs.go:274] 0 containers: []
W0630 20:55:01.449567 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:01.449575 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:01.449632 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:01.491134 166730 cri.go:87] found id: ""
I0630 20:55:01.491162 166730 logs.go:274] 0 containers: []
W0630 20:55:01.491170 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:01.491178 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:01.491226 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:01.522953 166730 cri.go:87] found id: ""
I0630 20:55:01.522979 166730 logs.go:274] 0 containers: []
W0630 20:55:01.522987 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:01.522994 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:01.523058 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:01.557198 166730 cri.go:87] found id: ""
I0630 20:55:01.557224 166730 logs.go:274] 0 containers: []
W0630 20:55:01.557237 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:01.557248 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:01.557265 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:01.614912 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:01 kubernetes-upgrade-20220630205032-9950 kubelet[5588]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:01.681604 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:01.681651 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:01.702883 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:01.702918 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:01.772344 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:01.772366 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:01.772377 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:01.816698 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:01.816733 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:01.858519 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:01.858554 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:01.858674 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:01.858696 166730 out.go:239] Jun 30 20:55:01 kubernetes-upgrade-20220630205032-9950 kubelet[5588]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:01 kubernetes-upgrade-20220630205032-9950 kubelet[5588]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:01.858703 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:01.858710 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:11.860054 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:12.282142 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:12.282209 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:12.314170 166730 cri.go:87] found id: ""
I0630 20:55:12.314201 166730 logs.go:274] 0 containers: []
W0630 20:55:12.314212 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:12.314220 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:12.314277 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:12.345301 166730 cri.go:87] found id: ""
I0630 20:55:12.345330 166730 logs.go:274] 0 containers: []
W0630 20:55:12.345339 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:12.345347 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:12.345404 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:12.371389 166730 cri.go:87] found id: ""
I0630 20:55:12.371415 166730 logs.go:274] 0 containers: []
W0630 20:55:12.371423 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:12.371432 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:12.371489 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:12.402108 166730 cri.go:87] found id: ""
I0630 20:55:12.402141 166730 logs.go:274] 0 containers: []
W0630 20:55:12.402149 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:12.402157 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:12.402208 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:12.443415 166730 cri.go:87] found id: ""
I0630 20:55:12.443448 166730 logs.go:274] 0 containers: []
W0630 20:55:12.443458 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:12.443467 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:12.443522 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:12.469676 166730 cri.go:87] found id: ""
I0630 20:55:12.469701 166730 logs.go:274] 0 containers: []
W0630 20:55:12.469709 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:12.469717 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:12.469773 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:12.506112 166730 cri.go:87] found id: ""
I0630 20:55:12.506145 166730 logs.go:274] 0 containers: []
W0630 20:55:12.506155 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:12.506163 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:12.506217 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:12.539673 166730 cri.go:87] found id: ""
I0630 20:55:12.539717 166730 logs.go:274] 0 containers: []
W0630 20:55:12.539726 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:12.539737 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:12.539753 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:12.605762 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:11 kubernetes-upgrade-20220630205032-9950 kubelet[5866]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:12.655001 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:12.655038 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:12.673642 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:12.673683 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:12.748987 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:12.749012 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:12.749025 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:12.808308 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:12.808363 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:12.843923 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:12.843957 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:12.844090 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:12.844106 166730 out.go:239] Jun 30 20:55:11 kubernetes-upgrade-20220630205032-9950 kubelet[5866]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:11 kubernetes-upgrade-20220630205032-9950 kubelet[5866]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:12.844124 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:12.844132 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:22.846057 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:23.281665 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:23.281743 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:23.308304 166730 cri.go:87] found id: ""
I0630 20:55:23.308331 166730 logs.go:274] 0 containers: []
W0630 20:55:23.308340 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:23.308348 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:23.308404 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:23.332259 166730 cri.go:87] found id: ""
I0630 20:55:23.332290 166730 logs.go:274] 0 containers: []
W0630 20:55:23.332299 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:23.332307 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:23.332368 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:23.356649 166730 cri.go:87] found id: ""
I0630 20:55:23.356684 166730 logs.go:274] 0 containers: []
W0630 20:55:23.356692 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:23.356700 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:23.356749 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:23.382607 166730 cri.go:87] found id: ""
I0630 20:55:23.382665 166730 logs.go:274] 0 containers: []
W0630 20:55:23.382674 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:23.382682 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:23.382733 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:23.407253 166730 cri.go:87] found id: ""
I0630 20:55:23.407283 166730 logs.go:274] 0 containers: []
W0630 20:55:23.407292 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:23.407300 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:23.407351 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:23.432416 166730 cri.go:87] found id: ""
I0630 20:55:23.432439 166730 logs.go:274] 0 containers: []
W0630 20:55:23.432445 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:23.432451 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:23.432493 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:23.456923 166730 cri.go:87] found id: ""
I0630 20:55:23.456949 166730 logs.go:274] 0 containers: []
W0630 20:55:23.456957 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:23.456965 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:23.457024 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:23.481678 166730 cri.go:87] found id: ""
I0630 20:55:23.481707 166730 logs.go:274] 0 containers: []
W0630 20:55:23.481714 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:23.481724 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:23.481742 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:23.530149 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:23 kubernetes-upgrade-20220630205032-9950 kubelet[6164]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:23.588501 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:23.588550 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:23.603780 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:23.603817 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:23.654235 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:23.654263 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:23.654276 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:23.690689 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:23.690724 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:23.717468 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:23.717493 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:23.717596 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:23.717609 166730 out.go:239] Jun 30 20:55:23 kubernetes-upgrade-20220630205032-9950 kubelet[6164]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:23 kubernetes-upgrade-20220630205032-9950 kubelet[6164]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:23.717615 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:23.717620 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:33.718491 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:33.782515 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:33.782601 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:33.813757 166730 cri.go:87] found id: ""
I0630 20:55:33.813783 166730 logs.go:274] 0 containers: []
W0630 20:55:33.813792 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:33.813801 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:33.813858 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:33.852916 166730 cri.go:87] found id: ""
I0630 20:55:33.852946 166730 logs.go:274] 0 containers: []
W0630 20:55:33.852954 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:33.852962 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:33.853024 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:33.886406 166730 cri.go:87] found id: ""
I0630 20:55:33.886437 166730 logs.go:274] 0 containers: []
W0630 20:55:33.886445 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:33.886453 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:33.886517 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:33.919173 166730 cri.go:87] found id: ""
I0630 20:55:33.919200 166730 logs.go:274] 0 containers: []
W0630 20:55:33.919209 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:33.919217 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:33.919275 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:33.952474 166730 cri.go:87] found id: ""
I0630 20:55:33.952512 166730 logs.go:274] 0 containers: []
W0630 20:55:33.952521 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:33.952529 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:33.952591 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:33.989550 166730 cri.go:87] found id: ""
I0630 20:55:33.989578 166730 logs.go:274] 0 containers: []
W0630 20:55:33.989586 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:33.989594 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:33.989653 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:34.026948 166730 cri.go:87] found id: ""
I0630 20:55:34.026976 166730 logs.go:274] 0 containers: []
W0630 20:55:34.026985 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:34.026992 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:34.027055 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:34.074656 166730 cri.go:87] found id: ""
I0630 20:55:34.074686 166730 logs.go:274] 0 containers: []
W0630 20:55:34.074696 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:34.074708 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:34.074721 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:34.143694 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:33 kubernetes-upgrade-20220630205032-9950 kubelet[6451]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:34.219939 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:34.219989 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:34.237752 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:34.237791 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:34.301853 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:34.301874 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:34.301885 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:34.356929 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:34.356991 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:34.389619 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:34.389652 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:34.389799 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:34.389821 166730 out.go:239] Jun 30 20:55:33 kubernetes-upgrade-20220630205032-9950 kubelet[6451]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:33 kubernetes-upgrade-20220630205032-9950 kubelet[6451]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:34.389828 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:34.389835 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:44.391595 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:44.782426 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:44.782503 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:44.815920 166730 cri.go:87] found id: ""
I0630 20:55:44.815947 166730 logs.go:274] 0 containers: []
W0630 20:55:44.815956 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:44.815966 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:44.816019 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:44.853352 166730 cri.go:87] found id: ""
I0630 20:55:44.853381 166730 logs.go:274] 0 containers: []
W0630 20:55:44.853390 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:44.853398 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:44.853451 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:44.883758 166730 cri.go:87] found id: ""
I0630 20:55:44.883788 166730 logs.go:274] 0 containers: []
W0630 20:55:44.883797 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:44.883808 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:44.883861 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:45.010738 166730 cri.go:87] found id: ""
I0630 20:55:45.010767 166730 logs.go:274] 0 containers: []
W0630 20:55:45.010774 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:45.010780 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:45.010834 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:45.046022 166730 cri.go:87] found id: ""
I0630 20:55:45.046052 166730 logs.go:274] 0 containers: []
W0630 20:55:45.046061 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:45.046069 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:45.046121 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:45.082049 166730 cri.go:87] found id: ""
I0630 20:55:45.082081 166730 logs.go:274] 0 containers: []
W0630 20:55:45.082089 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:45.082100 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:45.082155 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:45.111253 166730 cri.go:87] found id: ""
I0630 20:55:45.111278 166730 logs.go:274] 0 containers: []
W0630 20:55:45.111285 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:45.111291 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:45.111339 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:45.141541 166730 cri.go:87] found id: ""
I0630 20:55:45.141569 166730 logs.go:274] 0 containers: []
W0630 20:55:45.141579 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:45.141591 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:45.141607 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:45.162712 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:45.162748 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:45.229843 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:45.229876 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:45.229891 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:45.279559 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:45.279595 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:45.310726 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:45.310766 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:45.374121 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:44 kubernetes-upgrade-20220630205032-9950 kubelet[6758]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:45.449500 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:45.449538 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:45.449706 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:45.449724 166730 out.go:239] Jun 30 20:55:44 kubernetes-upgrade-20220630205032-9950 kubelet[6758]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:44 kubernetes-upgrade-20220630205032-9950 kubelet[6758]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:45.449730 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:45.449738 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:55.450574 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:55.782242 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:55.782313 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:55.812323 166730 cri.go:87] found id: ""
I0630 20:55:55.812354 166730 logs.go:274] 0 containers: []
W0630 20:55:55.812363 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:55.812371 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:55.812425 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:55.838877 166730 cri.go:87] found id: ""
I0630 20:55:55.838905 166730 logs.go:274] 0 containers: []
W0630 20:55:55.838917 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:55.838925 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:55.838985 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:55.867225 166730 cri.go:87] found id: ""
I0630 20:55:55.867249 166730 logs.go:274] 0 containers: []
W0630 20:55:55.867273 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:55.867282 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:55.867338 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:55.893546 166730 cri.go:87] found id: ""
I0630 20:55:55.893578 166730 logs.go:274] 0 containers: []
W0630 20:55:55.893585 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:55.893591 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:55.893632 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:55.917280 166730 cri.go:87] found id: ""
I0630 20:55:55.917304 166730 logs.go:274] 0 containers: []
W0630 20:55:55.917313 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:55.917320 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:55.917378 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:55.942846 166730 cri.go:87] found id: ""
I0630 20:55:55.942873 166730 logs.go:274] 0 containers: []
W0630 20:55:55.942882 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:55.942890 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:55.942944 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:55.970944 166730 cri.go:87] found id: ""
I0630 20:55:55.970972 166730 logs.go:274] 0 containers: []
W0630 20:55:55.970981 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:55.970990 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:55.971044 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:55.995749 166730 cri.go:87] found id: ""
I0630 20:55:55.995773 166730 logs.go:274] 0 containers: []
W0630 20:55:55.995779 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:55.995787 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:55.995801 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:56.047220 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:55 kubernetes-upgrade-20220630205032-9950 kubelet[7036]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:56.098827 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:56.098857 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:56.114004 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:56.114038 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:56.165221 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:56.165247 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:56.165257 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:56.202035 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:56.202075 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:56.229905 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:56.229939 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:56.230076 166730 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0630 20:55:56.230094 166730 out.go:239] Jun 30 20:55:55 kubernetes-upgrade-20220630205032-9950 kubelet[7036]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
Jun 30 20:55:55 kubernetes-upgrade-20220630205032-9950 kubelet[7036]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:56.230100 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:56.230107 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:56:06.232825 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:56:06.243827 166730 kubeadm.go:630] restartCluster took 4m4.449118713s
W0630 20:56:06.243995 166730 out.go:239] ! Unable to restart cluster, will reset it: apiserver healthz: apiserver process never appeared
! Unable to restart cluster, will reset it: apiserver healthz: apiserver process never appeared
I0630 20:56:06.244034 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0630 20:56:06.946910 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:56:06.958177 166730 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0630 20:56:06.967186 166730 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 20:56:06.967255 166730 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:56:06.974506 166730 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 20:56:06.974558 166730 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 20:56:07.227007 166730 out.go:204] - Generating certificates and keys ...
I0630 20:56:08.883897 166730 out.go:204] - Booting up control plane ...
W0630 20:58:03.899565 166730 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:56:07.008038 7586 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:56:07.008038 7586 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0630 20:58:03.899635 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0630 20:58:04.591252 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:58:04.601287 166730 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 20:58:04.601351 166730 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:58:04.608513 166730 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 20:58:04.608564 166730 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 21:00:00.464899 166730 out.go:204] - Generating certificates and keys ...
I0630 21:00:00.467813 166730 out.go:204] - Booting up control plane ...
I0630 21:00:00.470158 166730 kubeadm.go:397] StartCluster complete in 7m58.712661025s
I0630 21:00:00.470198 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 21:00:00.470254 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 21:00:00.494852 166730 cri.go:87] found id: ""
I0630 21:00:00.494875 166730 logs.go:274] 0 containers: []
W0630 21:00:00.494882 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 21:00:00.494888 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 21:00:00.494937 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 21:00:00.518642 166730 cri.go:87] found id: ""
I0630 21:00:00.518668 166730 logs.go:274] 0 containers: []
W0630 21:00:00.518674 166730 logs.go:276] No container was found matching "etcd"
I0630 21:00:00.518680 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 21:00:00.518728 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 21:00:00.544697 166730 cri.go:87] found id: ""
I0630 21:00:00.544733 166730 logs.go:274] 0 containers: []
W0630 21:00:00.544742 166730 logs.go:276] No container was found matching "coredns"
I0630 21:00:00.544750 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 21:00:00.544808 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 21:00:00.569280 166730 cri.go:87] found id: ""
I0630 21:00:00.569311 166730 logs.go:274] 0 containers: []
W0630 21:00:00.569320 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 21:00:00.569327 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 21:00:00.569383 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 21:00:00.593566 166730 cri.go:87] found id: ""
I0630 21:00:00.593592 166730 logs.go:274] 0 containers: []
W0630 21:00:00.593601 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 21:00:00.593609 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 21:00:00.593665 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 21:00:00.618029 166730 cri.go:87] found id: ""
I0630 21:00:00.618053 166730 logs.go:274] 0 containers: []
W0630 21:00:00.618061 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 21:00:00.618067 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 21:00:00.618111 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 21:00:00.642327 166730 cri.go:87] found id: ""
I0630 21:00:00.642352 166730 logs.go:274] 0 containers: []
W0630 21:00:00.642358 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 21:00:00.642365 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 21:00:00.642412 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 21:00:00.671530 166730 cri.go:87] found id: ""
I0630 21:00:00.671557 166730 logs.go:274] 0 containers: []
W0630 21:00:00.671566 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 21:00:00.671576 166730 logs.go:123] Gathering logs for kubelet ...
I0630 21:00:00.671591 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 21:00:00.735542 166730 logs.go:138] Found kubelet problem: Jun 30 21:00:00 kubernetes-upgrade-20220630205032-9950 kubelet[11593]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 21:00:00.809676 166730 logs.go:123] Gathering logs for dmesg ...
I0630 21:00:00.809714 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 21:00:00.837921 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 21:00:00.837959 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 21:00:00.891715 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 21:00:00.891742 166730 logs.go:123] Gathering logs for containerd ...
I0630 21:00:00.891751 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 21:00:00.937402 166730 logs.go:123] Gathering logs for container status ...
I0630 21:00:00.937438 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0630 21:00:00.966782 166730 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.966835 166730 out.go:239] *
*
W0630 21:00:00.967083 166730 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.967126 166730 out.go:239] *
*
W0630 21:00:00.968129 166730 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0630 21:00:00.970508 166730 out.go:177] X Problems detected in kubelet:
I0630 21:00:00.972058 166730 out.go:177] Jun 30 21:00:00 kubernetes-upgrade-20220630205032-9950 kubelet[11593]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 21:00:00.976653 166730 out.go:177]
W0630 21:00:00.978457 166730 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.978591 166730 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0630 21:00:00.978662 166730 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
* Related issue: https://github.com/kubernetes/minikube/issues/4172
I0630 21:00:00.981508 166730 out.go:177]
** /stderr **
version_upgrade_test.go:252: failed to upgrade with newest k8s version. args: out/minikube-linux-amd64 start -p kubernetes-upgrade-20220630205032-9950 --memory=2200 --kubernetes-version=v1.24.2 --alsologtostderr -v=1 --driver=docker --container-runtime=containerd : exit status 109
version_upgrade_test.go:255: (dbg) Run: kubectl --context kubernetes-upgrade-20220630205032-9950 version --output=json
version_upgrade_test.go:255: (dbg) Non-zero exit: kubectl --context kubernetes-upgrade-20220630205032-9950 version --output=json: exit status 1 (44.228139ms)
-- stdout --
{
"clientVersion": {
"major": "1",
"minor": "24",
"gitVersion": "v1.24.2",
"gitCommit": "f66044f4361b9f1f96f0053dd46cb7dce5e990a8",
"gitTreeState": "clean",
"buildDate": "2022-06-15T14:22:29Z",
"goVersion": "go1.18.3",
"compiler": "gc",
"platform": "linux/amd64"
},
"kustomizeVersion": "v4.5.4"
}
-- /stdout --
** stderr **
The connection to the server 192.168.67.2:8443 was refused - did you specify the right host or port?
** /stderr **
version_upgrade_test.go:257: error running kubectl: exit status 1
panic.go:482: *** TestKubernetesUpgrade FAILED at 2022-06-30 21:00:01.17155219 +0000 UTC m=+2349.410690452
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestKubernetesUpgrade]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect kubernetes-upgrade-20220630205032-9950
helpers_test.go:235: (dbg) docker inspect kubernetes-upgrade-20220630205032-9950:
-- stdout --
[
{
"Id": "3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9",
"Created": "2022-06-30T20:50:46.703399671Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 167062,
"ExitCode": 0,
"Error": "",
"StartedAt": "2022-06-30T20:51:27.571953859Z",
"FinishedAt": "2022-06-30T20:51:25.512851125Z"
},
"Image": "sha256:573e7be5768273a7845baee1ae90fa2e33b83b10a7fbb0f0f41efbf29b53d1f1",
"ResolvConfPath": "/var/lib/docker/containers/3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9/hostname",
"HostsPath": "/var/lib/docker/containers/3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9/hosts",
"LogPath": "/var/lib/docker/containers/3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9/3933a5e883763c8f5e9da8b4882268e07350288449ee510c219ba0b884c94ad9-json.log",
"Name": "/kubernetes-upgrade-20220630205032-9950",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"kubernetes-upgrade-20220630205032-9950:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "kubernetes-upgrade-20220630205032-9950",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/6a1120910baa8020db8ace8ab2c43ee409ff31973d68110156b9688f9732111b-init/diff:/var/lib/docker/overlay2/8b88d9f6f12c48be642a133456203e58549a93f1da67d0349744bd0f78917fba/diff:/var/lib/docker/overlay2/2bcbb3b4718a64188802301a25d7f413bb307cb16d077e7e5af799f9152a2970/diff:/var/lib/docker/overlay2/70b0888bf22fcef1c004704b71288ac581cf26a3b3b19fbca3da296ba10d6978/diff:/var/lib/docker/overlay2/7c5332f38fe652f379ef9b6d23e795158e9994317cfb547b581dd2b1fdf1303b/diff:/var/lib/docker/overlay2/0ddb5a3abf433b644a8bfa124e3c87878eed67878faa35493f2befc2b392e9cb/diff:/var/lib/docker/overlay2/3b7cda5fb0232fdeecbec0531ac704a59eb38fe57664c25777193b07e2df0077/diff:/var/lib/docker/overlay2/a14687ada786c65465b9924ae2d77ba27afc07942851cafaade28e80dfe9da16/diff:/var/lib/docker/overlay2/c49a9b7157aa6e8714e192b2849866d035b59e18de462c0ee9b72f898cc5a485/diff:/var/lib/docker/overlay2/efd71a50e3a1044a6d1c4c1a7d4d91d9f070c066e03e51c6c73a5764b778ad72/diff:/var/lib/docker/overlay2/55476a
1c0a20aa4238a0bb0e43c6402c33dcad312938ad48ee81be0160e12a49/diff:/var/lib/docker/overlay2/c48c4a348a797b9a38f01568a6adbcb4b05b4f56f7b986a4bf1b509d8cb00732/diff:/var/lib/docker/overlay2/568e3f354ab0428de5cf82d1fdfb780c3377f5cc55d9f524731c53e39e0ecae7/diff:/var/lib/docker/overlay2/2857943e320cdcdae9af3d8c2acda1fb61e2f142a88b8b902908071cea67816e/diff:/var/lib/docker/overlay2/bdd9a57f0b65e65c5a16395f239425e794c678b0e8db2f4a368e0a9427fe5116/diff:/var/lib/docker/overlay2/b150183f1b73e8c13c751911bcee09fed8ef973478f4dc3cfdb9e509118bf1ea/diff:/var/lib/docker/overlay2/0a053e7c29417cd6feafb351c7944e81342a52898871bc779f7bf61868c78ec4/diff:/var/lib/docker/overlay2/d3c0de3a44d75d0a05acb7843d53deba5ce0457649d6ed757da9b4c532a81e4f/diff:/var/lib/docker/overlay2/53f6170132880e1697dd802b8577382b31c6f1ba144256f8cc62542998842a6b/diff:/var/lib/docker/overlay2/b5c0be6d8efdb1fcb12e73b8f72853f77baa4bc014ac2c1e28b7b972b07f3325/diff:/var/lib/docker/overlay2/36c908f082a1e5388bb25277f1c957eaecf70fcd958080422fa85a3105724fb2/diff:/var/lib/d
ocker/overlay2/b147fe9dce365cfaf9bfeca2e370829e4d87e2671c133a9836ecb02a85422cc7/diff:/var/lib/docker/overlay2/65f488eb9181a721f01b333f5fa0cb91d216abd7842a9ae193bd95316f378721/diff:/var/lib/docker/overlay2/b04269aa247b68912ddc5b1c98ad55d67af4103f576a900eef4069c196b8a263/diff:/var/lib/docker/overlay2/b714c37e15257ae9d0e15d271dd18a330c38ad765901d9b7e7e3feb73e1b03e4/diff:/var/lib/docker/overlay2/3c253b721193edeaade7bba7ce25ff687c889b8fa86e9489c7ee20d5624e6c5b/diff:/var/lib/docker/overlay2/7f67b74e101fa21dd320393c10576eefa4ce1cec2e210bff405784fa45b360f3/diff:/var/lib/docker/overlay2/c3b2adaa63a06a5e6ed966d7b6f20a757aeadd3f25eb09ef3c7e9a66a8904143/diff:/var/lib/docker/overlay2/ddb5d653d19e8cafc4625a8fe9597667d98b277a7f679e532ad84caa2c6da636/diff:/var/lib/docker/overlay2/03eb172b20323ebeee7cb09025726a6e10de6c3dd8661f4f6bbc5a3a3778bad4/diff:/var/lib/docker/overlay2/80ea9bf34d028e95a8a418b3aeb634be72ab3336448946d7855c51ffdf8258d7/diff:/var/lib/docker/overlay2/e28987ea3fb4d431dee111f38cc76814b85f73e86434be3c463c01fdd59
9ad46/diff:/var/lib/docker/overlay2/6f9a46c7c6cf4fd30ce32a1eb08ed9b1e8dd61cb0e42ce54f0f736f06558c306/diff:/var/lib/docker/overlay2/6fc942eb9ccfc09898edc9a5ef3b3d305a6fcb22e8cefb1a9f9c344d10b59b25/diff:/var/lib/docker/overlay2/f6b8c57540e798dc1dfd51fd29c020193c02a4362f31c6d07aa220327bb07b90/diff:/var/lib/docker/overlay2/85d0aa01a3fab8c77262c59d8a7b0809a72673a3da7d33e4e19c04a66db44a11/diff:/var/lib/docker/overlay2/58dcd81a1605641f785ced5aeb38657eec1f02a31b7adfaf4f6fdd49f4bab1d6/diff:/var/lib/docker/overlay2/9d537e0a65325e4bf85b26da9828002339f5c442ca85f931b340457a80ec8c94/diff:/var/lib/docker/overlay2/a6f73707ee6a74e948a02d03c4a3f12b57c2eb408dc67da3c39961bb3cefcc97/diff:/var/lib/docker/overlay2/922526c310b066dbb73b9a106eef8bf0064ad7907a8d7d7b353f5307a17a4901/diff:/var/lib/docker/overlay2/a4185854870ebf8d34a4c7e1827dbe3bc461559839301ffb23c5efff5e3c9f14/diff:/var/lib/docker/overlay2/8c02b1fcb9d79f5d7b2ee4d44f6625a692c44a0d18df0a5c01cac0cca0fdf3f3/diff",
"MergedDir": "/var/lib/docker/overlay2/6a1120910baa8020db8ace8ab2c43ee409ff31973d68110156b9688f9732111b/merged",
"UpperDir": "/var/lib/docker/overlay2/6a1120910baa8020db8ace8ab2c43ee409ff31973d68110156b9688f9732111b/diff",
"WorkDir": "/var/lib/docker/overlay2/6a1120910baa8020db8ace8ab2c43ee409ff31973d68110156b9688f9732111b/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "kubernetes-upgrade-20220630205032-9950",
"Source": "/var/lib/docker/volumes/kubernetes-upgrade-20220630205032-9950/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "kubernetes-upgrade-20220630205032-9950",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "kubernetes-upgrade-20220630205032-9950",
"name.minikube.sigs.k8s.io": "kubernetes-upgrade-20220630205032-9950",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "a7af35f4f12d65dab8cbf2161ed70a15c30b0116559f0273fb3c50bd980e9897",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49344"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49343"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49340"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49342"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "49341"
}
]
},
"SandboxKey": "/var/run/docker/netns/a7af35f4f12d",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"kubernetes-upgrade-20220630205032-9950": {
"IPAMConfig": {
"IPv4Address": "192.168.67.2"
},
"Links": null,
"Aliases": [
"3933a5e88376",
"kubernetes-upgrade-20220630205032-9950"
],
"NetworkID": "fbd8d6c05e6ba8adf4329c183c9e1d40722efd453e5652cb543464956f975274",
"EndpointID": "11b1db4b4f0318851294fb2aea0254bef09975e72bf05d3ef3237f3dbf979d6b",
"Gateway": "192.168.67.1",
"IPAddress": "192.168.67.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:43:02",
"DriverOpts": null
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-amd64 status --format={{.Host}} -p kubernetes-upgrade-20220630205032-9950 -n kubernetes-upgrade-20220630205032-9950
=== CONT TestKubernetesUpgrade
helpers_test.go:239: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.Host}} -p kubernetes-upgrade-20220630205032-9950 -n kubernetes-upgrade-20220630205032-9950: exit status 2 (446.740367ms)
-- stdout --
Running
-- /stdout --
helpers_test.go:239: status error: exit status 2 (may be ok)
helpers_test.go:244: <<< TestKubernetesUpgrade FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestKubernetesUpgrade]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-amd64 -p kubernetes-upgrade-20220630205032-9950 logs -n 25
E0630 21:00:01.922787 9950 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/kindnet-20220630204858-9950/client.crt: no such file or directory
=== CONT TestKubernetesUpgrade
helpers_test.go:252: TestKubernetesUpgrade logs:
-- stdout --
*
* ==> Audit <==
* |---------|----------------------------------------|----------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|----------------------------------------|----------|---------|---------|---------------------|---------------------|
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:53 UTC |
| | running-upgrade-20220630205232-9950 | | | | | |
| | --memory=2200 --alsologtostderr | | | | | |
| | -v=1 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:53 UTC |
| | cert-expiration-20220630204940-9950 | | | | | |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:53 UTC |
| | cert-expiration-20220630204940-9950 | | | | | |
| start | -p auto-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:54 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:53 UTC |
| | running-upgrade-20220630205232-9950 | | | | | |
| start | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:54 UTC |
| | --alsologtostderr | | | | | |
| | -v=1 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| start | -p kindnet-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:53 UTC | 30 Jun 22 20:54 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=kindnet --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| pause | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | --alsologtostderr -v=5 | | | | | |
| unpause | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | --alsologtostderr -v=5 | | | | | |
| pause | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | --alsologtostderr -v=5 | | | | | |
| delete | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | --alsologtostderr -v=5 | | | | | |
| profile | list --output json | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| delete | -p pause-20220630205252-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| start | -p cilium-20220630204859-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:55 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=cilium --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p auto-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | pgrep -a kubelet | | | | | |
| ssh | -p kindnet-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| | pgrep -a kubelet | | | | | |
| delete | -p auto-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:54 UTC |
| start | -p calico-20220630204859-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=calico --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p kindnet-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:54 UTC | 30 Jun 22 20:55 UTC |
| start | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 20:55 UTC | 30 Jun 22 21:00 UTC |
| | enable-default-cni-20220630204858-9950 | | | | | |
| | --memory=2048 --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --enable-default-cni=true | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p cilium-20220630204859-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:55 UTC | 30 Jun 22 20:55 UTC |
| | pgrep -a kubelet | | | | | |
| delete | -p cilium-20220630204859-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:55 UTC | 30 Jun 22 20:55 UTC |
| start | -p bridge-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:55 UTC | 30 Jun 22 20:56 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --wait-timeout=5m | | | | | |
| | --cni=bridge --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | -p bridge-20220630204858-9950 | minikube | jenkins | v1.26.0 | 30 Jun 22 20:56 UTC | 30 Jun 22 20:56 UTC |
| | pgrep -a kubelet | | | | | |
| ssh | -p | minikube | jenkins | v1.26.0 | 30 Jun 22 21:00 UTC | 30 Jun 22 21:00 UTC |
| | enable-default-cni-20220630204858-9950 | | | | | |
| | pgrep -a kubelet | | | | | |
|---------|----------------------------------------|----------|---------|---------|---------------------|---------------------|
*
* ==> Last Start <==
* Log file created at: 2022/06/30 20:55:46
Running on machine: ubuntu-20-agent-8
Binary: Built with gc go1.18.3 for linux/amd64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0630 20:55:46.188135 212673 out.go:296] Setting OutFile to fd 1 ...
I0630 20:55:46.188261 212673 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:46.188271 212673 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:46.188278 212673 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:46.188778 212673 root.go:329] Updating PATH: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/bin
I0630 20:55:46.189121 212673 out.go:303] Setting JSON to false
I0630 20:55:46.190939 212673 start.go:115] hostinfo: {"hostname":"ubuntu-20-agent-8","uptime":2298,"bootTime":1656620248,"procs":693,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1012-gcp","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"guest","hostId":"591c9f12-2938-3743-e2bf-c56a050d43d1"}
I0630 20:55:46.191018 212673 start.go:125] virtualization: kvm guest
I0630 20:55:46.193977 212673 out.go:177] * [bridge-20220630204858-9950] minikube v1.26.0 on Ubuntu 20.04 (kvm/amd64)
I0630 20:55:46.195709 212673 out.go:177] - MINIKUBE_LOCATION=14420
I0630 20:55:46.195623 212673 notify.go:193] Checking for updates...
I0630 20:55:46.198832 212673 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0630 20:55:46.200393 212673 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 20:55:46.201847 212673 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube
I0630 20:55:46.203668 212673 out.go:177] - MINIKUBE_BIN=out/minikube-linux-amd64
I0630 20:55:46.205764 212673 config.go:178] Loaded profile config "calico-20220630204859-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:55:46.205913 212673 config.go:178] Loaded profile config "enable-default-cni-20220630204858-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:55:46.206064 212673 config.go:178] Loaded profile config "kubernetes-upgrade-20220630205032-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:55:46.206136 212673 driver.go:360] Setting default libvirt URI to qemu:///system
I0630 20:55:46.253958 212673 docker.go:137] docker version: linux-20.10.17
I0630 20:55:46.254102 212673 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 20:55:46.369508 212673 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:56 OomKillDisable:true NGoroutines:57 SystemTime:2022-06-30 20:55:46.290698743 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1012-gcp OperatingSystem:Ubuntu 20.04.4 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33662451712 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-8 Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.0] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 20:55:46.369632 212673 docker.go:254] overlay module found
I0630 20:55:46.371835 212673 out.go:177] * Using the docker driver based on user configuration
I0630 20:55:41.659430 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:42.159444 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:42.660167 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:43.159758 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:43.659304 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:44.159358 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:44.659952 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:45.159356 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:45.659758 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:46.159295 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:46.373158 212673 start.go:284] selected driver: docker
I0630 20:55:46.373183 212673 start.go:808] validating driver "docker" against <nil>
I0630 20:55:46.373204 212673 start.go:819] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0630 20:55:46.374071 212673 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0630 20:55:46.518248 212673 info.go:265] docker info: {ID:TS6T:UINC:MIYS:RZPA:KS6T:4JQK:7JHN:D6RA:LDP2:MHAE:G32M:C5NQ Containers:3 ContainersRunning:3 ContainersPaused:0 ContainersStopped:0 Images:8 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:56 OomKillDisable:true NGoroutines:57 SystemTime:2022-06-30 20:55:46.405285199 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1012-gcp OperatingSystem:Ubuntu 20.04.4 LTS OSType:linux Architecture:x86_64 IndexServ
erAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33662451712 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ubuntu-20-agent-8 Labels:[] ExperimentalBuild:false ServerVersion:20.10.17 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 Expected:10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1} RuncCommit:{ID:v1.1.2-0-ga916309 Expected:v1.1.2-0-ga916309} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:<nil> ServerErrors:[] Client
Info:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.8.2-docker] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.6.0] map[Name:scan Path:/usr/libexec/docker/cli-plugins/docker-scan SchemaVersion:0.1.0 ShortDescription:Docker Scan Vendor:Docker Inc. Version:v0.17.0]] Warnings:<nil>}}
I0630 20:55:46.518419 212673 start_flags.go:296] no existing cluster config was found, will generate one from the flags
I0630 20:55:46.518656 212673 start_flags.go:853] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0630 20:55:46.520549 212673 out.go:177] * Using Docker driver with root privileges
I0630 20:55:46.521915 212673 cni.go:95] Creating CNI manager for "bridge"
I0630 20:55:46.521937 212673 start_flags.go:305] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0630 20:55:46.521958 212673 start_flags.go:310] config:
{Name:bridge-20220630204858-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:bridge-20220630204858-9950 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 20:55:46.523711 212673 out.go:177] * Starting control plane node bridge-20220630204858-9950 in cluster bridge-20220630204858-9950
I0630 20:55:46.525174 212673 cache.go:120] Beginning downloading kic base image for docker with containerd
I0630 20:55:46.526653 212673 out.go:177] * Pulling base image ...
I0630 20:55:46.528127 212673 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime containerd
I0630 20:55:46.528154 212673 image.go:75] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon
I0630 20:55:46.528184 212673 preload.go:148] Found local preload: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4
I0630 20:55:46.528196 212673 cache.go:57] Caching tarball of preloaded images
I0630 20:55:46.528486 212673 preload.go:174] Found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4 in cache, skipping download
I0630 20:55:46.528514 212673 cache.go:60] Finished verifying existence of preloaded tar for v1.24.2 on containerd
I0630 20:55:46.528660 212673 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/config.json ...
I0630 20:55:46.528690 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/config.json: {Name:mk69b93e3215d75eba8b8292232181feb9f3fbbf Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:46.567450 212673 image.go:79] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e in local docker daemon, skipping pull
I0630 20:55:46.567476 212673 cache.go:142] gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e exists in daemon, skipping load
I0630 20:55:46.567491 212673 cache.go:208] Successfully downloaded all kic artifacts
I0630 20:55:46.567528 212673 start.go:352] acquiring machines lock for bridge-20220630204858-9950: {Name:mkc17e61ee90ac583c061a9d43e5074539c94220 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0630 20:55:46.567664 212673 start.go:356] acquired machines lock for "bridge-20220630204858-9950" in 112.941µs
I0630 20:55:46.567688 212673 start.go:91] Provisioning new machine with config: &{Name:bridge-20220630204858-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:bridge-20220630204858-9950 Namespace:default
APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker Bin
aryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:} &{Name: IP: Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0630 20:55:46.567766 212673 start.go:131] createHost starting for "" (driver="docker")
I0630 20:55:44.391595 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:44.782426 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:44.782503 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:44.815920 166730 cri.go:87] found id: ""
I0630 20:55:44.815947 166730 logs.go:274] 0 containers: []
W0630 20:55:44.815956 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:44.815966 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:44.816019 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:44.853352 166730 cri.go:87] found id: ""
I0630 20:55:44.853381 166730 logs.go:274] 0 containers: []
W0630 20:55:44.853390 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:44.853398 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:44.853451 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:44.883758 166730 cri.go:87] found id: ""
I0630 20:55:44.883788 166730 logs.go:274] 0 containers: []
W0630 20:55:44.883797 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:44.883808 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:44.883861 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:45.010738 166730 cri.go:87] found id: ""
I0630 20:55:45.010767 166730 logs.go:274] 0 containers: []
W0630 20:55:45.010774 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:45.010780 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:45.010834 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:45.046022 166730 cri.go:87] found id: ""
I0630 20:55:45.046052 166730 logs.go:274] 0 containers: []
W0630 20:55:45.046061 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:45.046069 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:45.046121 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:45.082049 166730 cri.go:87] found id: ""
I0630 20:55:45.082081 166730 logs.go:274] 0 containers: []
W0630 20:55:45.082089 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:45.082100 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:45.082155 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:45.111253 166730 cri.go:87] found id: ""
I0630 20:55:45.111278 166730 logs.go:274] 0 containers: []
W0630 20:55:45.111285 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:45.111291 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:45.111339 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:45.141541 166730 cri.go:87] found id: ""
I0630 20:55:45.141569 166730 logs.go:274] 0 containers: []
W0630 20:55:45.141579 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:45.141591 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:45.141607 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:45.162712 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:45.162748 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:45.229843 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:45.229876 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:45.229891 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:45.279559 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:45.279595 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:45.310726 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:45.310766 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:45.374121 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:44 kubernetes-upgrade-20220630205032-9950 kubelet[6758]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:45.449500 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:45.449538 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:45.449706 166730 out.go:239] X Problems detected in kubelet:
W0630 20:55:45.449724 166730 out.go:239] Jun 30 20:55:44 kubernetes-upgrade-20220630205032-9950 kubelet[6758]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:45.449730 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:45.449738 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:46.659846 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:47.159958 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:47.659877 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:48.159360 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:48.659328 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:49.160018 204258 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:55:49.219762 204258 kubeadm.go:1045] duration metric: took 12.158222474s to wait for elevateKubeSystemPrivileges.
I0630 20:55:49.219805 204258 kubeadm.go:397] StartCluster complete in 24.204884044s
I0630 20:55:49.219827 204258 settings.go:142] acquiring lock: {Name:mkcbd229caa05b60d3551ef0b47eea08424c9b5d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:49.219922 204258 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 20:55:49.221667 204258 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig: {Name:mk34e4a4e7b04f624b9adbda4f77ee20b38308ba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:49.738694 204258 kapi.go:244] deployment "coredns" in namespace "kube-system" and context "enable-default-cni-20220630204858-9950" rescaled to 1
I0630 20:55:49.738753 204258 start.go:208] Will wait 5m0s for node &{Name: IP:192.168.94.2 Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0630 20:55:49.740978 204258 out.go:177] * Verifying Kubernetes components...
I0630 20:55:49.738826 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0630 20:55:49.738836 204258 addons.go:412] enableAddons start: toEnable=map[], additional=[]
I0630 20:55:49.739009 204258 config.go:178] Loaded profile config "enable-default-cni-20220630204858-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:55:49.742489 204258 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:55:49.742536 204258 addons.go:65] Setting default-storageclass=true in profile "enable-default-cni-20220630204858-9950"
I0630 20:55:49.742552 204258 addons.go:65] Setting storage-provisioner=true in profile "enable-default-cni-20220630204858-9950"
I0630 20:55:49.742558 204258 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "enable-default-cni-20220630204858-9950"
I0630 20:55:49.742573 204258 addons.go:153] Setting addon storage-provisioner=true in "enable-default-cni-20220630204858-9950"
W0630 20:55:49.742586 204258 addons.go:162] addon storage-provisioner should already be in state true
I0630 20:55:49.742638 204258 host.go:66] Checking if "enable-default-cni-20220630204858-9950" exists ...
I0630 20:55:49.742945 204258 cli_runner.go:164] Run: docker container inspect enable-default-cni-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:49.743183 204258 cli_runner.go:164] Run: docker container inspect enable-default-cni-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:49.803577 204258 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:55:45.267856 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:47.768421 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:49.773482 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:46.570008 212673 out.go:204] * Creating docker container (CPUs=2, Memory=2048MB) ...
I0630 20:55:46.570204 212673 start.go:165] libmachine.API.Create for "bridge-20220630204858-9950" (driver="docker")
I0630 20:55:46.570237 212673 client.go:168] LocalClient.Create starting
I0630 20:55:46.570306 212673 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem
I0630 20:55:46.570334 212673 main.go:134] libmachine: Decoding PEM data...
I0630 20:55:46.570351 212673 main.go:134] libmachine: Parsing certificate...
I0630 20:55:46.570408 212673 main.go:134] libmachine: Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem
I0630 20:55:46.570428 212673 main.go:134] libmachine: Decoding PEM data...
I0630 20:55:46.570441 212673 main.go:134] libmachine: Parsing certificate...
I0630 20:55:46.570745 212673 cli_runner.go:164] Run: docker network inspect bridge-20220630204858-9950 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0630 20:55:46.605231 212673 cli_runner.go:211] docker network inspect bridge-20220630204858-9950 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0630 20:55:46.605294 212673 network_create.go:272] running [docker network inspect bridge-20220630204858-9950] to gather additional debugging logs...
I0630 20:55:46.605316 212673 cli_runner.go:164] Run: docker network inspect bridge-20220630204858-9950
W0630 20:55:46.641026 212673 cli_runner.go:211] docker network inspect bridge-20220630204858-9950 returned with exit code 1
I0630 20:55:46.641090 212673 network_create.go:275] error running [docker network inspect bridge-20220630204858-9950]: docker network inspect bridge-20220630204858-9950: exit status 1
stdout:
[]
stderr:
Error: No such network: bridge-20220630204858-9950
I0630 20:55:46.641125 212673 network_create.go:277] output of [docker network inspect bridge-20220630204858-9950]: -- stdout --
[]
-- /stdout --
** stderr **
Error: No such network: bridge-20220630204858-9950
** /stderr **
I0630 20:55:46.641179 212673 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0630 20:55:46.678059 212673 network.go:240] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 Interface:{IfaceName:br-c7ee26fec5a3 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:cc:d0:4a:9e}}
I0630 20:55:46.678831 212673 network.go:240] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 Interface:{IfaceName:br-3e97afadef02 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:9e:e7:fd:62}}
I0630 20:55:46.679666 212673 network.go:240] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 Interface:{IfaceName:br-fbd8d6c05e6b IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:6b:9f:58:b3}}
I0630 20:55:46.680958 212673 network.go:288] reserving subnet 192.168.76.0 for 1m0s: &{mu:{state:0 sema:0} read:{v:{m:map[] amended:true}} dirty:map[192.168.76.0:0xc00064c288] misses:0}
I0630 20:55:46.681008 212673 network.go:235] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:}}
I0630 20:55:46.681023 212673 network_create.go:115] attempt to create docker network bridge-20220630204858-9950 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0630 20:55:46.681100 212673 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=bridge-20220630204858-9950 bridge-20220630204858-9950
I0630 20:55:46.759260 212673 network_create.go:99] docker network bridge-20220630204858-9950 192.168.76.0/24 created
I0630 20:55:46.759290 212673 kic.go:106] calculated static IP "192.168.76.2" for the "bridge-20220630204858-9950" container
I0630 20:55:46.759340 212673 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0630 20:55:46.797407 212673 cli_runner.go:164] Run: docker volume create bridge-20220630204858-9950 --label name.minikube.sigs.k8s.io=bridge-20220630204858-9950 --label created_by.minikube.sigs.k8s.io=true
I0630 20:55:46.833736 212673 oci.go:103] Successfully created a docker volume bridge-20220630204858-9950
I0630 20:55:46.833825 212673 cli_runner.go:164] Run: docker run --rm --name bridge-20220630204858-9950-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=bridge-20220630204858-9950 --entrypoint /usr/bin/test -v bridge-20220630204858-9950:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -d /var/lib
I0630 20:55:47.536791 212673 oci.go:107] Successfully prepared a docker volume bridge-20220630204858-9950
I0630 20:55:47.536859 212673 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime containerd
I0630 20:55:47.536881 212673 kic.go:179] Starting extracting preloaded images to volume ...
I0630 20:55:47.536958 212673 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v bridge-20220630204858-9950:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -I lz4 -xf /preloaded.tar -C /extractDir
I0630 20:55:49.805221 204258 addons.go:345] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0630 20:55:49.805247 204258 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0630 20:55:49.805301 204258 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" enable-default-cni-20220630204858-9950
I0630 20:55:49.829524 204258 addons.go:153] Setting addon default-storageclass=true in "enable-default-cni-20220630204858-9950"
W0630 20:55:49.829557 204258 addons.go:162] addon default-storageclass should already be in state true
I0630 20:55:49.829595 204258 host.go:66] Checking if "enable-default-cni-20220630204858-9950" exists ...
I0630 20:55:49.830125 204258 cli_runner.go:164] Run: docker container inspect enable-default-cni-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:49.859824 204258 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49387 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/enable-default-cni-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:49.870020 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.94.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0630 20:55:49.870233 204258 node_ready.go:35] waiting up to 5m0s for node "enable-default-cni-20220630204858-9950" to be "Ready" ...
I0630 20:55:49.880492 204258 addons.go:345] installing /etc/kubernetes/addons/storageclass.yaml
I0630 20:55:49.880520 204258 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0630 20:55:49.880568 204258 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" enable-default-cni-20220630204858-9950
I0630 20:55:49.917841 204258 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49387 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/enable-default-cni-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:49.926188 204258 node_ready.go:49] node "enable-default-cni-20220630204858-9950" has status "Ready":"True"
I0630 20:55:49.926214 204258 node_ready.go:38] duration metric: took 55.942047ms waiting for node "enable-default-cni-20220630204858-9950" to be "Ready" ...
I0630 20:55:49.926225 204258 pod_ready.go:35] extra waiting up to 5m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0630 20:55:49.942701 204258 pod_ready.go:78] waiting up to 5m0s for pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace to be "Ready" ...
I0630 20:55:50.044298 204258 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0630 20:55:50.127996 204258 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0630 20:55:51.536821 204258 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.94.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (1.666758473s)
I0630 20:55:51.561712 204258 start.go:806] {"host.minikube.internal": 192.168.94.1} host record injected into CoreDNS
I0630 20:55:52.099073 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:52.108346 204258 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (2.064004464s)
I0630 20:55:52.108435 204258 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (1.980408563s)
I0630 20:55:52.252343 204258 out.go:177] * Enabled addons: storage-provisioner, default-storageclass
I0630 20:55:52.266814 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:54.267384 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:54.657023 212673 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.24.2-containerd-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v bridge-20220630204858-9950:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e -I lz4 -xf /preloaded.tar -C /extractDir: (7.119988103s)
I0630 20:55:54.657091 212673 kic.go:188] duration metric: took 7.120203 seconds to extract preloaded images to volume
W0630 20:55:54.657275 212673 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0630 20:55:54.657396 212673 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0630 20:55:54.778561 212673 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname bridge-20220630204858-9950 --name bridge-20220630204858-9950 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=bridge-20220630204858-9950 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=bridge-20220630204858-9950 --network bridge-20220630204858-9950 --ip 192.168.76.2 --volume bridge-20220630204858-9950:/var --security-opt apparmor=unconfined --memory=2048mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e
I0630 20:55:55.227194 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Running}}
I0630 20:55:55.271122 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:55.312288 212673 cli_runner.go:164] Run: docker exec bridge-20220630204858-9950 stat /var/lib/dpkg/alternatives/iptables
I0630 20:55:55.379124 212673 oci.go:144] the created container "bridge-20220630204858-9950" has a running status.
I0630 20:55:55.379157 212673 kic.go:210] Creating ssh key for kic: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa...
I0630 20:55:55.492394 212673 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0630 20:55:55.585540 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:55.627175 212673 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0630 20:55:55.627209 212673 kic_runner.go:114] Args: [docker exec --privileged bridge-20220630204858-9950 chown docker:docker /home/docker/.ssh/authorized_keys]
I0630 20:55:55.722547 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:55:55.760752 212673 machine.go:88] provisioning docker machine ...
I0630 20:55:55.760800 212673 ubuntu.go:169] provisioning hostname "bridge-20220630204858-9950"
I0630 20:55:55.760885 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:55.802550 212673 main.go:134] libmachine: Using SSH client type: native
I0630 20:55:55.802826 212673 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil> [] 0s} 127.0.0.1 49392 <nil> <nil>}
I0630 20:55:55.802860 212673 main.go:134] libmachine: About to run SSH command:
sudo hostname bridge-20220630204858-9950 && echo "bridge-20220630204858-9950" | sudo tee /etc/hostname
I0630 20:55:55.933414 212673 main.go:134] libmachine: SSH cmd err, output: <nil>: bridge-20220630204858-9950
I0630 20:55:55.933501 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:55.975720 212673 main.go:134] libmachine: Using SSH client type: native
I0630 20:55:55.975938 212673 main.go:134] libmachine: &{{{<nil> 0 [] [] []} docker [0x7dae00] 0x7dde60 <nil> [] 0s} 127.0.0.1 49392 <nil> <nil>}
I0630 20:55:55.975967 212673 main.go:134] libmachine: About to run SSH command:
if ! grep -xq '.*\sbridge-20220630204858-9950' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 bridge-20220630204858-9950/g' /etc/hosts;
else
echo '127.0.1.1 bridge-20220630204858-9950' | sudo tee -a /etc/hosts;
fi
fi
I0630 20:55:56.097075 212673 main.go:134] libmachine: SSH cmd err, output: <nil>:
I0630 20:55:56.097111 212673 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/c
erts/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube}
I0630 20:55:56.097138 212673 ubuntu.go:177] setting up certificates
I0630 20:55:56.097148 212673 provision.go:83] configureAuth start
I0630 20:55:56.097201 212673 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220630204858-9950
I0630 20:55:56.132663 212673 provision.go:138] copyHostCerts
I0630 20:55:56.132731 212673 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem, removing ...
I0630 20:55:56.132745 212673 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem
I0630 20:55:56.132827 212673 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/key.pem (1679 bytes)
I0630 20:55:56.132924 212673 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem, removing ...
I0630 20:55:56.132937 212673 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem
I0630 20:55:56.132984 212673 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.pem (1082 bytes)
I0630 20:55:56.133155 212673 exec_runner.go:144] found /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem, removing ...
I0630 20:55:56.133171 212673 exec_runner.go:207] rm: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem
I0630 20:55:56.133213 212673 exec_runner.go:151] cp: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/cert.pem (1123 bytes)
I0630 20:55:56.133280 212673 provision.go:112] generating server cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem org=jenkins.bridge-20220630204858-9950 san=[192.168.76.2 127.0.0.1 localhost 127.0.0.1 minikube bridge-20220630204858-9950]
I0630 20:55:52.424782 204258 addons.go:414] enableAddons completed in 2.685935701s
I0630 20:55:54.569634 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:55.450574 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:55:55.782242 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:55:55.782313 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:55:55.812323 166730 cri.go:87] found id: ""
I0630 20:55:55.812354 166730 logs.go:274] 0 containers: []
W0630 20:55:55.812363 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 20:55:55.812371 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:55:55.812425 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:55:55.838877 166730 cri.go:87] found id: ""
I0630 20:55:55.838905 166730 logs.go:274] 0 containers: []
W0630 20:55:55.838917 166730 logs.go:276] No container was found matching "etcd"
I0630 20:55:55.838925 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:55:55.838985 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:55:55.867225 166730 cri.go:87] found id: ""
I0630 20:55:55.867249 166730 logs.go:274] 0 containers: []
W0630 20:55:55.867273 166730 logs.go:276] No container was found matching "coredns"
I0630 20:55:55.867282 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:55:55.867338 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:55:55.893546 166730 cri.go:87] found id: ""
I0630 20:55:55.893578 166730 logs.go:274] 0 containers: []
W0630 20:55:55.893585 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 20:55:55.893591 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:55:55.893632 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:55:55.917280 166730 cri.go:87] found id: ""
I0630 20:55:55.917304 166730 logs.go:274] 0 containers: []
W0630 20:55:55.917313 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 20:55:55.917320 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:55:55.917378 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:55:55.942846 166730 cri.go:87] found id: ""
I0630 20:55:55.942873 166730 logs.go:274] 0 containers: []
W0630 20:55:55.942882 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:55:55.942890 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:55:55.942944 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:55:55.970944 166730 cri.go:87] found id: ""
I0630 20:55:55.970972 166730 logs.go:274] 0 containers: []
W0630 20:55:55.970981 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 20:55:55.970990 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:55:55.971044 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:55:55.995749 166730 cri.go:87] found id: ""
I0630 20:55:55.995773 166730 logs.go:274] 0 containers: []
W0630 20:55:55.995779 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 20:55:55.995787 166730 logs.go:123] Gathering logs for kubelet ...
I0630 20:55:55.995801 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 20:55:56.047220 166730 logs.go:138] Found kubelet problem: Jun 30 20:55:55 kubernetes-upgrade-20220630205032-9950 kubelet[7036]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:56.098827 166730 logs.go:123] Gathering logs for dmesg ...
I0630 20:55:56.098857 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:55:56.114004 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 20:55:56.114038 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 20:55:56.165221 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 20:55:56.165247 166730 logs.go:123] Gathering logs for containerd ...
I0630 20:55:56.165257 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:55:56.202035 166730 logs.go:123] Gathering logs for container status ...
I0630 20:55:56.202075 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:55:56.229905 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:56.229939 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
W0630 20:55:56.230076 166730 out.go:239] X Problems detected in kubelet:
W0630 20:55:56.230094 166730 out.go:239] Jun 30 20:55:55 kubernetes-upgrade-20220630205032-9950 kubelet[7036]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 20:55:56.230100 166730 out.go:309] Setting ErrFile to fd 2...
I0630 20:55:56.230107 166730 out.go:343] TERM=,COLORTERM=, which probably does not support color
I0630 20:55:56.206130 212673 provision.go:172] copyRemoteCerts
I0630 20:55:56.206183 212673 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0630 20:55:56.206213 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:56.242358 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:56.332675 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0630 20:55:56.352336 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server.pem --> /etc/docker/server.pem (1253 bytes)
I0630 20:55:56.372659 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0630 20:55:56.391569 212673 provision.go:86] duration metric: configureAuth took 294.40903ms
I0630 20:55:56.391598 212673 ubuntu.go:193] setting minikube options for container-runtime
I0630 20:55:56.391784 212673 config.go:178] Loaded profile config "bridge-20220630204858-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:55:56.391798 212673 machine.go:91] provisioned docker machine in 631.018103ms
I0630 20:55:56.391803 212673 client.go:171] LocalClient.Create took 9.821559055s
I0630 20:55:56.391831 212673 start.go:173] duration metric: libmachine.API.Create for "bridge-20220630204858-9950" took 9.821618626s
I0630 20:55:56.391845 212673 start.go:306] post-start starting for "bridge-20220630204858-9950" (driver="docker")
I0630 20:55:56.391856 212673 start.go:316] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0630 20:55:56.391910 212673 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0630 20:55:56.391959 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:56.427609 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:56.517207 212673 ssh_runner.go:195] Run: cat /etc/os-release
I0630 20:55:56.520436 212673 main.go:134] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0630 20:55:56.520469 212673 main.go:134] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0630 20:55:56.520486 212673 main.go:134] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0630 20:55:56.520493 212673 info.go:137] Remote host: Ubuntu 20.04.4 LTS
I0630 20:55:56.520506 212673 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/addons for local assets ...
I0630 20:55:56.520572 212673 filesync.go:126] Scanning /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files for local assets ...
I0630 20:55:56.520652 212673 filesync.go:149] local asset: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem -> 99502.pem in /etc/ssl/certs
I0630 20:55:56.520728 212673 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0630 20:55:56.529533 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem --> /etc/ssl/certs/99502.pem (1708 bytes)
I0630 20:55:56.549147 212673 start.go:309] post-start completed in 157.285272ms
I0630 20:55:56.549626 212673 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220630204858-9950
I0630 20:55:56.585571 212673 profile.go:148] Saving config to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/config.json ...
I0630 20:55:56.585820 212673 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0630 20:55:56.585860 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:56.622616 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:56.709706 212673 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0630 20:55:56.714181 212673 start.go:134] duration metric: createHost completed in 10.146405101s
I0630 20:55:56.714204 212673 start.go:81] releasing machines lock for "bridge-20220630204858-9950", held for 10.146527662s
I0630 20:55:56.714281 212673 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" bridge-20220630204858-9950
I0630 20:55:56.749842 212673 ssh_runner.go:195] Run: systemctl --version
I0630 20:55:56.749896 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:56.749962 212673 ssh_runner.go:195] Run: curl -sS -m 2 https://k8s.gcr.io/
I0630 20:55:56.750025 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:55:56.791635 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:56.792958 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:55:56.873413 212673 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0630 20:55:56.901712 212673 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0630 20:55:56.911179 212673 docker.go:179] disabling docker service ...
I0630 20:55:56.911231 212673 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0630 20:55:56.927760 212673 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0630 20:55:56.937114 212673 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0630 20:55:57.025376 212673 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0630 20:55:57.106230 212673 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0630 20:55:57.115609 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0630 20:55:57.128489 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*sandbox_image = .*$|sandbox_image = "k8s.gcr.io/pause:3.7"|' -i /etc/containerd/config.toml"
I0630 20:55:57.136528 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*restrict_oom_score_adj = .*$|restrict_oom_score_adj = false|' -i /etc/containerd/config.toml"
I0630 20:55:57.144580 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*SystemdCgroup = .*$|SystemdCgroup = false|' -i /etc/containerd/config.toml"
I0630 20:55:57.152450 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^.*conf_dir = .*$|conf_dir = "/etc/cni/net.d"|' -i /etc/containerd/config.toml"
I0630 20:55:57.160485 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo sed -e 's|^# imports|imports = ["/etc/containerd/containerd.conf.d/02-containerd.conf"]|' -i /etc/containerd/config.toml"
I0630 20:55:57.168217 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc/containerd/containerd.conf.d && printf %!s(MISSING) "dmVyc2lvbiA9IDIK" | base64 -d | sudo tee /etc/containerd/containerd.conf.d/02-containerd.conf"
I0630 20:55:57.181426 212673 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0630 20:55:57.187814 212673 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0630 20:55:57.194165 212673 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0630 20:55:57.270971 212673 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0630 20:55:57.351565 212673 start.go:447] Will wait 60s for socket path /run/containerd/containerd.sock
I0630 20:55:57.351635 212673 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0630 20:55:57.355115 212673 start.go:468] Will wait 60s for crictl version
I0630 20:55:57.355174 212673 ssh_runner.go:195] Run: sudo crictl version
I0630 20:55:57.382252 212673 start.go:477] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.6.6
RuntimeApiVersion: v1alpha2
I0630 20:55:57.382314 212673 ssh_runner.go:195] Run: containerd --version
I0630 20:55:57.411658 212673 ssh_runner.go:195] Run: containerd --version
I0630 20:55:57.441942 212673 out.go:177] * Preparing Kubernetes v1.24.2 on containerd 1.6.6 ...
I0630 20:55:57.443202 212673 cli_runner.go:164] Run: docker network inspect bridge-20220630204858-9950 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0630 20:55:57.478316 212673 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0630 20:55:57.481544 212673 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 20:55:57.490769 212673 preload.go:132] Checking if preload exists for k8s version v1.24.2 and runtime containerd
I0630 20:55:57.490829 212673 ssh_runner.go:195] Run: sudo crictl images --output json
I0630 20:55:57.514889 212673 containerd.go:547] all images are preloaded for containerd runtime.
I0630 20:55:57.514916 212673 containerd.go:461] Images already preloaded, skipping extraction
I0630 20:55:57.514965 212673 ssh_runner.go:195] Run: sudo crictl images --output json
I0630 20:55:57.541114 212673 containerd.go:547] all images are preloaded for containerd runtime.
I0630 20:55:57.541146 212673 cache_images.go:84] Images are preloaded, skipping loading
I0630 20:55:57.541216 212673 ssh_runner.go:195] Run: sudo crictl info
I0630 20:55:57.565945 212673 cni.go:95] Creating CNI manager for "bridge"
I0630 20:55:57.565975 212673 kubeadm.go:87] Using pod CIDR: 10.244.0.0/16
I0630 20:55:57.565992 212673 kubeadm.go:158] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.24.2 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:bridge-20220630204858-9950 NodeName:bridge-20220630204858-9950 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NoTaintMaster:true NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var
/lib/minikube/certs/ca.crt StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[]}
I0630 20:55:57.566146 212673 kubeadm.go:162] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "bridge-20220630204858-9950"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.24.2
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0630 20:55:57.566252 212673 kubeadm.go:961] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.24.2/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=bridge-20220630204858-9950 --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2 --runtime-request-timeout=15m
[Install]
config:
{KubernetesVersion:v1.24.2 ClusterName:bridge-20220630204858-9950 Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:}
I0630 20:55:57.566309 212673 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.24.2
I0630 20:55:57.573514 212673 binaries.go:44] Found k8s binaries, skipping transfer
I0630 20:55:57.573581 212673 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0630 20:55:57.580676 212673 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (519 bytes)
I0630 20:55:57.593558 212673 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0630 20:55:57.606351 212673 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2055 bytes)
I0630 20:55:57.619170 212673 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0630 20:55:57.622326 212673 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0630 20:55:57.631440 212673 certs.go:54] Setting up /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950 for IP: 192.168.76.2
I0630 20:55:57.631536 212673 certs.go:182] skipping minikubeCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.key
I0630 20:55:57.631601 212673 certs.go:182] skipping proxyClientCA CA generation: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.key
I0630 20:55:57.631667 212673 certs.go:302] generating minikube-user signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.key
I0630 20:55:57.631685 212673 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.crt with IP's: []
I0630 20:55:57.739977 212673 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.crt ...
I0630 20:55:57.740010 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.crt: {Name:mke6e0277c25124b31dbfd55a9095ff296fb5e57 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.740220 212673 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.key ...
I0630 20:55:57.740233 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/client.key: {Name:mk31207fe37dadc8e8fb0be86159a561f753729a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.740339 212673 certs.go:302] generating minikube signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key.31bdca25
I0630 20:55:57.740356 212673 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt.31bdca25 with IP's: [192.168.76.2 10.96.0.1 127.0.0.1 10.0.0.1]
I0630 20:55:57.847242 212673 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt.31bdca25 ...
I0630 20:55:57.847271 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt.31bdca25: {Name:mkef3ab2d396bd7bc4a1e6e3eac66a07094e9f66 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.847453 212673 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key.31bdca25 ...
I0630 20:55:57.847471 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key.31bdca25: {Name:mk65a591a79b2ab802426a2267969384992b3f19 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.847555 212673 certs.go:320] copying /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt
I0630 20:55:57.847612 212673 certs.go:324] copying /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key.31bdca25 -> /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key
I0630 20:55:57.847657 212673 certs.go:302] generating aggregator signed cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.key
I0630 20:55:57.847671 212673 crypto.go:68] Generating cert /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.crt with IP's: []
I0630 20:55:57.991662 212673 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.crt ...
I0630 20:55:57.991690 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.crt: {Name:mk9bf7d4f80b03254ffe765ccc5814378ccf23dd Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.991865 212673 crypto.go:164] Writing key to /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.key ...
I0630 20:55:57.991879 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.key: {Name:mk3925cbd66d11df655124f6a1b9bb448c367f33 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:55:57.992046 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950.pem (1338 bytes)
W0630 20:55:57.992082 212673 certs.go:384] ignoring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950_empty.pem, impossibly tiny 0 bytes
I0630 20:55:57.992096 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca-key.pem (1679 bytes)
I0630 20:55:57.992122 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/ca.pem (1082 bytes)
I0630 20:55:57.992145 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/cert.pem (1123 bytes)
I0630 20:55:57.992168 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/key.pem (1679 bytes)
I0630 20:55:57.992204 212673 certs.go:388] found cert: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem (1708 bytes)
I0630 20:55:57.992739 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1399 bytes)
I0630 20:55:58.011864 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0630 20:55:58.029305 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0630 20:55:58.046575 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/profiles/bridge-20220630204858-9950/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0630 20:55:58.064286 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0630 20:55:58.082393 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0630 20:55:58.099428 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0630 20:55:58.116136 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0630 20:55:58.132910 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0630 20:55:58.150213 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/certs/9950.pem --> /usr/share/ca-certificates/9950.pem (1338 bytes)
I0630 20:55:58.167331 212673 ssh_runner.go:362] scp /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/files/etc/ssl/certs/99502.pem --> /usr/share/ca-certificates/99502.pem (1708 bytes)
I0630 20:55:58.184814 212673 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0630 20:55:58.197448 212673 ssh_runner.go:195] Run: openssl version
I0630 20:55:58.202381 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0630 20:55:58.209813 212673 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0630 20:55:58.212769 212673 certs.go:431] hashing: -rw-r--r-- 1 root root 1111 Jun 30 20:22 /usr/share/ca-certificates/minikubeCA.pem
I0630 20:55:58.212814 212673 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0630 20:55:58.217544 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0630 20:55:58.224605 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/9950.pem && ln -fs /usr/share/ca-certificates/9950.pem /etc/ssl/certs/9950.pem"
I0630 20:55:58.231576 212673 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/9950.pem
I0630 20:55:58.234676 212673 certs.go:431] hashing: -rw-r--r-- 1 root root 1338 Jun 30 20:26 /usr/share/ca-certificates/9950.pem
I0630 20:55:58.234730 212673 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/9950.pem
I0630 20:55:58.239471 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/9950.pem /etc/ssl/certs/51391683.0"
I0630 20:55:58.246703 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/99502.pem && ln -fs /usr/share/ca-certificates/99502.pem /etc/ssl/certs/99502.pem"
I0630 20:55:58.254058 212673 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/99502.pem
I0630 20:55:58.257570 212673 certs.go:431] hashing: -rw-r--r-- 1 root root 1708 Jun 30 20:26 /usr/share/ca-certificates/99502.pem
I0630 20:55:58.257619 212673 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/99502.pem
I0630 20:55:58.263132 212673 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/99502.pem /etc/ssl/certs/3ec20f2e.0"
I0630 20:55:58.271162 212673 kubeadm.go:395] StartCluster: {Name:bridge-20220630204858-9950 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.32-1656350719-14420@sha256:e7b7f38d1a2eba7828afc2c4c3d24e1d391db431976e47aa6dc5c7a6b038ca4e Memory:2048 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:0 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.24.2 ClusterName:bridge-20220630204858-9950 Namespace:default APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:bridge NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:5m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror:
DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath:}
I0630 20:55:58.271274 212673 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0630 20:55:58.271330 212673 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0630 20:55:58.298446 212673 cri.go:87] found id: ""
I0630 20:55:58.298512 212673 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0630 20:55:58.305833 212673 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0630 20:55:58.312649 212673 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 20:55:58.312715 212673 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:55:58.319391 212673 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 20:55:58.319439 212673 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 20:55:56.267709 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:58.767508 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:58.577922 212673 out.go:204] - Generating certificates and keys ...
I0630 20:55:56.954225 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:55:58.954495 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:00.955091 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:00.769496 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:03.267547 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:01.441329 212673 out.go:204] - Booting up control plane ...
I0630 20:56:02.956084 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:05.454102 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:06.232825 166730 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:56:06.243827 166730 kubeadm.go:630] restartCluster took 4m4.449118713s
W0630 20:56:06.243995 166730 out.go:239] ! Unable to restart cluster, will reset it: apiserver healthz: apiserver process never appeared
I0630 20:56:06.244034 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0630 20:56:06.946910 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:56:06.958177 166730 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0630 20:56:06.967186 166730 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 20:56:06.967255 166730 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:56:06.974506 166730 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 20:56:06.974558 166730 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 20:56:07.227007 166730 out.go:204] - Generating certificates and keys ...
I0630 20:56:09.494725 212673 out.go:204] - Configuring RBAC rules ...
I0630 20:56:09.933498 212673 cni.go:95] Creating CNI manager for "bridge"
I0630 20:56:09.935412 212673 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...
I0630 20:56:05.267982 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:07.767572 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:09.936890 212673 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d
I0630 20:56:09.947228 212673 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (457 bytes)
I0630 20:56:10.034620 212673 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0630 20:56:10.034684 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:10.034705 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl label nodes minikube.k8s.io/version=v1.26.0 minikube.k8s.io/commit=251ffcec0e4c997cd7732864187747cc579943c9 minikube.k8s.io/name=bridge-20220630204858-9950 minikube.k8s.io/updated_at=2022_06_30T20_56_10_0700 minikube.k8s.io/primary=true --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:10.042633 212673 ops.go:34] apiserver oom_adj: -16
I0630 20:56:10.327568 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:10.887764 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:07.455661 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:09.955902 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:08.883897 166730 out.go:204] - Booting up control plane ...
I0630 20:56:10.266599 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:12.267197 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:14.267220 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:11.387685 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:11.887360 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:12.387431 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:12.887406 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:13.388302 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:13.888115 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:14.388161 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:14.888060 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:15.387604 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:15.887424 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:12.453867 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:14.454223 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:16.454995 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:16.766493 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:18.766918 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:16.387813 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:16.888055 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:17.388251 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:17.888018 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:18.387545 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:18.887990 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:19.387909 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:19.887750 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:20.387800 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:20.887436 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:18.954675 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:21.454198 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:21.388166 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:21.887360 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:22.388184 212673 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.24.2/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0630 20:56:22.486209 212673 kubeadm.go:1045] duration metric: took 12.451585026s to wait for elevateKubeSystemPrivileges.
I0630 20:56:22.486245 212673 kubeadm.go:397] StartCluster complete in 24.215089821s
I0630 20:56:22.486264 212673 settings.go:142] acquiring lock: {Name:mkcbd229caa05b60d3551ef0b47eea08424c9b5d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:56:22.486381 212673 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig
I0630 20:56:22.487939 212673 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/kubeconfig: {Name:mk34e4a4e7b04f624b9adbda4f77ee20b38308ba Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0630 20:56:23.005043 212673 kapi.go:244] deployment "coredns" in namespace "kube-system" and context "bridge-20220630204858-9950" rescaled to 1
I0630 20:56:23.005165 212673 start.go:208] Will wait 5m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.24.2 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0630 20:56:23.005196 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml"
I0630 20:56:23.007687 212673 out.go:177] * Verifying Kubernetes components...
I0630 20:56:23.005229 212673 addons.go:412] enableAddons start: toEnable=map[], additional=[]
I0630 20:56:23.005422 212673 config.go:178] Loaded profile config "bridge-20220630204858-9950": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.24.2
I0630 20:56:23.009432 212673 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:56:23.009456 212673 addons.go:65] Setting default-storageclass=true in profile "bridge-20220630204858-9950"
I0630 20:56:23.009458 212673 addons.go:65] Setting storage-provisioner=true in profile "bridge-20220630204858-9950"
I0630 20:56:23.009476 212673 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "bridge-20220630204858-9950"
I0630 20:56:23.009484 212673 addons.go:153] Setting addon storage-provisioner=true in "bridge-20220630204858-9950"
W0630 20:56:23.009493 212673 addons.go:162] addon storage-provisioner should already be in state true
I0630 20:56:23.009556 212673 host.go:66] Checking if "bridge-20220630204858-9950" exists ...
I0630 20:56:23.009877 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:56:23.010079 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:56:23.070413 212673 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0630 20:56:23.071605 212673 addons.go:153] Setting addon default-storageclass=true in "bridge-20220630204858-9950"
W0630 20:56:23.073520 212673 addons.go:162] addon default-storageclass should already be in state true
I0630 20:56:23.073545 212673 addons.go:345] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0630 20:56:23.073564 212673 host.go:66] Checking if "bridge-20220630204858-9950" exists ...
I0630 20:56:23.073568 212673 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0630 20:56:23.073631 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:56:23.074017 212673 cli_runner.go:164] Run: docker container inspect bridge-20220630204858-9950 --format={{.State.Status}}
I0630 20:56:23.115501 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:56:23.120591 212673 addons.go:345] installing /etc/kubernetes/addons/storageclass.yaml
I0630 20:56:23.120617 212673 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0630 20:56:23.120683 212673 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" bridge-20220630204858-9950
I0630 20:56:23.124656 212673 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.76.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -"
I0630 20:56:23.126278 212673 node_ready.go:35] waiting up to 5m0s for node "bridge-20220630204858-9950" to be "Ready" ...
I0630 20:56:23.141976 212673 node_ready.go:49] node "bridge-20220630204858-9950" has status "Ready":"True"
I0630 20:56:23.142001 212673 node_ready.go:38] duration metric: took 15.686277ms waiting for node "bridge-20220630204858-9950" to be "Ready" ...
I0630 20:56:23.142012 212673 pod_ready.go:35] extra waiting up to 5m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0630 20:56:23.177884 212673 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:49392 SSHKeyPath:/home/jenkins/minikube-integration/linux-amd64-docker-containerd-14420-3397-b28e6de4ba37ffec66170588afce60c61d1bcd44/.minikube/machines/bridge-20220630204858-9950/id_rsa Username:docker}
I0630 20:56:23.231559 212673 pod_ready.go:78] waiting up to 5m0s for pod "coredns-6d4b75cb6d-2d7vc" in "kube-system" namespace to be "Ready" ...
I0630 20:56:23.343141 212673 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0630 20:56:23.438387 212673 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0630 20:56:24.524626 212673 ssh_runner.go:235] Completed: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig -n kube-system get configmap coredns -o yaml | sed '/^ forward . \/etc\/resolv.conf.*/i \ hosts {\n 192.168.76.1 host.minikube.internal\n fallthrough\n }' | sudo /var/lib/minikube/binaries/v1.24.2/kubectl --kubeconfig=/var/lib/minikube/kubeconfig replace -f -": (1.399924554s)
I0630 20:56:24.524663 212673 start.go:806] {"host.minikube.internal": 192.168.76.1} host record injected into CoreDNS
I0630 20:56:24.575448 212673 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (1.232252027s)
I0630 20:56:24.575513 212673 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.24.2/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (1.137086474s)
I0630 20:56:24.578504 212673 out.go:177] * Enabled addons: storage-provisioner, default-storageclass
I0630 20:56:21.266248 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:23.267196 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:24.579993 212673 addons.go:414] enableAddons completed in 1.574773753s
I0630 20:56:25.244470 212673 pod_ready.go:92] pod "coredns-6d4b75cb6d-2d7vc" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.244521 212673 pod_ready.go:81] duration metric: took 2.012924285s waiting for pod "coredns-6d4b75cb6d-2d7vc" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.244535 212673 pod_ready.go:78] waiting up to 5m0s for pod "etcd-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.252574 212673 pod_ready.go:92] pod "etcd-bridge-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.252599 212673 pod_ready.go:81] duration metric: took 8.055921ms waiting for pod "etcd-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.252611 212673 pod_ready.go:78] waiting up to 5m0s for pod "kube-apiserver-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.258053 212673 pod_ready.go:92] pod "kube-apiserver-bridge-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.258076 212673 pod_ready.go:81] duration metric: took 5.457376ms waiting for pod "kube-apiserver-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.258090 212673 pod_ready.go:78] waiting up to 5m0s for pod "kube-controller-manager-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.263133 212673 pod_ready.go:92] pod "kube-controller-manager-bridge-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.263162 212673 pod_ready.go:81] duration metric: took 5.064416ms waiting for pod "kube-controller-manager-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.263174 212673 pod_ready.go:78] waiting up to 5m0s for pod "kube-proxy-vqj7p" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.270707 212673 pod_ready.go:92] pod "kube-proxy-vqj7p" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.270735 212673 pod_ready.go:81] duration metric: took 7.553393ms waiting for pod "kube-proxy-vqj7p" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.270747 212673 pod_ready.go:78] waiting up to 5m0s for pod "kube-scheduler-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.640789 212673 pod_ready.go:92] pod "kube-scheduler-bridge-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:56:25.640810 212673 pod_ready.go:81] duration metric: took 370.055923ms waiting for pod "kube-scheduler-bridge-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:56:25.640818 212673 pod_ready.go:38] duration metric: took 2.498795221s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0630 20:56:25.640843 212673 api_server.go:51] waiting for apiserver process to appear ...
I0630 20:56:25.640884 212673 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:56:25.650841 212673 api_server.go:71] duration metric: took 2.645641539s to wait for apiserver process to appear ...
I0630 20:56:25.650866 212673 api_server.go:87] waiting for apiserver healthz status ...
I0630 20:56:25.650877 212673 api_server.go:240] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0630 20:56:25.655263 212673 api_server.go:266] https://192.168.76.2:8443/healthz returned 200:
ok
I0630 20:56:25.656214 212673 api_server.go:140] control plane version: v1.24.2
I0630 20:56:25.656242 212673 api_server.go:130] duration metric: took 5.370601ms to wait for apiserver health ...
I0630 20:56:25.656253 212673 system_pods.go:43] waiting for kube-system pods to appear ...
I0630 20:56:25.843857 212673 system_pods.go:59] 7 kube-system pods found
I0630 20:56:25.843891 212673 system_pods.go:61] "coredns-6d4b75cb6d-2d7vc" [9e6f4b13-cb93-4ebf-8d54-9a03450f7193] Running
I0630 20:56:25.843899 212673 system_pods.go:61] "etcd-bridge-20220630204858-9950" [a07a2b3e-d9cd-421e-ab62-27c50daa13ac] Running
I0630 20:56:25.843905 212673 system_pods.go:61] "kube-apiserver-bridge-20220630204858-9950" [a8c62683-6ce1-4cce-b639-9121e0366a9d] Running
I0630 20:56:25.843916 212673 system_pods.go:61] "kube-controller-manager-bridge-20220630204858-9950" [b15660ff-05cc-4505-a7f8-cfcb0a48f952] Running
I0630 20:56:25.843923 212673 system_pods.go:61] "kube-proxy-vqj7p" [3fad7c40-a7cd-43af-9a92-57efd325e52b] Running
I0630 20:56:25.843930 212673 system_pods.go:61] "kube-scheduler-bridge-20220630204858-9950" [29b8124f-854b-4349-9c98-782d63a08799] Running
I0630 20:56:25.843946 212673 system_pods.go:61] "storage-provisioner" [5eb17b16-d2d1-4aa3-bc28-4633f96dbdb5] Pending / Ready:ContainersNotReady (containers with unready status: [storage-provisioner]) / ContainersReady:ContainersNotReady (containers with unready status: [storage-provisioner])
I0630 20:56:25.843960 212673 system_pods.go:74] duration metric: took 187.698828ms to wait for pod list to return data ...
I0630 20:56:25.843972 212673 default_sa.go:34] waiting for default service account to be created ...
I0630 20:56:26.041197 212673 default_sa.go:45] found service account: "default"
I0630 20:56:26.041220 212673 default_sa.go:55] duration metric: took 197.238332ms for default service account to be created ...
I0630 20:56:26.041227 212673 system_pods.go:116] waiting for k8s-apps to be running ...
I0630 20:56:26.244687 212673 system_pods.go:86] 7 kube-system pods found
I0630 20:56:26.244731 212673 system_pods.go:89] "coredns-6d4b75cb6d-2d7vc" [9e6f4b13-cb93-4ebf-8d54-9a03450f7193] Running
I0630 20:56:26.244741 212673 system_pods.go:89] "etcd-bridge-20220630204858-9950" [a07a2b3e-d9cd-421e-ab62-27c50daa13ac] Running
I0630 20:56:26.244749 212673 system_pods.go:89] "kube-apiserver-bridge-20220630204858-9950" [a8c62683-6ce1-4cce-b639-9121e0366a9d] Running
I0630 20:56:26.244756 212673 system_pods.go:89] "kube-controller-manager-bridge-20220630204858-9950" [b15660ff-05cc-4505-a7f8-cfcb0a48f952] Running
I0630 20:56:26.244773 212673 system_pods.go:89] "kube-proxy-vqj7p" [3fad7c40-a7cd-43af-9a92-57efd325e52b] Running
I0630 20:56:26.244782 212673 system_pods.go:89] "kube-scheduler-bridge-20220630204858-9950" [29b8124f-854b-4349-9c98-782d63a08799] Running
I0630 20:56:26.244793 212673 system_pods.go:89] "storage-provisioner" [5eb17b16-d2d1-4aa3-bc28-4633f96dbdb5] Running
I0630 20:56:26.244806 212673 system_pods.go:126] duration metric: took 203.573154ms to wait for k8s-apps to be running ...
I0630 20:56:26.244818 212673 system_svc.go:44] waiting for kubelet service to be running ....
I0630 20:56:26.244875 212673 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:56:26.255878 212673 system_svc.go:56] duration metric: took 11.051437ms WaitForService to wait for kubelet.
I0630 20:56:26.255913 212673 kubeadm.go:572] duration metric: took 3.250714017s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...
I0630 20:56:26.255940 212673 node_conditions.go:102] verifying NodePressure condition ...
I0630 20:56:26.440917 212673 node_conditions.go:122] node storage ephemeral capacity is 304695084Ki
I0630 20:56:26.440943 212673 node_conditions.go:123] node cpu capacity is 8
I0630 20:56:26.440956 212673 node_conditions.go:105] duration metric: took 185.006001ms to run NodePressure ...
I0630 20:56:26.440966 212673 start.go:213] waiting for startup goroutines ...
I0630 20:56:26.479593 212673 start.go:506] kubectl: 1.24.2, cluster: 1.24.2 (minor skew: 0)
I0630 20:56:26.481941 212673 out.go:177] * Done! kubectl is now configured to use "bridge-20220630204858-9950" cluster and "default" namespace by default
I0630 20:56:23.456321 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:25.954519 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:25.270864 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:27.767716 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:29.768071 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:27.955153 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:29.955539 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:32.266879 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:34.267104 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:32.454358 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:34.454649 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:36.767348 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:39.267374 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:36.954920 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:39.454330 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:41.454501 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:41.267972 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:43.766977 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:43.455153 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:45.955314 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:45.767596 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:48.267529 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:48.454474 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:50.953742 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:50.268708 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:52.770822 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:52.954704 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:55.454327 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:55.266690 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:57.266812 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:59.766755 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:57.954245 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:56:59.954492 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:01.767175 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:03.767340 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:01.955202 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:03.955295 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:06.454135 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:06.267243 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:08.267493 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:08.455283 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:10.953635 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:10.267604 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:12.767831 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:12.954813 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:15.454243 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:15.267908 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:17.767619 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:17.454899 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:19.954623 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:20.268189 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:22.766829 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:24.769945 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:22.454121 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:24.454617 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:26.454994 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:27.267444 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:29.268045 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:28.954124 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:31.454576 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:31.767298 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:34.266266 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:33.454803 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:35.954706 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:36.266331 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:38.267361 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:38.453670 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:40.454674 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:40.767647 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:43.266649 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:42.955644 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:45.454193 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:45.767285 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:48.266303 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:47.954761 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:50.454790 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:50.266748 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:52.766570 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:54.770823 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:52.454914 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:54.954860 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:57.267030 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:59.267416 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:57.455058 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:57:59.954848 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:01.767836 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:04.266488 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:01.956047 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:04.455232 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
W0630 20:58:03.899565 166730 out.go:239] ! initialization failed, will try again: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:56:07.008038 7586 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I0630 20:58:03.899635 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm reset --cri-socket /run/containerd/containerd.sock --force"
I0630 20:58:04.591252 166730 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 20:58:04.601287 166730 kubeadm.go:221] ignoring SystemVerification for kubeadm because of docker driver
I0630 20:58:04.601351 166730 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0630 20:58:04.608513 166730 kubeadm.go:152] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0630 20:58:04.608564 166730 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0630 20:58:06.267464 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:08.766678 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:06.953923 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:09.454146 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:11.454724 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:10.766750 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:12.767407 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:14.767560 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:13.954829 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:16.454782 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:17.266846 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:19.767336 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:18.953606 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:20.954459 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:21.767511 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:24.267252 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:23.453769 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:25.954614 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:26.267525 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:28.767713 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:27.955380 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:30.453844 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:31.267674 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:33.766748 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:32.953384 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:34.954880 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:35.767244 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:37.767528 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:37.454850 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:39.455008 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:40.267428 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:42.766176 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:44.767181 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:41.954471 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:44.454092 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:46.767809 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:49.266904 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:46.954306 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:48.954479 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:51.454417 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:51.268184 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:53.766719 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:53.954849 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:55.954979 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:55.768768 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:58.266495 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:58:57.955189 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:00.454315 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:00.267616 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:02.767801 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:04.767965 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:02.955090 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:05.453946 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:07.267048 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:09.267596 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:07.454977 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:09.955336 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:11.766973 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:13.767471 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:12.453951 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:14.454847 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:16.455526 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:16.267132 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:18.766378 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:18.953607 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:20.954733 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:20.768732 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:23.266667 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:23.454571 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:25.954662 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:25.266948 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:27.267421 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:29.767806 201358 pod_ready.go:102] pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:27.954819 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:30.454784 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:30.272033 201358 pod_ready.go:81] duration metric: took 4m0.022637966s waiting for pod "calico-kube-controllers-c44b4545-jtql7" in "kube-system" namespace to be "Ready" ...
E0630 20:59:30.272057 201358 pod_ready.go:66] WaitExtra: waitPodCondition: timed out waiting for the condition
I0630 20:59:30.272067 201358 pod_ready.go:78] waiting up to 5m0s for pod "calico-node-wkbg5" in "kube-system" namespace to be "Ready" ...
I0630 20:59:32.284194 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:34.284965 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:32.954169 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:34.954426 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:36.784237 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:38.784754 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:37.455154 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:39.954504 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:41.283852 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:43.284466 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:41.956399 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:44.454331 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:45.284750 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:47.287832 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:49.784604 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:46.953690 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:48.954131 204258 pod_ready.go:102] pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:49.958782 204258 pod_ready.go:81] duration metric: took 4m0.01604374s waiting for pod "coredns-6d4b75cb6d-5577k" in "kube-system" namespace to be "Ready" ...
E0630 20:59:49.958804 204258 pod_ready.go:66] WaitExtra: waitPodCondition: timed out waiting for the condition
I0630 20:59:49.958813 204258 pod_ready.go:78] waiting up to 5m0s for pod "coredns-6d4b75cb6d-z5jnw" in "kube-system" namespace to be "Ready" ...
I0630 20:59:49.960490 204258 pod_ready.go:97] error getting pod "coredns-6d4b75cb6d-z5jnw" in "kube-system" namespace (skipping!): pods "coredns-6d4b75cb6d-z5jnw" not found
I0630 20:59:49.960508 204258 pod_ready.go:81] duration metric: took 1.689263ms waiting for pod "coredns-6d4b75cb6d-z5jnw" in "kube-system" namespace to be "Ready" ...
E0630 20:59:49.960515 204258 pod_ready.go:66] WaitExtra: waitPodCondition: error getting pod "coredns-6d4b75cb6d-z5jnw" in "kube-system" namespace (skipping!): pods "coredns-6d4b75cb6d-z5jnw" not found
I0630 20:59:49.960523 204258 pod_ready.go:78] waiting up to 5m0s for pod "etcd-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:49.964683 204258 pod_ready.go:92] pod "etcd-enable-default-cni-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:59:49.964702 204258 pod_ready.go:81] duration metric: took 4.173488ms waiting for pod "etcd-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:49.964711 204258 pod_ready.go:78] waiting up to 5m0s for pod "kube-apiserver-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:49.968405 204258 pod_ready.go:92] pod "kube-apiserver-enable-default-cni-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:59:49.968420 204258 pod_ready.go:81] duration metric: took 3.703287ms waiting for pod "kube-apiserver-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:49.968428 204258 pod_ready.go:78] waiting up to 5m0s for pod "kube-controller-manager-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.152903 204258 pod_ready.go:92] pod "kube-controller-manager-enable-default-cni-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:59:50.152932 204258 pod_ready.go:81] duration metric: took 184.49678ms waiting for pod "kube-controller-manager-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.152949 204258 pod_ready.go:78] waiting up to 5m0s for pod "kube-proxy-98pfx" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.555523 204258 pod_ready.go:92] pod "kube-proxy-98pfx" in "kube-system" namespace has status "Ready":"True"
I0630 20:59:50.555547 204258 pod_ready.go:81] duration metric: took 402.591041ms waiting for pod "kube-proxy-98pfx" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.555559 204258 pod_ready.go:78] waiting up to 5m0s for pod "kube-scheduler-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.952877 204258 pod_ready.go:92] pod "kube-scheduler-enable-default-cni-20220630204858-9950" in "kube-system" namespace has status "Ready":"True"
I0630 20:59:50.952901 204258 pod_ready.go:81] duration metric: took 397.336151ms waiting for pod "kube-scheduler-enable-default-cni-20220630204858-9950" in "kube-system" namespace to be "Ready" ...
I0630 20:59:50.952910 204258 pod_ready.go:38] duration metric: took 4m1.026664026s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0630 20:59:50.952931 204258 api_server.go:51] waiting for apiserver process to appear ...
I0630 20:59:50.952947 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:59:50.952994 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:59:50.979317 204258 cri.go:87] found id: "62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:50.979351 204258 cri.go:87] found id: ""
I0630 20:59:50.979361 204258 logs.go:274] 1 containers: [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc]
I0630 20:59:50.979408 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:50.982704 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:59:50.982779 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:59:51.007904 204258 cri.go:87] found id: "43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:51.007937 204258 cri.go:87] found id: ""
I0630 20:59:51.007945 204258 logs.go:274] 1 containers: [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a]
I0630 20:59:51.007995 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.011506 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:59:51.011584 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:59:51.037245 204258 cri.go:87] found id: "f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:51.037269 204258 cri.go:87] found id: ""
I0630 20:59:51.037275 204258 logs.go:274] 1 containers: [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38]
I0630 20:59:51.037316 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.040485 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:59:51.040550 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:59:51.065417 204258 cri.go:87] found id: "55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:51.065456 204258 cri.go:87] found id: ""
I0630 20:59:51.065466 204258 logs.go:274] 1 containers: [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7]
I0630 20:59:51.065529 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.068686 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:59:51.068755 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:59:51.093012 204258 cri.go:87] found id: "8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:51.093086 204258 cri.go:87] found id: ""
I0630 20:59:51.093097 204258 logs.go:274] 1 containers: [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442]
I0630 20:59:51.093158 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.096486 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:59:51.096577 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:59:51.120299 204258 cri.go:87] found id: ""
I0630 20:59:51.120338 204258 logs.go:274] 0 containers: []
W0630 20:59:51.120348 204258 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:59:51.120359 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:59:51.120448 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:59:51.145822 204258 cri.go:87] found id: "459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:51.145844 204258 cri.go:87] found id: "15419e1d1ccf663ed38f52e8605540432e16581e4754237fc230d2e135dfafc8"
I0630 20:59:51.145851 204258 cri.go:87] found id: ""
I0630 20:59:51.145856 204258 logs.go:274] 2 containers: [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64 15419e1d1ccf663ed38f52e8605540432e16581e4754237fc230d2e135dfafc8]
I0630 20:59:51.145900 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.148909 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.151875 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:59:51.151952 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:59:51.178414 204258 cri.go:87] found id: "7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:51.178450 204258 cri.go:87] found id: ""
I0630 20:59:51.178461 204258 logs.go:274] 1 containers: [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281]
I0630 20:59:51.178525 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:51.181572 204258 logs.go:123] Gathering logs for etcd [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a] ...
I0630 20:59:51.181594 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:51.227719 204258 logs.go:123] Gathering logs for coredns [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38] ...
I0630 20:59:51.227758 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:51.254451 204258 logs.go:123] Gathering logs for kube-scheduler [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7] ...
I0630 20:59:51.254493 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:51.300065 204258 logs.go:123] Gathering logs for kube-proxy [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442] ...
I0630 20:59:51.300108 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:51.331866 204258 logs.go:123] Gathering logs for kube-controller-manager [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281] ...
I0630 20:59:51.331904 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:51.371714 204258 logs.go:123] Gathering logs for containerd ...
I0630 20:59:51.371748 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:59:51.423922 204258 logs.go:123] Gathering logs for dmesg ...
I0630 20:59:51.423957 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:59:51.446891 204258 logs.go:123] Gathering logs for describe nodes ...
I0630 20:59:51.446930 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0630 20:59:51.784852 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:53.785175 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:51.537407 204258 logs.go:123] Gathering logs for container status ...
I0630 20:59:51.537444 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:59:51.566151 204258 logs.go:123] Gathering logs for storage-provisioner [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64] ...
I0630 20:59:51.566180 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:51.596107 204258 logs.go:123] Gathering logs for storage-provisioner [15419e1d1ccf663ed38f52e8605540432e16581e4754237fc230d2e135dfafc8] ...
I0630 20:59:51.596211 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 15419e1d1ccf663ed38f52e8605540432e16581e4754237fc230d2e135dfafc8"
I0630 20:59:51.621767 204258 logs.go:123] Gathering logs for kubelet ...
I0630 20:59:51.621810 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0630 20:59:51.694615 204258 logs.go:123] Gathering logs for kube-apiserver [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc] ...
I0630 20:59:51.694655 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:54.229480 204258 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0630 20:59:54.239288 204258 api_server.go:71] duration metric: took 4m4.500497506s to wait for apiserver process to appear ...
I0630 20:59:54.239322 204258 api_server.go:87] waiting for apiserver healthz status ...
I0630 20:59:54.239343 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:59:54.239384 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:59:54.267806 204258 cri.go:87] found id: "62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:54.267838 204258 cri.go:87] found id: ""
I0630 20:59:54.267846 204258 logs.go:274] 1 containers: [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc]
I0630 20:59:54.267896 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.271045 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:59:54.271117 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:59:54.296816 204258 cri.go:87] found id: "43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:54.296842 204258 cri.go:87] found id: ""
I0630 20:59:54.296849 204258 logs.go:274] 1 containers: [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a]
I0630 20:59:54.296891 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.299954 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:59:54.300019 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:59:54.325250 204258 cri.go:87] found id: "f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:54.325282 204258 cri.go:87] found id: ""
I0630 20:59:54.325290 204258 logs.go:274] 1 containers: [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38]
I0630 20:59:54.325340 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.328486 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:59:54.328549 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:59:54.353675 204258 cri.go:87] found id: "55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:54.353704 204258 cri.go:87] found id: ""
I0630 20:59:54.353710 204258 logs.go:274] 1 containers: [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7]
I0630 20:59:54.353750 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.356816 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:59:54.356881 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:59:54.381125 204258 cri.go:87] found id: "8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:54.381150 204258 cri.go:87] found id: ""
I0630 20:59:54.381158 204258 logs.go:274] 1 containers: [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442]
I0630 20:59:54.381206 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.384143 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:59:54.384203 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:59:54.411352 204258 cri.go:87] found id: ""
I0630 20:59:54.411376 204258 logs.go:274] 0 containers: []
W0630 20:59:54.411387 204258 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:59:54.411394 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:59:54.411451 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:59:54.435468 204258 cri.go:87] found id: "459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:54.435503 204258 cri.go:87] found id: ""
I0630 20:59:54.435512 204258 logs.go:274] 1 containers: [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64]
I0630 20:59:54.435558 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.438924 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:59:54.438993 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:59:54.463319 204258 cri.go:87] found id: "7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:54.463343 204258 cri.go:87] found id: ""
I0630 20:59:54.463349 204258 logs.go:274] 1 containers: [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281]
I0630 20:59:54.463391 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:54.466589 204258 logs.go:123] Gathering logs for kube-proxy [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442] ...
I0630 20:59:54.466609 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:54.492311 204258 logs.go:123] Gathering logs for kube-controller-manager [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281] ...
I0630 20:59:54.492341 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:54.529848 204258 logs.go:123] Gathering logs for kubelet ...
I0630 20:59:54.529879 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0630 20:59:54.599564 204258 logs.go:123] Gathering logs for describe nodes ...
I0630 20:59:54.599603 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0630 20:59:54.678695 204258 logs.go:123] Gathering logs for kube-apiserver [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc] ...
I0630 20:59:54.678728 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:54.710288 204258 logs.go:123] Gathering logs for etcd [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a] ...
I0630 20:59:54.710330 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:54.752655 204258 logs.go:123] Gathering logs for coredns [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38] ...
I0630 20:59:54.752694 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:54.778040 204258 logs.go:123] Gathering logs for dmesg ...
I0630 20:59:54.778071 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:59:54.800904 204258 logs.go:123] Gathering logs for kube-scheduler [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7] ...
I0630 20:59:54.800943 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:54.832676 204258 logs.go:123] Gathering logs for storage-provisioner [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64] ...
I0630 20:59:54.832718 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:54.857300 204258 logs.go:123] Gathering logs for containerd ...
I0630 20:59:54.857328 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:59:54.905705 204258 logs.go:123] Gathering logs for container status ...
I0630 20:59:54.905744 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 20:59:56.285224 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 20:59:58.784686 201358 pod_ready.go:102] pod "calico-node-wkbg5" in "kube-system" namespace has status "Ready":"False"
I0630 21:00:00.464899 166730 out.go:204] - Generating certificates and keys ...
I0630 20:59:57.434162 204258 api_server.go:240] Checking apiserver healthz at https://192.168.94.2:8443/healthz ...
I0630 20:59:57.439505 204258 api_server.go:266] https://192.168.94.2:8443/healthz returned 200:
ok
I0630 20:59:57.440554 204258 api_server.go:140] control plane version: v1.24.2
I0630 20:59:57.440578 204258 api_server.go:130] duration metric: took 3.201250344s to wait for apiserver health ...
I0630 20:59:57.440586 204258 system_pods.go:43] waiting for kube-system pods to appear ...
I0630 20:59:57.440603 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 20:59:57.440646 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 20:59:57.467697 204258 cri.go:87] found id: "62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:57.467735 204258 cri.go:87] found id: ""
I0630 20:59:57.467744 204258 logs.go:274] 1 containers: [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc]
I0630 20:59:57.467801 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.471107 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 20:59:57.471167 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 20:59:57.498030 204258 cri.go:87] found id: "43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:57.498060 204258 cri.go:87] found id: ""
I0630 20:59:57.498068 204258 logs.go:274] 1 containers: [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a]
I0630 20:59:57.498120 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.501635 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 20:59:57.501716 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 20:59:57.528336 204258 cri.go:87] found id: "f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:57.528365 204258 cri.go:87] found id: ""
I0630 20:59:57.528374 204258 logs.go:274] 1 containers: [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38]
I0630 20:59:57.528432 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.531827 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 20:59:57.531899 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 20:59:57.556704 204258 cri.go:87] found id: "55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:57.556729 204258 cri.go:87] found id: ""
I0630 20:59:57.556735 204258 logs.go:274] 1 containers: [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7]
I0630 20:59:57.556774 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.559924 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 20:59:57.559986 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 20:59:57.584978 204258 cri.go:87] found id: "8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:57.585003 204258 cri.go:87] found id: ""
I0630 20:59:57.585009 204258 logs.go:274] 1 containers: [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442]
I0630 20:59:57.585097 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.588137 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 20:59:57.588218 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 20:59:57.614969 204258 cri.go:87] found id: ""
I0630 20:59:57.615013 204258 logs.go:274] 0 containers: []
W0630 20:59:57.615023 204258 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 20:59:57.615034 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 20:59:57.615094 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 20:59:57.640634 204258 cri.go:87] found id: "459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:57.640659 204258 cri.go:87] found id: ""
I0630 20:59:57.640668 204258 logs.go:274] 1 containers: [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64]
I0630 20:59:57.640727 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.643826 204258 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 20:59:57.643900 204258 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 20:59:57.668848 204258 cri.go:87] found id: "7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:57.668871 204258 cri.go:87] found id: ""
I0630 20:59:57.668878 204258 logs.go:274] 1 containers: [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281]
I0630 20:59:57.668924 204258 ssh_runner.go:195] Run: which crictl
I0630 20:59:57.672088 204258 logs.go:123] Gathering logs for describe nodes ...
I0630 20:59:57.672116 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0630 20:59:57.751793 204258 logs.go:123] Gathering logs for etcd [43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a] ...
I0630 20:59:57.751830 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 43ff3e48ab0fa1dc09c974a67ca4ffb8d9dd86186955f1c260b1d1ff6678952a"
I0630 20:59:57.802962 204258 logs.go:123] Gathering logs for coredns [f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38] ...
I0630 20:59:57.802998 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f2b46f99300c8cc82d9d8388a1160b169ac4e6e48c4dbd8a34623c826b312e38"
I0630 20:59:57.829859 204258 logs.go:123] Gathering logs for kube-scheduler [55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7] ...
I0630 20:59:57.829886 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 55a1817f09afa57603335e844d713f0fc6174e6abda688b905f35746d1ae59a7"
I0630 20:59:57.861172 204258 logs.go:123] Gathering logs for containerd ...
I0630 20:59:57.861209 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 20:59:57.911802 204258 logs.go:123] Gathering logs for kubelet ...
I0630 20:59:57.911839 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0630 20:59:57.979386 204258 logs.go:123] Gathering logs for dmesg ...
I0630 20:59:57.979425 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 20:59:58.001788 204258 logs.go:123] Gathering logs for kube-apiserver [62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc] ...
I0630 20:59:58.001823 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62b811c6ffa14a2829fd32d7de984945c828fdd5dec3a2a4b9b1798a3f8789fc"
I0630 20:59:58.034067 204258 logs.go:123] Gathering logs for kube-proxy [8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442] ...
I0630 20:59:58.034105 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8cfcad6cc8f0f78e60857082b51808647c3c2ab4df93788ba7f032cea1e99442"
I0630 20:59:58.067392 204258 logs.go:123] Gathering logs for storage-provisioner [459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64] ...
I0630 20:59:58.067421 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 459ec697c2224562149dc5e975c768fcef5502de232c45f26f23b269eae1ed64"
I0630 20:59:58.092362 204258 logs.go:123] Gathering logs for kube-controller-manager [7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281] ...
I0630 20:59:58.092393 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7d0d5a7231d313f078021e6ba364cf4bfdd4fd52074797fa961d6c9b09cfe281"
I0630 20:59:58.128409 204258 logs.go:123] Gathering logs for container status ...
I0630 20:59:58.128442 204258 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0630 21:00:00.663945 204258 system_pods.go:59] 7 kube-system pods found
I0630 21:00:00.663989 204258 system_pods.go:61] "coredns-6d4b75cb6d-5577k" [5614622f-41c8-4c3b-bec4-7716260f9e3c] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0630 21:00:00.663999 204258 system_pods.go:61] "etcd-enable-default-cni-20220630204858-9950" [4b9cdffc-5698-449c-b72e-71b6d19c7a1e] Running
I0630 21:00:00.664009 204258 system_pods.go:61] "kube-apiserver-enable-default-cni-20220630204858-9950" [c8150026-56f5-4d2c-b98d-ecb47dc4b3bb] Running
I0630 21:00:00.664017 204258 system_pods.go:61] "kube-controller-manager-enable-default-cni-20220630204858-9950" [3cff4b69-7fc7-461f-aef0-d75c0c8a37fa] Running
I0630 21:00:00.664030 204258 system_pods.go:61] "kube-proxy-98pfx" [6557e09c-bbc8-45ab-bbed-ffa24b3f882d] Running
I0630 21:00:00.664043 204258 system_pods.go:61] "kube-scheduler-enable-default-cni-20220630204858-9950" [89d45250-3f87-464b-9c30-493f41f99dec] Running
I0630 21:00:00.664058 204258 system_pods.go:61] "storage-provisioner" [465f8143-d118-43c0-920c-092dc51d9f42] Running / Ready:ContainersNotReady (containers with unready status: [storage-provisioner]) / ContainersReady:ContainersNotReady (containers with unready status: [storage-provisioner])
I0630 21:00:00.664066 204258 system_pods.go:74] duration metric: took 3.223474423s to wait for pod list to return data ...
I0630 21:00:00.664076 204258 default_sa.go:34] waiting for default service account to be created ...
I0630 21:00:00.666902 204258 default_sa.go:45] found service account: "default"
I0630 21:00:00.666929 204258 default_sa.go:55] duration metric: took 2.846736ms for default service account to be created ...
I0630 21:00:00.666938 204258 system_pods.go:116] waiting for k8s-apps to be running ...
I0630 21:00:00.672431 204258 system_pods.go:86] 7 kube-system pods found
I0630 21:00:00.672470 204258 system_pods.go:89] "coredns-6d4b75cb6d-5577k" [5614622f-41c8-4c3b-bec4-7716260f9e3c] Running / Ready:ContainersNotReady (containers with unready status: [coredns]) / ContainersReady:ContainersNotReady (containers with unready status: [coredns])
I0630 21:00:00.672482 204258 system_pods.go:89] "etcd-enable-default-cni-20220630204858-9950" [4b9cdffc-5698-449c-b72e-71b6d19c7a1e] Running
I0630 21:00:00.672502 204258 system_pods.go:89] "kube-apiserver-enable-default-cni-20220630204858-9950" [c8150026-56f5-4d2c-b98d-ecb47dc4b3bb] Running
I0630 21:00:00.672510 204258 system_pods.go:89] "kube-controller-manager-enable-default-cni-20220630204858-9950" [3cff4b69-7fc7-461f-aef0-d75c0c8a37fa] Running
I0630 21:00:00.672522 204258 system_pods.go:89] "kube-proxy-98pfx" [6557e09c-bbc8-45ab-bbed-ffa24b3f882d] Running
I0630 21:00:00.672537 204258 system_pods.go:89] "kube-scheduler-enable-default-cni-20220630204858-9950" [89d45250-3f87-464b-9c30-493f41f99dec] Running
I0630 21:00:00.672549 204258 system_pods.go:89] "storage-provisioner" [465f8143-d118-43c0-920c-092dc51d9f42] Running / Ready:ContainersNotReady (containers with unready status: [storage-provisioner]) / ContainersReady:ContainersNotReady (containers with unready status: [storage-provisioner])
I0630 21:00:00.672558 204258 system_pods.go:126] duration metric: took 5.614113ms to wait for k8s-apps to be running ...
I0630 21:00:00.672571 204258 system_svc.go:44] waiting for kubelet service to be running ....
I0630 21:00:00.672621 204258 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0630 21:00:00.682748 204258 system_svc.go:56] duration metric: took 10.168132ms WaitForService to wait for kubelet.
I0630 21:00:00.682778 204258 kubeadm.go:572] duration metric: took 4m10.943996394s to wait for : map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] ...
I0630 21:00:00.682822 204258 node_conditions.go:102] verifying NodePressure condition ...
I0630 21:00:00.686187 204258 node_conditions.go:122] node storage ephemeral capacity is 304695084Ki
I0630 21:00:00.686214 204258 node_conditions.go:123] node cpu capacity is 8
I0630 21:00:00.686226 204258 node_conditions.go:105] duration metric: took 3.394027ms to run NodePressure ...
I0630 21:00:00.686240 204258 start.go:213] waiting for startup goroutines ...
I0630 21:00:00.733915 204258 start.go:506] kubectl: 1.24.2, cluster: 1.24.2 (minor skew: 0)
I0630 21:00:00.737781 204258 out.go:177] * Done! kubectl is now configured to use "enable-default-cni-20220630204858-9950" cluster and "default" namespace by default
I0630 21:00:00.467813 166730 out.go:204] - Booting up control plane ...
I0630 21:00:00.470158 166730 kubeadm.go:397] StartCluster complete in 7m58.712661025s
I0630 21:00:00.470198 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0630 21:00:00.470254 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0630 21:00:00.494852 166730 cri.go:87] found id: ""
I0630 21:00:00.494875 166730 logs.go:274] 0 containers: []
W0630 21:00:00.494882 166730 logs.go:276] No container was found matching "kube-apiserver"
I0630 21:00:00.494888 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0630 21:00:00.494937 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0630 21:00:00.518642 166730 cri.go:87] found id: ""
I0630 21:00:00.518668 166730 logs.go:274] 0 containers: []
W0630 21:00:00.518674 166730 logs.go:276] No container was found matching "etcd"
I0630 21:00:00.518680 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0630 21:00:00.518728 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0630 21:00:00.544697 166730 cri.go:87] found id: ""
I0630 21:00:00.544733 166730 logs.go:274] 0 containers: []
W0630 21:00:00.544742 166730 logs.go:276] No container was found matching "coredns"
I0630 21:00:00.544750 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0630 21:00:00.544808 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0630 21:00:00.569280 166730 cri.go:87] found id: ""
I0630 21:00:00.569311 166730 logs.go:274] 0 containers: []
W0630 21:00:00.569320 166730 logs.go:276] No container was found matching "kube-scheduler"
I0630 21:00:00.569327 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0630 21:00:00.569383 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0630 21:00:00.593566 166730 cri.go:87] found id: ""
I0630 21:00:00.593592 166730 logs.go:274] 0 containers: []
W0630 21:00:00.593601 166730 logs.go:276] No container was found matching "kube-proxy"
I0630 21:00:00.593609 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0630 21:00:00.593665 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0630 21:00:00.618029 166730 cri.go:87] found id: ""
I0630 21:00:00.618053 166730 logs.go:274] 0 containers: []
W0630 21:00:00.618061 166730 logs.go:276] No container was found matching "kubernetes-dashboard"
I0630 21:00:00.618067 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0630 21:00:00.618111 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0630 21:00:00.642327 166730 cri.go:87] found id: ""
I0630 21:00:00.642352 166730 logs.go:274] 0 containers: []
W0630 21:00:00.642358 166730 logs.go:276] No container was found matching "storage-provisioner"
I0630 21:00:00.642365 166730 cri.go:52] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0630 21:00:00.642412 166730 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0630 21:00:00.671530 166730 cri.go:87] found id: ""
I0630 21:00:00.671557 166730 logs.go:274] 0 containers: []
W0630 21:00:00.671566 166730 logs.go:276] No container was found matching "kube-controller-manager"
I0630 21:00:00.671576 166730 logs.go:123] Gathering logs for kubelet ...
I0630 21:00:00.671591 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0630 21:00:00.735542 166730 logs.go:138] Found kubelet problem: Jun 30 21:00:00 kubernetes-upgrade-20220630205032-9950 kubelet[11593]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 21:00:00.809676 166730 logs.go:123] Gathering logs for dmesg ...
I0630 21:00:00.809714 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0630 21:00:00.837921 166730 logs.go:123] Gathering logs for describe nodes ...
I0630 21:00:00.837959 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
W0630 21:00:00.891715 166730 logs.go:130] failed describe nodes: command: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output:
** stderr **
The connection to the server localhost:8443 was refused - did you specify the right host or port?
** /stderr **
I0630 21:00:00.891742 166730 logs.go:123] Gathering logs for containerd ...
I0630 21:00:00.891751 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0630 21:00:00.937402 166730 logs.go:123] Gathering logs for container status ...
I0630 21:00:00.937438 166730 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
W0630 21:00:00.966782 166730 out.go:369] Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.966835 166730 out.go:239] *
W0630 21:00:00.967083 166730 out.go:239] X Error starting cluster: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.967126 166730 out.go:239] *
W0630 21:00:00.968129 166730 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0630 21:00:00.970508 166730 out.go:177] X Problems detected in kubelet:
I0630 21:00:00.972058 166730 out.go:177] Jun 30 21:00:00 kubernetes-upgrade-20220630205032-9950 kubelet[11593]: Error: failed to parse kubelet flag: unknown flag: --cni-conf-dir
I0630 21:00:00.976653 166730 out.go:177]
W0630 21:00:00.978457 166730 out.go:239] X Exiting due to K8S_KUBELET_NOT_RUNNING: wait: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.24.2:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables": Process exited with status 1
stdout:
[init] Using Kubernetes version: v1.24.2
[preflight] Running pre-flight checks
[preflight] The system verification failed. Printing the output from the verification:
[0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1012-gcp[0m
[0;37mOS[0m: [0;32mLinux[0m
[0;37mCGROUPS_CPU[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
[0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
[0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
[0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
[0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
[0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
[0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
[0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/var/lib/minikube/certs"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///run/containerd/containerd.sock logs CONTAINERID'
stderr:
W0630 20:58:04.642467 9717 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1012-gcp\n", err: exit status 1
[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
W0630 21:00:00.978591 166730 out.go:239] * Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
W0630 21:00:00.978662 166730 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/4172
I0630 21:00:00.981508 166730 out.go:177]
*
* ==> container status <==
* CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
*
* ==> containerd <==
* -- Logs begin at Thu 2022-06-30 20:51:27 UTC, end at Thu 2022-06-30 21:00:02 UTC. --
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.376253759Z" level=error msg="StopPodSandbox for \"\\\"Using\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"\\\"Using\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.393572166Z" level=info msg="StopPodSandbox for \"this\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.393635040Z" level=error msg="StopPodSandbox for \"this\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"this\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.410923206Z" level=info msg="StopPodSandbox for \"endpoint\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.410994487Z" level=error msg="StopPodSandbox for \"endpoint\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.428654634Z" level=info msg="StopPodSandbox for \"is\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.428705019Z" level=error msg="StopPodSandbox for \"is\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"is\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.446153688Z" level=info msg="StopPodSandbox for \"deprecated,\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.446208238Z" level=error msg="StopPodSandbox for \"deprecated,\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"deprecated,\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.463938104Z" level=info msg="StopPodSandbox for \"please\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.464001700Z" level=error msg="StopPodSandbox for \"please\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"please\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.481241720Z" level=info msg="StopPodSandbox for \"consider\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.481291044Z" level=error msg="StopPodSandbox for \"consider\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"consider\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.498254893Z" level=info msg="StopPodSandbox for \"using\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.498309424Z" level=error msg="StopPodSandbox for \"using\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"using\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.515161183Z" level=info msg="StopPodSandbox for \"full\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.515222508Z" level=error msg="StopPodSandbox for \"full\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"full\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.533130729Z" level=info msg="StopPodSandbox for \"URL\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.533192816Z" level=error msg="StopPodSandbox for \"URL\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.550894877Z" level=info msg="StopPodSandbox for \"format\\\"\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.550995498Z" level=error msg="StopPodSandbox for \"format\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"format\\\"\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.568366789Z" level=info msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.568442929Z" level=error msg="StopPodSandbox for \"endpoint=\\\"/run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"endpoint=\\\"/run/containerd/containerd.sock\\\"\": not found"
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.585785757Z" level=info msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\""
Jun 30 20:58:04 kubernetes-upgrade-20220630205032-9950 containerd[504]: time="2022-06-30T20:58:04.585845269Z" level=error msg="StopPodSandbox for \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find sandbox \"URL=\\\"unix:///run/containerd/containerd.sock\\\"\": not found"
*
* ==> describe nodes <==
*
* ==> dmesg <==
* [ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +2.959763] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.003892] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000007] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.027898] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +21.516116] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.005700] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000014] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.023898] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +2.967774] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.031816] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.023920] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +2.943775] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.027830] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
[ +1.019907] IPv4: martian source 10.244.0.98 from 10.244.0.3, on dev eth0
[ +0.000006] ll header: 00000000: ff ff ff ff ff ff 1e b8 96 f0 c5 1d 08 06
*
* ==> kernel <==
* 21:00:02 up 42 min, 0 users, load average: 0.79, 2.51, 2.31
Linux kubernetes-upgrade-20220630205032-9950 5.15.0-1012-gcp #17~20.04.1-Ubuntu SMP Thu Jun 23 16:10:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 20.04.4 LTS"
*
* ==> kubelet <==
* -- Logs begin at Thu 2022-06-30 20:51:27 UTC, end at Thu 2022-06-30 21:00:02 UTC. --
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-buffer-duration duration Writes in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction (default 1m0s) (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-db string database name (default "cadvisor") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-host string database host:port (default "localhost:8086") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-password string database password (default "root") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-secure use secure connection with database (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-table string table name (default "stats") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --storage-driver-user string database username (default "root") (DEPRECATED: This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --streaming-connection-idle-timeout duration Maximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: '5m'. Note: All connections to the kubelet server have a maximum duration of 4 hours. (default 4h0m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --sync-frequency duration Max period between synchronizing running containers and config (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --system-cgroups string Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under '/'. Empty for no container. Rolling back the flag requires a reboot. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --system-reserved mapStringString A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ for more detail. [default=none] (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --system-reserved-cgroup string Absolute name of the top level cgroup that is used to manage non-kubernetes components for which compute resources were reserved via '--system-reserved' flag. Ex. '/system-reserved'. [default=''] (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --tls-cert-file string File containing x509 Certificate used for serving HTTPS (with intermediate certs, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13 (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --tls-private-key-file string File containing x509 private key matching --tls-cert-file. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --topology-manager-policy string Topology Manager policy to use. Possible values: 'none', 'best-effort', 'restricted', 'single-numa-node'. (default "none") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --topology-manager-scope string Scope to which topology hints applied. Topology Manager collects hints from Hint Providers and applies them to defined scope to ensure the pod admission. Possible values: 'container', 'pod'. (default "container") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: -v, --v Level number for the log level verbosity
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --version version[=true] Print version information and quit
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --vmodule pattern=N,... comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --volume-plugin-dir string The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
Jun 30 21:00:01 kubernetes-upgrade-20220630205032-9950 kubelet[11763]: --volume-stats-agg-period duration Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to a negative number. (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
-- /stdout --
** stderr **
E0630 21:00:02.426023 228084 logs.go:192] command /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" failed with error: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.24.2/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output: "\n** stderr ** \nThe connection to the server localhost:8443 was refused - did you specify the right host or port?\n\n** /stderr **"
! unable to fetch logs for: describe nodes
** /stderr **
helpers_test.go:254: (dbg) Run: out/minikube-linux-amd64 status --format={{.APIServer}} -p kubernetes-upgrade-20220630205032-9950 -n kubernetes-upgrade-20220630205032-9950
helpers_test.go:254: (dbg) Non-zero exit: out/minikube-linux-amd64 status --format={{.APIServer}} -p kubernetes-upgrade-20220630205032-9950 -n kubernetes-upgrade-20220630205032-9950: exit status 2 (476.347377ms)
-- stdout --
Stopped
-- /stdout --
helpers_test.go:254: status error: exit status 2 (may be ok)
helpers_test.go:256: "kubernetes-upgrade-20220630205032-9950" apiserver is not running, skipping kubectl commands (state="Stopped")
helpers_test.go:175: Cleaning up "kubernetes-upgrade-20220630205032-9950" profile ...
helpers_test.go:178: (dbg) Run: out/minikube-linux-amd64 delete -p kubernetes-upgrade-20220630205032-9950
helpers_test.go:178: (dbg) Done: out/minikube-linux-amd64 delete -p kubernetes-upgrade-20220630205032-9950: (2.375287003s)
--- FAIL: TestKubernetesUpgrade (572.65s)