=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-654890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-654890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m17.122001464s)
-- stdout --
* [old-k8s-version-654890] minikube v1.34.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19696
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19696-296322/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19696-296322/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-654890" primary control-plane node in "old-k8s-version-654890" cluster
* Pulling base image v0.0.45-1727108449-19696 ...
* Restarting existing docker container for "old-k8s-version-654890" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-654890 addons enable metrics-server
* Enabled addons: storage-provisioner, default-storageclass, metrics-server, dashboard
-- /stdout --
** stderr **
I0924 01:13:54.204124 503471 out.go:345] Setting OutFile to fd 1 ...
I0924 01:13:54.204329 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:13:54.204350 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:13:54.204369 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:13:54.204651 503471 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19696-296322/.minikube/bin
I0924 01:13:54.205051 503471 out.go:352] Setting JSON to false
I0924 01:13:54.206073 503471 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":10580,"bootTime":1727129855,"procs":184,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0924 01:13:54.206164 503471 start.go:139] virtualization:
I0924 01:13:54.208866 503471 out.go:177] * [old-k8s-version-654890] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0924 01:13:54.211320 503471 notify.go:220] Checking for updates...
I0924 01:13:54.211917 503471 out.go:177] - MINIKUBE_LOCATION=19696
I0924 01:13:54.213608 503471 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0924 01:13:54.215428 503471 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:13:54.217309 503471 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19696-296322/.minikube
I0924 01:13:54.218765 503471 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0924 01:13:54.220327 503471 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0924 01:13:54.222795 503471 config.go:182] Loaded profile config "old-k8s-version-654890": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0924 01:13:54.225445 503471 out.go:177] * Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
I0924 01:13:54.227074 503471 driver.go:394] Setting default libvirt URI to qemu:///system
I0924 01:13:54.259027 503471 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0924 01:13:54.259150 503471 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0924 01:13:54.346326 503471 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:57 OomKillDisable:true NGoroutines:67 SystemTime:2024-09-24 01:13:54.327311133 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0924 01:13:54.346466 503471 docker.go:318] overlay module found
I0924 01:13:54.348628 503471 out.go:177] * Using the docker driver based on existing profile
I0924 01:13:54.350249 503471 start.go:297] selected driver: docker
I0924 01:13:54.350265 503471 start.go:901] validating driver "docker" against &{Name:old-k8s-version-654890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-654890 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:13:54.350378 503471 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0924 01:13:54.351028 503471 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0924 01:13:54.435614 503471 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:57 OomKillDisable:true NGoroutines:67 SystemTime:2024-09-24 01:13:54.42562535 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aa
rch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: bridg
e-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0924 01:13:54.435992 503471 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0924 01:13:54.436017 503471 cni.go:84] Creating CNI manager for ""
I0924 01:13:54.436062 503471 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0924 01:13:54.436105 503471 start.go:340] cluster config:
{Name:old-k8s-version-654890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-654890 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:13:54.438067 503471 out.go:177] * Starting "old-k8s-version-654890" primary control-plane node in "old-k8s-version-654890" cluster
I0924 01:13:54.439649 503471 cache.go:121] Beginning downloading kic base image for docker with containerd
I0924 01:13:54.441225 503471 out.go:177] * Pulling base image v0.0.45-1727108449-19696 ...
I0924 01:13:54.442798 503471 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0924 01:13:54.442855 503471 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19696-296322/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0924 01:13:54.442864 503471 cache.go:56] Caching tarball of preloaded images
I0924 01:13:54.443081 503471 preload.go:172] Found /home/jenkins/minikube-integration/19696-296322/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0924 01:13:54.443091 503471 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0924 01:13:54.443231 503471 profile.go:143] Saving config to /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/config.json ...
I0924 01:13:54.443443 503471 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local docker daemon
I0924 01:13:54.470462 503471 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local docker daemon, skipping pull
I0924 01:13:54.470483 503471 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 exists in daemon, skipping load
I0924 01:13:54.470497 503471 cache.go:194] Successfully downloaded all kic artifacts
I0924 01:13:54.470522 503471 start.go:360] acquireMachinesLock for old-k8s-version-654890: {Name:mkc8faf84579ab1d3e086b99a1098fa5376cb386 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:13:54.470574 503471 start.go:364] duration metric: took 36.02µs to acquireMachinesLock for "old-k8s-version-654890"
I0924 01:13:54.470594 503471 start.go:96] Skipping create...Using existing machine configuration
I0924 01:13:54.470599 503471 fix.go:54] fixHost starting:
I0924 01:13:54.470867 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:13:54.495007 503471 fix.go:112] recreateIfNeeded on old-k8s-version-654890: state=Stopped err=<nil>
W0924 01:13:54.495038 503471 fix.go:138] unexpected machine state, will restart: <nil>
I0924 01:13:54.497030 503471 out.go:177] * Restarting existing docker container for "old-k8s-version-654890" ...
I0924 01:13:54.498737 503471 cli_runner.go:164] Run: docker start old-k8s-version-654890
I0924 01:13:54.832711 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:13:54.857314 503471 kic.go:430] container "old-k8s-version-654890" state is running.
I0924 01:13:54.857691 503471 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-654890
I0924 01:13:54.886405 503471 profile.go:143] Saving config to /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/config.json ...
I0924 01:13:54.886642 503471 machine.go:93] provisionDockerMachine start ...
I0924 01:13:54.886700 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:54.911186 503471 main.go:141] libmachine: Using SSH client type: native
I0924 01:13:54.911452 503471 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33435 <nil> <nil>}
I0924 01:13:54.911462 503471 main.go:141] libmachine: About to run SSH command:
hostname
I0924 01:13:54.914536 503471 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0924 01:13:58.058864 503471 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-654890
I0924 01:13:58.058890 503471 ubuntu.go:169] provisioning hostname "old-k8s-version-654890"
I0924 01:13:58.058985 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:58.091802 503471 main.go:141] libmachine: Using SSH client type: native
I0924 01:13:58.092066 503471 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33435 <nil> <nil>}
I0924 01:13:58.092079 503471 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-654890 && echo "old-k8s-version-654890" | sudo tee /etc/hostname
I0924 01:13:58.276523 503471 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-654890
I0924 01:13:58.276672 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:58.314596 503471 main.go:141] libmachine: Using SSH client type: native
I0924 01:13:58.314847 503471 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33435 <nil> <nil>}
I0924 01:13:58.314865 503471 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-654890' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-654890/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-654890' | sudo tee -a /etc/hosts;
fi
fi
I0924 01:13:58.474948 503471 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0924 01:13:58.474979 503471 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19696-296322/.minikube CaCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19696-296322/.minikube}
I0924 01:13:58.475005 503471 ubuntu.go:177] setting up certificates
I0924 01:13:58.475015 503471 provision.go:84] configureAuth start
I0924 01:13:58.475079 503471 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-654890
I0924 01:13:58.495861 503471 provision.go:143] copyHostCerts
I0924 01:13:58.495929 503471 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem, removing ...
I0924 01:13:58.495943 503471 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem
I0924 01:13:58.496022 503471 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem (1078 bytes)
I0924 01:13:58.496134 503471 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem, removing ...
I0924 01:13:58.496145 503471 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem
I0924 01:13:58.496175 503471 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem (1123 bytes)
I0924 01:13:58.496236 503471 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem, removing ...
I0924 01:13:58.496245 503471 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem
I0924 01:13:58.496269 503471 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem (1675 bytes)
I0924 01:13:58.496322 503471 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-654890 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-654890]
I0924 01:13:58.832008 503471 provision.go:177] copyRemoteCerts
I0924 01:13:58.832079 503471 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0924 01:13:58.832126 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:58.850965 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:13:58.948792 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0924 01:13:58.982724 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0924 01:13:59.033314 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0924 01:13:59.069782 503471 provision.go:87] duration metric: took 594.749147ms to configureAuth
I0924 01:13:59.069808 503471 ubuntu.go:193] setting minikube options for container-runtime
I0924 01:13:59.070019 503471 config.go:182] Loaded profile config "old-k8s-version-654890": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0924 01:13:59.070028 503471 machine.go:96] duration metric: took 4.183377528s to provisionDockerMachine
I0924 01:13:59.070036 503471 start.go:293] postStartSetup for "old-k8s-version-654890" (driver="docker")
I0924 01:13:59.070050 503471 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0924 01:13:59.070111 503471 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0924 01:13:59.070163 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:59.100373 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:13:59.206282 503471 ssh_runner.go:195] Run: cat /etc/os-release
I0924 01:13:59.215721 503471 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0924 01:13:59.215762 503471 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0924 01:13:59.215785 503471 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0924 01:13:59.215798 503471 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0924 01:13:59.215809 503471 filesync.go:126] Scanning /home/jenkins/minikube-integration/19696-296322/.minikube/addons for local assets ...
I0924 01:13:59.215874 503471 filesync.go:126] Scanning /home/jenkins/minikube-integration/19696-296322/.minikube/files for local assets ...
I0924 01:13:59.215964 503471 filesync.go:149] local asset: /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem -> 3017112.pem in /etc/ssl/certs
I0924 01:13:59.216072 503471 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0924 01:13:59.230989 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem --> /etc/ssl/certs/3017112.pem (1708 bytes)
I0924 01:13:59.271637 503471 start.go:296] duration metric: took 201.585272ms for postStartSetup
I0924 01:13:59.271741 503471 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0924 01:13:59.271788 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:59.304130 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:13:59.404606 503471 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0924 01:13:59.411270 503471 fix.go:56] duration metric: took 4.940661217s for fixHost
I0924 01:13:59.411297 503471 start.go:83] releasing machines lock for "old-k8s-version-654890", held for 4.940714558s
I0924 01:13:59.411393 503471 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-654890
I0924 01:13:59.467129 503471 ssh_runner.go:195] Run: cat /version.json
I0924 01:13:59.467189 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:59.467413 503471 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0924 01:13:59.467488 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:13:59.504819 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:13:59.511041 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:13:59.603516 503471 ssh_runner.go:195] Run: systemctl --version
I0924 01:13:59.773399 503471 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0924 01:13:59.778292 503471 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0924 01:13:59.807609 503471 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0924 01:13:59.807685 503471 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0924 01:13:59.821105 503471 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0924 01:13:59.821222 503471 start.go:495] detecting cgroup driver to use...
I0924 01:13:59.821292 503471 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0924 01:13:59.821392 503471 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0924 01:13:59.843711 503471 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0924 01:13:59.866127 503471 docker.go:217] disabling cri-docker service (if available) ...
I0924 01:13:59.866240 503471 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0924 01:13:59.886781 503471 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0924 01:13:59.904338 503471 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0924 01:14:00.051030 503471 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0924 01:14:00.241188 503471 docker.go:233] disabling docker service ...
I0924 01:14:00.241406 503471 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0924 01:14:00.261071 503471 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0924 01:14:00.276926 503471 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0924 01:14:00.437871 503471 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0924 01:14:00.583750 503471 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0924 01:14:00.604367 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0924 01:14:00.626887 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0924 01:14:00.637514 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0924 01:14:00.653919 503471 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0924 01:14:00.654049 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0924 01:14:00.669121 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0924 01:14:00.680195 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0924 01:14:00.694572 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0924 01:14:00.714251 503471 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0924 01:14:00.728493 503471 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0924 01:14:00.742002 503471 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0924 01:14:00.756311 503471 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0924 01:14:00.768289 503471 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:14:00.916897 503471 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0924 01:14:01.230226 503471 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0924 01:14:01.230383 503471 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0924 01:14:01.239282 503471 start.go:563] Will wait 60s for crictl version
I0924 01:14:01.239352 503471 ssh_runner.go:195] Run: which crictl
I0924 01:14:01.243507 503471 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0924 01:14:01.331003 503471 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0924 01:14:01.331075 503471 ssh_runner.go:195] Run: containerd --version
I0924 01:14:01.367419 503471 ssh_runner.go:195] Run: containerd --version
I0924 01:14:01.404801 503471 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
I0924 01:14:01.406558 503471 cli_runner.go:164] Run: docker network inspect old-k8s-version-654890 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0924 01:14:01.428691 503471 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0924 01:14:01.432807 503471 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0924 01:14:01.447158 503471 kubeadm.go:883] updating cluster {Name:old-k8s-version-654890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-654890 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0924 01:14:01.447286 503471 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0924 01:14:01.447344 503471 ssh_runner.go:195] Run: sudo crictl images --output json
I0924 01:14:01.526877 503471 containerd.go:627] all images are preloaded for containerd runtime.
I0924 01:14:01.526898 503471 containerd.go:534] Images already preloaded, skipping extraction
I0924 01:14:01.526974 503471 ssh_runner.go:195] Run: sudo crictl images --output json
I0924 01:14:01.604332 503471 containerd.go:627] all images are preloaded for containerd runtime.
I0924 01:14:01.604413 503471 cache_images.go:84] Images are preloaded, skipping loading
I0924 01:14:01.604437 503471 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I0924 01:14:01.604595 503471 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-654890 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-654890 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0924 01:14:01.604704 503471 ssh_runner.go:195] Run: sudo crictl info
I0924 01:14:01.666077 503471 cni.go:84] Creating CNI manager for ""
I0924 01:14:01.666100 503471 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0924 01:14:01.666110 503471 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0924 01:14:01.666130 503471 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-654890 NodeName:old-k8s-version-654890 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0924 01:14:01.666271 503471 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-654890"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0924 01:14:01.666335 503471 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0924 01:14:01.676663 503471 binaries.go:44] Found k8s binaries, skipping transfer
I0924 01:14:01.676818 503471 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0924 01:14:01.692502 503471 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0924 01:14:01.725627 503471 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0924 01:14:01.757225 503471 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0924 01:14:01.793192 503471 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0924 01:14:01.796892 503471 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0924 01:14:01.813642 503471 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:14:01.968893 503471 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0924 01:14:01.988560 503471 certs.go:68] Setting up /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890 for IP: 192.168.76.2
I0924 01:14:01.988631 503471 certs.go:194] generating shared ca certs ...
I0924 01:14:01.988664 503471 certs.go:226] acquiring lock for ca certs: {Name:mk4a6ab65221805436b06c42ec4fde316fe470ee Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:14:01.988865 503471 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19696-296322/.minikube/ca.key
I0924 01:14:01.988951 503471 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.key
I0924 01:14:01.988978 503471 certs.go:256] generating profile certs ...
I0924 01:14:01.989115 503471 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/client.key
I0924 01:14:01.989226 503471 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/apiserver.key.bf70a5a8
I0924 01:14:01.989306 503471 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/proxy-client.key
I0924 01:14:01.989462 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711.pem (1338 bytes)
W0924 01:14:01.989527 503471 certs.go:480] ignoring /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711_empty.pem, impossibly tiny 0 bytes
I0924 01:14:01.989553 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem (1675 bytes)
I0924 01:14:01.989609 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem (1078 bytes)
I0924 01:14:01.989680 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem (1123 bytes)
I0924 01:14:01.989725 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem (1675 bytes)
I0924 01:14:01.989824 503471 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem (1708 bytes)
I0924 01:14:01.990670 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0924 01:14:02.076201 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0924 01:14:02.134196 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0924 01:14:02.213049 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0924 01:14:02.276858 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0924 01:14:02.341180 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0924 01:14:02.401001 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0924 01:14:02.440300 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/old-k8s-version-654890/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0924 01:14:02.476935 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711.pem --> /usr/share/ca-certificates/301711.pem (1338 bytes)
I0924 01:14:02.517375 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem --> /usr/share/ca-certificates/3017112.pem (1708 bytes)
I0924 01:14:02.555137 503471 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0924 01:14:02.598883 503471 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0924 01:14:02.633428 503471 ssh_runner.go:195] Run: openssl version
I0924 01:14:02.643815 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/301711.pem && ln -fs /usr/share/ca-certificates/301711.pem /etc/ssl/certs/301711.pem"
I0924 01:14:02.656320 503471 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/301711.pem
I0924 01:14:02.660467 503471 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 24 00:35 /usr/share/ca-certificates/301711.pem
I0924 01:14:02.660619 503471 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/301711.pem
I0924 01:14:02.674067 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/301711.pem /etc/ssl/certs/51391683.0"
I0924 01:14:02.687339 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3017112.pem && ln -fs /usr/share/ca-certificates/3017112.pem /etc/ssl/certs/3017112.pem"
I0924 01:14:02.704270 503471 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3017112.pem
I0924 01:14:02.708287 503471 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 24 00:35 /usr/share/ca-certificates/3017112.pem
I0924 01:14:02.708433 503471 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3017112.pem
I0924 01:14:02.715649 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/3017112.pem /etc/ssl/certs/3ec20f2e.0"
I0924 01:14:02.729215 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0924 01:14:02.744297 503471 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0924 01:14:02.754657 503471 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 24 00:24 /usr/share/ca-certificates/minikubeCA.pem
I0924 01:14:02.754795 503471 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0924 01:14:02.767733 503471 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0924 01:14:02.785969 503471 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0924 01:14:02.789939 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0924 01:14:02.801262 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0924 01:14:02.811497 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0924 01:14:02.818657 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0924 01:14:02.830685 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0924 01:14:02.843615 503471 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0924 01:14:02.854292 503471 kubeadm.go:392] StartCluster: {Name:old-k8s-version-654890 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-654890 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:14:02.854458 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0924 01:14:02.854572 503471 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0924 01:14:02.929748 503471 cri.go:89] found id: "e3cc2c47a4cf2f2faddb84e5a279c4f1763c7d9ec5a546753ea5403ac7a5df85"
I0924 01:14:02.929833 503471 cri.go:89] found id: "ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:14:02.929853 503471 cri.go:89] found id: "321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:14:02.929876 503471 cri.go:89] found id: "5f587d5f804b01bf19560ff7972d8a73b6e2ac92c381a2a9b560dd4e3c01ca76"
I0924 01:14:02.929908 503471 cri.go:89] found id: "a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:14:02.929931 503471 cri.go:89] found id: "a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:14:02.929950 503471 cri.go:89] found id: "92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:14:02.929968 503471 cri.go:89] found id: "840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:14:02.930000 503471 cri.go:89] found id: "4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:14:02.930026 503471 cri.go:89] found id: ""
I0924 01:14:02.930113 503471 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0924 01:14:02.954597 503471 cri.go:116] JSON = null
W0924 01:14:02.954720 503471 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 9
I0924 01:14:02.954845 503471 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0924 01:14:02.968780 503471 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0924 01:14:02.968803 503471 kubeadm.go:593] restartPrimaryControlPlane start ...
I0924 01:14:02.968860 503471 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0924 01:14:02.978764 503471 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0924 01:14:02.979248 503471 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-654890" does not appear in /home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:14:02.979360 503471 kubeconfig.go:62] /home/jenkins/minikube-integration/19696-296322/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-654890" cluster setting kubeconfig missing "old-k8s-version-654890" context setting]
I0924 01:14:02.979641 503471 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19696-296322/kubeconfig: {Name:mk12cf5f8c4244466c827b22ce4fe2341553290d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:14:02.980879 503471 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0924 01:14:02.992825 503471 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0924 01:14:02.992857 503471 kubeadm.go:597] duration metric: took 24.047364ms to restartPrimaryControlPlane
I0924 01:14:02.992866 503471 kubeadm.go:394] duration metric: took 138.58716ms to StartCluster
I0924 01:14:02.992881 503471 settings.go:142] acquiring lock: {Name:mk1b01c5281da0b61714a1aa76e5632af5b39da5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:14:02.992944 503471 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:14:02.993544 503471 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19696-296322/kubeconfig: {Name:mk12cf5f8c4244466c827b22ce4fe2341553290d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:14:02.993732 503471 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0924 01:14:02.994027 503471 config.go:182] Loaded profile config "old-k8s-version-654890": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0924 01:14:02.994069 503471 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0924 01:14:02.994189 503471 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-654890"
I0924 01:14:02.994218 503471 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-654890"
W0924 01:14:02.994230 503471 addons.go:243] addon storage-provisioner should already be in state true
I0924 01:14:02.994252 503471 host.go:66] Checking if "old-k8s-version-654890" exists ...
I0924 01:14:02.994757 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:14:02.995130 503471 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-654890"
I0924 01:14:02.995152 503471 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-654890"
I0924 01:14:02.995275 503471 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-654890"
I0924 01:14:02.995311 503471 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-654890"
W0924 01:14:02.995324 503471 addons.go:243] addon metrics-server should already be in state true
I0924 01:14:02.995349 503471 host.go:66] Checking if "old-k8s-version-654890" exists ...
I0924 01:14:02.995435 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:14:02.995820 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:14:02.997869 503471 addons.go:69] Setting dashboard=true in profile "old-k8s-version-654890"
I0924 01:14:02.998182 503471 addons.go:234] Setting addon dashboard=true in "old-k8s-version-654890"
W0924 01:14:02.998264 503471 addons.go:243] addon dashboard should already be in state true
I0924 01:14:02.998384 503471 host.go:66] Checking if "old-k8s-version-654890" exists ...
I0924 01:14:03.001793 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:14:02.998102 503471 out.go:177] * Verifying Kubernetes components...
I0924 01:14:03.008529 503471 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:14:03.059510 503471 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0924 01:14:03.062190 503471 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:14:03.062215 503471 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0924 01:14:03.062285 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:14:03.069493 503471 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-654890"
W0924 01:14:03.069517 503471 addons.go:243] addon default-storageclass should already be in state true
I0924 01:14:03.069544 503471 host.go:66] Checking if "old-k8s-version-654890" exists ...
I0924 01:14:03.069995 503471 cli_runner.go:164] Run: docker container inspect old-k8s-version-654890 --format={{.State.Status}}
I0924 01:14:03.078896 503471 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0924 01:14:03.081122 503471 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0924 01:14:03.082884 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0924 01:14:03.087364 503471 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0924 01:14:03.087455 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:14:03.103286 503471 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0924 01:14:03.107019 503471 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0924 01:14:03.107048 503471 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0924 01:14:03.107125 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:14:03.132337 503471 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:03.132358 503471 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0924 01:14:03.132445 503471 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-654890
I0924 01:14:03.152072 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:14:03.152069 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:14:03.187277 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:14:03.188768 503471 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33435 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/old-k8s-version-654890/id_rsa Username:docker}
I0924 01:14:03.274295 503471 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0924 01:14:03.328294 503471 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-654890" to be "Ready" ...
I0924 01:14:03.398167 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:14:03.452886 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:03.465793 503471 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0924 01:14:03.465865 503471 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0924 01:14:03.469256 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0924 01:14:03.469326 503471 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0924 01:14:03.564206 503471 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0924 01:14:03.564283 503471 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0924 01:14:03.579359 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0924 01:14:03.579474 503471 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0924 01:14:03.658250 503471 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0924 01:14:03.658291 503471 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0924 01:14:03.699741 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0924 01:14:03.699769 503471 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0924 01:14:03.762700 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0924 01:14:03.816205 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0924 01:14:03.816240 503471 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0924 01:14:03.834694 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:03.834729 503471 retry.go:31] will retry after 367.570998ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:03.834771 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:03.834784 503471 retry.go:31] will retry after 175.936732ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:03.874898 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0924 01:14:03.874954 503471 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0924 01:14:03.904154 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0924 01:14:03.904180 503471 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0924 01:14:03.937557 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0924 01:14:03.937577 503471 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0924 01:14:03.983939 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0924 01:14:03.983972 503471 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0924 01:14:04.011255 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:04.045630 503471 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:14:04.045657 503471 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
W0924 01:14:04.057883 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.057917 503471 retry.go:31] will retry after 200.183082ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.104278 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:14:04.203461 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0924 01:14:04.206292 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.206328 503471 retry.go:31] will retry after 512.627283ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.258388 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:04.369002 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.369034 503471 retry.go:31] will retry after 178.378753ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:04.377633 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.377665 503471 retry.go:31] will retry after 254.640462ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:04.439179 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.439210 503471 retry.go:31] will retry after 457.430386ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.548474 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:14:04.632920 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0924 01:14:04.649336 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.649380 503471 retry.go:31] will retry after 532.909782ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.719668 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0924 01:14:04.749312 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.749369 503471 retry.go:31] will retry after 696.192479ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:04.834603 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.834632 503471 retry.go:31] will retry after 436.646704ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.897903 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:04.994009 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:04.994088 503471 retry.go:31] will retry after 685.667663ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.182533 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0924 01:14:05.261623 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.261659 503471 retry.go:31] will retry after 801.400659ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.271825 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:05.329387 503471 node_ready.go:53] error getting node "old-k8s-version-654890": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-654890": dial tcp 192.168.76.2:8443: connect: connection refused
W0924 01:14:05.381331 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.381415 503471 retry.go:31] will retry after 1.163466849s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.446729 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0924 01:14:05.568128 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.568160 503471 retry.go:31] will retry after 864.970681ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.680416 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:05.798225 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:05.798308 503471 retry.go:31] will retry after 773.671596ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.064193 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0924 01:14:06.161894 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.161940 503471 retry.go:31] will retry after 768.779034ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.433938 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0924 01:14:06.538428 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.538465 503471 retry.go:31] will retry after 1.325643457s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.545754 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:06.573072 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:06.732737 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.732772 503471 retry.go:31] will retry after 941.091079ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:06.732854 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.732878 503471 retry.go:31] will retry after 1.548834318s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:06.931653 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0924 01:14:07.032048 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:07.032083 503471 retry.go:31] will retry after 1.645231173s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:07.674499 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:07.756406 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:07.756453 503471 retry.go:31] will retry after 1.745097125s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:07.828908 503471 node_ready.go:53] error getting node "old-k8s-version-654890": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-654890": dial tcp 192.168.76.2:8443: connect: connection refused
I0924 01:14:07.865140 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0924 01:14:07.944881 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:07.944916 503471 retry.go:31] will retry after 1.525652115s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:08.282519 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0924 01:14:08.369628 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:08.369663 503471 retry.go:31] will retry after 1.389512767s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:08.678200 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0924 01:14:08.754754 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:08.754842 503471 retry.go:31] will retry after 2.036571362s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:09.470784 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:14:09.502252 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:14:09.586530 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:09.586569 503471 retry.go:31] will retry after 2.225465399s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0924 01:14:09.644409 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:09.644439 503471 retry.go:31] will retry after 3.271834312s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:09.759822 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:09.829582 503471 node_ready.go:53] error getting node "old-k8s-version-654890": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-654890": dial tcp 192.168.76.2:8443: connect: connection refused
W0924 01:14:09.838784 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:09.838864 503471 retry.go:31] will retry after 2.427502636s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:10.792132 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0924 01:14:10.880161 503471 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:10.880195 503471 retry.go:31] will retry after 2.289911778s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0924 01:14:11.812239 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:14:12.266773 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:14:12.917312 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0924 01:14:13.170900 503471 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:14:22.426121 503471 node_ready.go:49] node "old-k8s-version-654890" has status "Ready":"True"
I0924 01:14:22.426145 503471 node_ready.go:38] duration metric: took 19.09780287s for node "old-k8s-version-654890" to be "Ready" ...
I0924 01:14:22.426156 503471 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0924 01:14:22.577325 503471 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-4h5vb" in "kube-system" namespace to be "Ready" ...
I0924 01:14:22.670256 503471 pod_ready.go:93] pod "coredns-74ff55c5b-4h5vb" in "kube-system" namespace has status "Ready":"True"
I0924 01:14:22.670331 503471 pod_ready.go:82] duration metric: took 92.920444ms for pod "coredns-74ff55c5b-4h5vb" in "kube-system" namespace to be "Ready" ...
I0924 01:14:22.670357 503471 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:14:23.563677 503471 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (11.751399049s)
I0924 01:14:23.563728 503471 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (11.296932192s)
I0924 01:14:23.610159 503471 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (10.692804081s)
I0924 01:14:23.610262 503471 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-654890"
I0924 01:14:23.662110 503471 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (10.491152355s)
I0924 01:14:23.664240 503471 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-654890 addons enable metrics-server
I0924 01:14:23.666388 503471 out.go:177] * Enabled addons: storage-provisioner, default-storageclass, metrics-server, dashboard
I0924 01:14:23.668218 503471 addons.go:510] duration metric: took 20.674138465s for enable addons: enabled=[storage-provisioner default-storageclass metrics-server dashboard]
I0924 01:14:24.680062 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:27.177292 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:29.676819 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:31.677286 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:34.177119 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:36.178004 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:38.677002 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:41.177107 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:43.178117 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:45.214448 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:47.683752 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:50.177393 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:52.178392 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:54.178754 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:56.676381 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:14:58.676785 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:00.677645 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:03.177370 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:05.180516 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:07.675923 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:09.676736 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:12.177175 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:14.177211 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:16.182786 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:18.677465 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:20.677676 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:23.177277 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:25.178088 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:27.678473 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:30.178376 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:32.713650 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:35.177286 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:37.177700 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:39.675978 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:41.681287 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:43.177206 503471 pod_ready.go:93] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:43.177231 503471 pod_ready.go:82] duration metric: took 1m20.506853276s for pod "etcd-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.177248 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.182450 503471 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:43.182476 503471 pod_ready.go:82] duration metric: took 5.21957ms for pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.182507 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:45.191626 503471 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:47.190352 503471 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.190379 503471 pod_ready.go:82] duration metric: took 4.00786197s for pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.190392 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-dctnp" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.195753 503471 pod_ready.go:93] pod "kube-proxy-dctnp" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.195776 503471 pod_ready.go:82] duration metric: took 5.355496ms for pod "kube-proxy-dctnp" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.195787 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.201308 503471 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.201337 503471 pod_ready.go:82] duration metric: took 5.540809ms for pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.201351 503471 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace to be "Ready" ...
I0924 01:15:49.208658 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:51.707222 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:53.708077 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:56.209641 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:58.707466 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:00.708419 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:03.207876 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:05.208533 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:07.707911 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:09.707959 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:12.207936 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:14.708194 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:17.208113 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:19.208473 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:21.710594 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:24.208195 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:26.209405 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:28.713846 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:31.208503 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:33.707734 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:35.708337 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:38.207839 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:40.213566 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:42.709635 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:45.211860 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:47.707193 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:49.707944 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:51.710202 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:53.712325 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:56.212429 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:58.708373 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:01.208889 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:03.227933 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:05.709089 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:08.208394 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:10.208508 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:12.707726 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:15.208067 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:17.208790 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:19.209149 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:21.708671 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:24.216251 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:26.707770 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:29.208529 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:31.709364 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:33.761534 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:36.208565 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:38.707551 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:40.707768 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:42.707814 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:44.707848 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:46.708038 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:48.708177 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:51.207438 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:53.208301 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:55.707839 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:58.208516 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:00.235354 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:02.707695 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:05.208024 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:07.707243 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:09.707408 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:11.707886 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:14.207930 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:16.707304 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:19.209677 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:21.708310 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:24.207628 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:26.208190 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:28.707753 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:31.207884 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:33.208047 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:35.208663 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:37.707832 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:40.207918 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:42.209391 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:44.710598 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:47.207389 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:49.207968 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:51.208089 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:53.215727 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:55.707466 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:57.707846 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:00.226503 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:02.708170 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:05.207792 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:07.208012 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:09.208719 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:11.707807 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:13.707883 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:16.208372 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:18.208583 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:20.707950 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:22.711943 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:24.759385 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:27.207971 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:29.209588 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:31.709364 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:34.212724 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:36.708375 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:39.208776 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:41.708313 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:43.708560 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:46.207038 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:47.207950 503471 pod_ready.go:82] duration metric: took 4m0.006584819s for pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace to be "Ready" ...
E0924 01:19:47.207975 503471 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0924 01:19:47.207985 503471 pod_ready.go:39] duration metric: took 5m24.781817041s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0924 01:19:47.207999 503471 api_server.go:52] waiting for apiserver process to appear ...
I0924 01:19:47.208031 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:47.208103 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:47.248170 503471 cri.go:89] found id: "0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:47.248237 503471 cri.go:89] found id: "a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:47.248255 503471 cri.go:89] found id: ""
I0924 01:19:47.248294 503471 logs.go:276] 2 containers: [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c]
I0924 01:19:47.248373 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.252444 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.255922 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:47.256037 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:47.299255 503471 cri.go:89] found id: "1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:47.299279 503471 cri.go:89] found id: "4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:47.299284 503471 cri.go:89] found id: ""
I0924 01:19:47.299291 503471 logs.go:276] 2 containers: [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2]
I0924 01:19:47.299363 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.303065 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.307165 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:47.307239 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:47.344657 503471 cri.go:89] found id: "726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:47.344678 503471 cri.go:89] found id: "ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:47.344683 503471 cri.go:89] found id: ""
I0924 01:19:47.344690 503471 logs.go:276] 2 containers: [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c]
I0924 01:19:47.344774 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.348345 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.352584 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:47.352658 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:47.397310 503471 cri.go:89] found id: "11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:47.397331 503471 cri.go:89] found id: "92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:47.397336 503471 cri.go:89] found id: ""
I0924 01:19:47.397343 503471 logs.go:276] 2 containers: [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3]
I0924 01:19:47.397400 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.401198 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.404571 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:47.404647 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:47.450078 503471 cri.go:89] found id: "a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:47.450102 503471 cri.go:89] found id: "a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:47.450107 503471 cri.go:89] found id: ""
I0924 01:19:47.450114 503471 logs.go:276] 2 containers: [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c]
I0924 01:19:47.450195 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.454086 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.457899 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:47.457973 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:47.505274 503471 cri.go:89] found id: "14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:47.505344 503471 cri.go:89] found id: "840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:47.505363 503471 cri.go:89] found id: ""
I0924 01:19:47.505390 503471 logs.go:276] 2 containers: [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55]
I0924 01:19:47.505489 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.509368 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.513105 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:47.513224 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:47.552883 503471 cri.go:89] found id: "c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:47.552915 503471 cri.go:89] found id: "321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:47.552922 503471 cri.go:89] found id: ""
I0924 01:19:47.552930 503471 logs.go:276] 2 containers: [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee]
I0924 01:19:47.553023 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.556760 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.560250 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:47.560322 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:47.601478 503471 cri.go:89] found id: "ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:47.601521 503471 cri.go:89] found id: ""
I0924 01:19:47.601530 503471 logs.go:276] 1 containers: [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63]
I0924 01:19:47.601588 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.605414 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:47.605493 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:47.662058 503471 cri.go:89] found id: "fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:47.662083 503471 cri.go:89] found id: "fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:47.662088 503471 cri.go:89] found id: ""
I0924 01:19:47.662096 503471 logs.go:276] 2 containers: [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d]
I0924 01:19:47.662156 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.666136 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.669638 503471 logs.go:123] Gathering logs for coredns [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9] ...
I0924 01:19:47.669676 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:47.710636 503471 logs.go:123] Gathering logs for kube-scheduler [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57] ...
I0924 01:19:47.710669 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:47.753288 503471 logs.go:123] Gathering logs for container status ...
I0924 01:19:47.753320 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:19:47.800841 503471 logs.go:123] Gathering logs for dmesg ...
I0924 01:19:47.800872 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:19:47.817994 503471 logs.go:123] Gathering logs for describe nodes ...
I0924 01:19:47.818024 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:19:47.981580 503471 logs.go:123] Gathering logs for kube-apiserver [a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c] ...
I0924 01:19:47.981616 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:48.052176 503471 logs.go:123] Gathering logs for containerd ...
I0924 01:19:48.052216 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:19:48.118198 503471 logs.go:123] Gathering logs for kube-proxy [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d] ...
I0924 01:19:48.118240 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:48.160574 503471 logs.go:123] Gathering logs for kubernetes-dashboard [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63] ...
I0924 01:19:48.160607 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:48.205909 503471 logs.go:123] Gathering logs for storage-provisioner [fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d] ...
I0924 01:19:48.205939 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:48.245163 503471 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:48.245190 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:48.298751 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431464 666 reflector.go:138] object-"kube-system"/"kube-proxy-token-6n88c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-6n88c" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299014 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431566 666 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299243 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431638 666 reflector.go:138] object-"kube-system"/"kindnet-token-jt6n9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jt6n9" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299478 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431704 666 reflector.go:138] object-"kube-system"/"storage-provisioner-token-g5gtv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-g5gtv" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299693 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431771 666 reflector.go:138] object-"default"/"default-token-2t7hj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-2t7hj" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299915 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431832 666 reflector.go:138] object-"kube-system"/"metrics-server-token-dpjw8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-dpjw8" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.300143 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433069 666 reflector.go:138] object-"kube-system"/"coredns-token-djfwt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-djfwt" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.300363 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433138 666 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.307885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:24 old-k8s-version-654890 kubelet[666]: E0924 01:14:24.244333 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.309456 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:25 old-k8s-version-654890 kubelet[666]: E0924 01:14:25.186030 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.312301 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:39 old-k8s-version-654890 kubelet[666]: E0924 01:14:39.793083 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.314771 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:51 old-k8s-version-654890 kubelet[666]: E0924 01:14:51.304991 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.315114 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:52 old-k8s-version-654890 kubelet[666]: E0924 01:14:52.318075 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.315302 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:53 old-k8s-version-654890 kubelet[666]: E0924 01:14:53.784377 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.315745 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:55 old-k8s-version-654890 kubelet[666]: E0924 01:14:55.344748 666 pod_workers.go:191] Error syncing pod c12ca6a0-fd9b-45bf-9da0-2ec1193cce32 ("storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"
W0924 01:19:48.316678 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:01 old-k8s-version-654890 kubelet[666]: E0924 01:15:01.366255 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.319205 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:07 old-k8s-version-654890 kubelet[666]: E0924 01:15:07.792722 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.319672 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:11 old-k8s-version-654890 kubelet[666]: E0924 01:15:11.008383 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.319860 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:20 old-k8s-version-654890 kubelet[666]: E0924 01:15:20.784619 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.320481 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:26 old-k8s-version-654890 kubelet[666]: E0924 01:15:26.452520 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.320821 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:30 old-k8s-version-654890 kubelet[666]: E0924 01:15:30.986615 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.321011 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:33 old-k8s-version-654890 kubelet[666]: E0924 01:15:33.783850 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.321345 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.783545 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.321534 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.784908 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.321873 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.784164 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.324342 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.792225 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.324660 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:14 old-k8s-version-654890 kubelet[666]: E0924 01:16:14.789130 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.325123 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:15 old-k8s-version-654890 kubelet[666]: E0924 01:16:15.580475 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.325467 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:20 old-k8s-version-654890 kubelet[666]: E0924 01:16:20.992161 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.325652 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:29 old-k8s-version-654890 kubelet[666]: E0924 01:16:29.787709 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.326017 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:34 old-k8s-version-654890 kubelet[666]: E0924 01:16:34.784335 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.326235 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:42 old-k8s-version-654890 kubelet[666]: E0924 01:16:42.784427 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.326575 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:45 old-k8s-version-654890 kubelet[666]: E0924 01:16:45.783505 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.326768 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:53 old-k8s-version-654890 kubelet[666]: E0924 01:16:53.784108 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.327107 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:57 old-k8s-version-654890 kubelet[666]: E0924 01:16:57.783801 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.327294 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:06 old-k8s-version-654890 kubelet[666]: E0924 01:17:06.784121 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.327631 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:11 old-k8s-version-654890 kubelet[666]: E0924 01:17:11.783481 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.327817 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:18 old-k8s-version-654890 kubelet[666]: E0924 01:17:18.783911 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.328150 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:24 old-k8s-version-654890 kubelet[666]: E0924 01:17:24.784083 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.330620 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:31 old-k8s-version-654890 kubelet[666]: E0924 01:17:31.792392 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.331220 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:39 old-k8s-version-654890 kubelet[666]: E0924 01:17:39.856284 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.331553 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:40 old-k8s-version-654890 kubelet[666]: E0924 01:17:40.985923 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.331738 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:45 old-k8s-version-654890 kubelet[666]: E0924 01:17:45.784024 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.332067 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:55 old-k8s-version-654890 kubelet[666]: E0924 01:17:55.783900 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.332252 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:59 old-k8s-version-654890 kubelet[666]: E0924 01:17:59.784186 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.332585 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:06 old-k8s-version-654890 kubelet[666]: E0924 01:18:06.788365 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.332774 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:12 old-k8s-version-654890 kubelet[666]: E0924 01:18:12.783859 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.333106 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:18 old-k8s-version-654890 kubelet[666]: E0924 01:18:18.783560 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.333291 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:27 old-k8s-version-654890 kubelet[666]: E0924 01:18:27.783904 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.333627 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:30 old-k8s-version-654890 kubelet[666]: E0924 01:18:30.783887 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.333814 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:41 old-k8s-version-654890 kubelet[666]: E0924 01:18:41.784268 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.334149 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: E0924 01:18:44.783947 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.334334 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:55 old-k8s-version-654890 kubelet[666]: E0924 01:18:55.783966 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.334664 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: E0924 01:18:58.784580 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.334851 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:09 old-k8s-version-654890 kubelet[666]: E0924 01:19:09.783890 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.335185 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: E0924 01:19:11.783671 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.335370 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.335699 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.336028 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.336213 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.336399 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0924 01:19:48.336409 503471 logs.go:123] Gathering logs for kube-controller-manager [840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55] ...
I0924 01:19:48.336425 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:48.394895 503471 logs.go:123] Gathering logs for kindnet [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335] ...
I0924 01:19:48.394935 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:48.445555 503471 logs.go:123] Gathering logs for coredns [ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c] ...
I0924 01:19:48.445586 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:48.484819 503471 logs.go:123] Gathering logs for kube-scheduler [92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3] ...
I0924 01:19:48.484886 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:48.531995 503471 logs.go:123] Gathering logs for kube-proxy [a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c] ...
I0924 01:19:48.532082 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:48.573118 503471 logs.go:123] Gathering logs for kube-controller-manager [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b] ...
I0924 01:19:48.573189 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:48.633525 503471 logs.go:123] Gathering logs for kindnet [321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee] ...
I0924 01:19:48.633563 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:48.680372 503471 logs.go:123] Gathering logs for kube-apiserver [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f] ...
I0924 01:19:48.680403 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:48.742350 503471 logs.go:123] Gathering logs for etcd [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e] ...
I0924 01:19:48.742384 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:48.797001 503471 logs.go:123] Gathering logs for etcd [4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2] ...
I0924 01:19:48.797035 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:48.847657 503471 logs.go:123] Gathering logs for storage-provisioner [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2] ...
I0924 01:19:48.847687 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:48.891111 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:48.891138 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:19:48.891192 503471 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0924 01:19:48.891209 503471 out.go:270] Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.891225 503471 out.go:270] Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.891256 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.891265 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.891275 503471 out.go:270] Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0924 01:19:48.891281 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:48.891290 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:19:58.892677 503471 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0924 01:19:58.904580 503471 api_server.go:72] duration metric: took 5m55.910809038s to wait for apiserver process to appear ...
I0924 01:19:58.904607 503471 api_server.go:88] waiting for apiserver healthz status ...
I0924 01:19:58.904644 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:58.904701 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:58.944094 503471 cri.go:89] found id: "0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:58.944123 503471 cri.go:89] found id: "a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:58.944129 503471 cri.go:89] found id: ""
I0924 01:19:58.944140 503471 logs.go:276] 2 containers: [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c]
I0924 01:19:58.944210 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.948097 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.952102 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:58.952189 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:58.990627 503471 cri.go:89] found id: "1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:58.990651 503471 cri.go:89] found id: "4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:58.990657 503471 cri.go:89] found id: ""
I0924 01:19:58.990664 503471 logs.go:276] 2 containers: [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2]
I0924 01:19:58.990744 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.994962 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.998358 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:58.998428 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:59.039355 503471 cri.go:89] found id: "726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:59.039379 503471 cri.go:89] found id: "ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:59.039384 503471 cri.go:89] found id: ""
I0924 01:19:59.039391 503471 logs.go:276] 2 containers: [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c]
I0924 01:19:59.039451 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.043628 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.047352 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:59.047432 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:59.088932 503471 cri.go:89] found id: "11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:59.088957 503471 cri.go:89] found id: "92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:59.088963 503471 cri.go:89] found id: ""
I0924 01:19:59.088970 503471 logs.go:276] 2 containers: [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3]
I0924 01:19:59.089029 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.093313 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.096780 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:59.096850 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:59.137471 503471 cri.go:89] found id: "a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:59.137492 503471 cri.go:89] found id: "a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:59.137497 503471 cri.go:89] found id: ""
I0924 01:19:59.137505 503471 logs.go:276] 2 containers: [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c]
I0924 01:19:59.137584 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.141423 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.144785 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:59.144903 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:59.183946 503471 cri.go:89] found id: "14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:59.183968 503471 cri.go:89] found id: "840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:59.183973 503471 cri.go:89] found id: ""
I0924 01:19:59.183980 503471 logs.go:276] 2 containers: [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55]
I0924 01:19:59.184038 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.187604 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.191086 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:59.191163 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:59.233371 503471 cri.go:89] found id: "c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:59.233394 503471 cri.go:89] found id: "321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:59.233399 503471 cri.go:89] found id: ""
I0924 01:19:59.233407 503471 logs.go:276] 2 containers: [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee]
I0924 01:19:59.233487 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.237332 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.241220 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:59.241332 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:59.284694 503471 cri.go:89] found id: "fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:59.284770 503471 cri.go:89] found id: "fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:59.284783 503471 cri.go:89] found id: ""
I0924 01:19:59.284791 503471 logs.go:276] 2 containers: [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d]
I0924 01:19:59.284904 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.288841 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.292850 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:59.292961 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:59.342825 503471 cri.go:89] found id: "ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:59.342879 503471 cri.go:89] found id: ""
I0924 01:19:59.342902 503471 logs.go:276] 1 containers: [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63]
I0924 01:19:59.343028 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.346892 503471 logs.go:123] Gathering logs for kindnet [321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee] ...
I0924 01:19:59.346956 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:59.390245 503471 logs.go:123] Gathering logs for storage-provisioner [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2] ...
I0924 01:19:59.390298 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:59.430145 503471 logs.go:123] Gathering logs for coredns [ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c] ...
I0924 01:19:59.430171 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:59.477526 503471 logs.go:123] Gathering logs for kube-scheduler [92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3] ...
I0924 01:19:59.477553 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:59.522254 503471 logs.go:123] Gathering logs for kube-proxy [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d] ...
I0924 01:19:59.522285 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:59.578762 503471 logs.go:123] Gathering logs for etcd [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e] ...
I0924 01:19:59.578860 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:59.621417 503471 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:59.621447 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:59.677632 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431464 666 reflector.go:138] object-"kube-system"/"kube-proxy-token-6n88c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-6n88c" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.677885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431566 666 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678107 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431638 666 reflector.go:138] object-"kube-system"/"kindnet-token-jt6n9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jt6n9" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678337 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431704 666 reflector.go:138] object-"kube-system"/"storage-provisioner-token-g5gtv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-g5gtv" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678549 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431771 666 reflector.go:138] object-"default"/"default-token-2t7hj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-2t7hj" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678769 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431832 666 reflector.go:138] object-"kube-system"/"metrics-server-token-dpjw8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-dpjw8" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.679001 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433069 666 reflector.go:138] object-"kube-system"/"coredns-token-djfwt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-djfwt" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.679205 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433138 666 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.686595 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:24 old-k8s-version-654890 kubelet[666]: E0924 01:14:24.244333 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.688182 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:25 old-k8s-version-654890 kubelet[666]: E0924 01:14:25.186030 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.690962 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:39 old-k8s-version-654890 kubelet[666]: E0924 01:14:39.793083 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.693368 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:51 old-k8s-version-654890 kubelet[666]: E0924 01:14:51.304991 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.693697 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:52 old-k8s-version-654890 kubelet[666]: E0924 01:14:52.318075 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.693885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:53 old-k8s-version-654890 kubelet[666]: E0924 01:14:53.784377 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.694358 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:55 old-k8s-version-654890 kubelet[666]: E0924 01:14:55.344748 666 pod_workers.go:191] Error syncing pod c12ca6a0-fd9b-45bf-9da0-2ec1193cce32 ("storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"
W0924 01:19:59.695337 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:01 old-k8s-version-654890 kubelet[666]: E0924 01:15:01.366255 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.697828 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:07 old-k8s-version-654890 kubelet[666]: E0924 01:15:07.792722 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.698297 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:11 old-k8s-version-654890 kubelet[666]: E0924 01:15:11.008383 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.698482 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:20 old-k8s-version-654890 kubelet[666]: E0924 01:15:20.784619 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.699079 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:26 old-k8s-version-654890 kubelet[666]: E0924 01:15:26.452520 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.699407 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:30 old-k8s-version-654890 kubelet[666]: E0924 01:15:30.986615 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.699593 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:33 old-k8s-version-654890 kubelet[666]: E0924 01:15:33.783850 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.699929 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.783545 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.700115 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.784908 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.700443 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.784164 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.702897 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.792225 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.703245 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:14 old-k8s-version-654890 kubelet[666]: E0924 01:16:14.789130 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.703708 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:15 old-k8s-version-654890 kubelet[666]: E0924 01:16:15.580475 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704038 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:20 old-k8s-version-654890 kubelet[666]: E0924 01:16:20.992161 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704222 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:29 old-k8s-version-654890 kubelet[666]: E0924 01:16:29.787709 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.704549 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:34 old-k8s-version-654890 kubelet[666]: E0924 01:16:34.784335 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704734 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:42 old-k8s-version-654890 kubelet[666]: E0924 01:16:42.784427 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.705065 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:45 old-k8s-version-654890 kubelet[666]: E0924 01:16:45.783505 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.705249 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:53 old-k8s-version-654890 kubelet[666]: E0924 01:16:53.784108 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.705575 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:57 old-k8s-version-654890 kubelet[666]: E0924 01:16:57.783801 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.705759 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:06 old-k8s-version-654890 kubelet[666]: E0924 01:17:06.784121 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.706092 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:11 old-k8s-version-654890 kubelet[666]: E0924 01:17:11.783481 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.706279 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:18 old-k8s-version-654890 kubelet[666]: E0924 01:17:18.783911 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.706609 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:24 old-k8s-version-654890 kubelet[666]: E0924 01:17:24.784083 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.709045 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:31 old-k8s-version-654890 kubelet[666]: E0924 01:17:31.792392 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.709642 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:39 old-k8s-version-654890 kubelet[666]: E0924 01:17:39.856284 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.709971 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:40 old-k8s-version-654890 kubelet[666]: E0924 01:17:40.985923 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.710166 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:45 old-k8s-version-654890 kubelet[666]: E0924 01:17:45.784024 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.710493 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:55 old-k8s-version-654890 kubelet[666]: E0924 01:17:55.783900 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.710677 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:59 old-k8s-version-654890 kubelet[666]: E0924 01:17:59.784186 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.711009 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:06 old-k8s-version-654890 kubelet[666]: E0924 01:18:06.788365 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.711196 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:12 old-k8s-version-654890 kubelet[666]: E0924 01:18:12.783859 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.711569 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:18 old-k8s-version-654890 kubelet[666]: E0924 01:18:18.783560 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.711755 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:27 old-k8s-version-654890 kubelet[666]: E0924 01:18:27.783904 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.712084 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:30 old-k8s-version-654890 kubelet[666]: E0924 01:18:30.783887 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.712268 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:41 old-k8s-version-654890 kubelet[666]: E0924 01:18:41.784268 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.712597 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: E0924 01:18:44.783947 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.712784 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:55 old-k8s-version-654890 kubelet[666]: E0924 01:18:55.783966 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.713114 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: E0924 01:18:58.784580 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.713298 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:09 old-k8s-version-654890 kubelet[666]: E0924 01:19:09.783890 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.713628 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: E0924 01:19:11.783671 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.713812 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.714144 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.714487 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.714673 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.714857 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.715195 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
I0924 01:19:59.715208 503471 logs.go:123] Gathering logs for kube-apiserver [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f] ...
I0924 01:19:59.715224 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:59.775878 503471 logs.go:123] Gathering logs for kube-apiserver [a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c] ...
I0924 01:19:59.775913 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:59.848947 503471 logs.go:123] Gathering logs for coredns [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9] ...
I0924 01:19:59.848982 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:59.893787 503471 logs.go:123] Gathering logs for kube-scheduler [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57] ...
I0924 01:19:59.893817 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:59.934822 503471 logs.go:123] Gathering logs for kube-proxy [a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c] ...
I0924 01:19:59.934854 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:59.975700 503471 logs.go:123] Gathering logs for kube-controller-manager [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b] ...
I0924 01:19:59.975727 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:20:00.150686 503471 logs.go:123] Gathering logs for containerd ...
I0924 01:20:00.150774 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:20:00.348822 503471 logs.go:123] Gathering logs for dmesg ...
I0924 01:20:00.348953 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:20:00.421331 503471 logs.go:123] Gathering logs for describe nodes ...
I0924 01:20:00.421432 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:20:00.860083 503471 logs.go:123] Gathering logs for etcd [4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2] ...
I0924 01:20:00.860117 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:20:00.933708 503471 logs.go:123] Gathering logs for kubernetes-dashboard [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63] ...
I0924 01:20:00.933741 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:20:00.983639 503471 logs.go:123] Gathering logs for container status ...
I0924 01:20:00.983672 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:20:01.031385 503471 logs.go:123] Gathering logs for kube-controller-manager [840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55] ...
I0924 01:20:01.031544 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:20:01.119721 503471 logs.go:123] Gathering logs for kindnet [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335] ...
I0924 01:20:01.119762 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:20:01.189736 503471 logs.go:123] Gathering logs for storage-provisioner [fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d] ...
I0924 01:20:01.189781 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:20:01.236887 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:20:01.236929 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:20:01.236990 503471 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0924 01:20:01.237011 503471 out.go:270] Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:20:01.237022 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:20:01.237031 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:20:01.237044 503471 out.go:270] Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:20:01.237050 503471 out.go:270] Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
I0924 01:20:01.237057 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:20:01.237068 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:20:11.238141 503471 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0924 01:20:11.249366 503471 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0924 01:20:11.251766 503471 out.go:201]
W0924 01:20:11.253416 503471 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0924 01:20:11.253457 503471 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0924 01:20:11.253478 503471 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0924 01:20:11.253487 503471 out.go:270] *
*
W0924 01:20:11.254517 503471 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0924 01:20:11.256620 503471 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-654890 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-654890
helpers_test.go:235: (dbg) docker inspect old-k8s-version-654890:
-- stdout --
[
{
"Id": "da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5",
"Created": "2024-09-24T01:10:36.906966417Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 503665,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-09-24T01:13:54.628022762Z",
"FinishedAt": "2024-09-24T01:13:53.610212588Z"
},
"Image": "sha256:62002f6a97ad1f6cd4117c29b1c488a6bf3b6255c8231f0d600b1bc7ba1bcfd6",
"ResolvConfPath": "/var/lib/docker/containers/da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5/hostname",
"HostsPath": "/var/lib/docker/containers/da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5/hosts",
"LogPath": "/var/lib/docker/containers/da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5/da622e672fcf5cf588c49527db1e3748578c64f06c54c05957df1fb2a8c5aee5-json.log",
"Name": "/old-k8s-version-654890",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-654890:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-654890",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/20df8a8b727b9375e61e46d086c438e7862ee0d73e487486c0c1f794393c3059-init/diff:/var/lib/docker/overlay2/7ad1ac86d8d84caef983ee398d28a66996d884096876cd745ca39b66abf10752/diff",
"MergedDir": "/var/lib/docker/overlay2/20df8a8b727b9375e61e46d086c438e7862ee0d73e487486c0c1f794393c3059/merged",
"UpperDir": "/var/lib/docker/overlay2/20df8a8b727b9375e61e46d086c438e7862ee0d73e487486c0c1f794393c3059/diff",
"WorkDir": "/var/lib/docker/overlay2/20df8a8b727b9375e61e46d086c438e7862ee0d73e487486c0c1f794393c3059/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-654890",
"Source": "/var/lib/docker/volumes/old-k8s-version-654890/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-654890",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-654890",
"name.minikube.sigs.k8s.io": "old-k8s-version-654890",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "e05c30016d00e3d42c9b9d7b22fa78cf703a8009d523f5f936437bbed520dcb0",
"SandboxKey": "/var/run/docker/netns/e05c30016d00",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33435"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33436"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33439"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33437"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33438"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-654890": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "63ec0b812ffc8f6f22c1ba958f67fb90363268d030eeb4421b582536774bee5a",
"EndpointID": "e7930192aacfde574dcf8e6ee8d40306a8904afb51403138a46d67eca5fe98f2",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-654890",
"da622e672fcf"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-654890 -n old-k8s-version-654890
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-654890 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-654890 logs -n 25: (2.889601066s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| pause | -p pause-005476 | pause-005476 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:09 UTC |
| | --alsologtostderr -v=5 | | | | | |
| unpause | -p pause-005476 | pause-005476 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:09 UTC |
| | --alsologtostderr -v=5 | | | | | |
| pause | -p pause-005476 | pause-005476 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:09 UTC |
| | --alsologtostderr -v=5 | | | | | |
| delete | -p pause-005476 | pause-005476 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:09 UTC |
| | --alsologtostderr -v=5 | | | | | |
| delete | -p pause-005476 | pause-005476 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:09 UTC |
| start | -p cert-options-649069 | cert-options-649069 | jenkins | v1.34.0 | 24 Sep 24 01:09 UTC | 24 Sep 24 01:10 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-649069 ssh | cert-options-649069 | jenkins | v1.34.0 | 24 Sep 24 01:10 UTC | 24 Sep 24 01:10 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-649069 -- sudo | cert-options-649069 | jenkins | v1.34.0 | 24 Sep 24 01:10 UTC | 24 Sep 24 01:10 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-649069 | cert-options-649069 | jenkins | v1.34.0 | 24 Sep 24 01:10 UTC | 24 Sep 24 01:10 UTC |
| start | -p old-k8s-version-654890 | old-k8s-version-654890 | jenkins | v1.34.0 | 24 Sep 24 01:10 UTC | 24 Sep 24 01:13 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-136100 | cert-expiration-136100 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:13 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-136100 | cert-expiration-136100 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:13 UTC |
| start | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:14 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-654890 | old-k8s-version-654890 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:13 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-654890 | old-k8s-version-654890 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:13 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-654890 | old-k8s-version-654890 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | 24 Sep 24 01:13 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-654890 | old-k8s-version-654890 | jenkins | v1.34.0 | 24 Sep 24 01:13 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:14 UTC | 24 Sep 24 01:14 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:14 UTC | 24 Sep 24 01:15 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:15 UTC | 24 Sep 24 01:15 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:15 UTC | 24 Sep 24 01:19 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| image | no-preload-558135 image list | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:20 UTC | 24 Sep 24 01:20 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:20 UTC | 24 Sep 24 01:20 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:20 UTC | 24 Sep 24 01:20 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-558135 | no-preload-558135 | jenkins | v1.34.0 | 24 Sep 24 01:20 UTC | |
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/24 01:15:05
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.23.0 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0924 01:15:05.131324 508400 out.go:345] Setting OutFile to fd 1 ...
I0924 01:15:05.131506 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:15:05.131561 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:15:05.131584 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:15:05.131888 508400 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19696-296322/.minikube/bin
I0924 01:15:05.132306 508400 out.go:352] Setting JSON to false
I0924 01:15:05.133461 508400 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":10651,"bootTime":1727129855,"procs":219,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0924 01:15:05.133577 508400 start.go:139] virtualization:
I0924 01:15:05.135985 508400 out.go:177] * [no-preload-558135] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0924 01:15:05.139120 508400 out.go:177] - MINIKUBE_LOCATION=19696
I0924 01:15:05.139217 508400 notify.go:220] Checking for updates...
I0924 01:15:05.142883 508400 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0924 01:15:05.144600 508400 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:15:05.146569 508400 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19696-296322/.minikube
I0924 01:15:05.148223 508400 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0924 01:15:05.150213 508400 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0924 01:15:05.152393 508400 config.go:182] Loaded profile config "no-preload-558135": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0924 01:15:05.152985 508400 driver.go:394] Setting default libvirt URI to qemu:///system
I0924 01:15:05.192354 508400 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0924 01:15:05.192491 508400 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0924 01:15:05.253848 508400 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:36 OomKillDisable:true NGoroutines:52 SystemTime:2024-09-24 01:15:05.242858864 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0924 01:15:05.253962 508400 docker.go:318] overlay module found
I0924 01:15:05.256550 508400 out.go:177] * Using the docker driver based on existing profile
I0924 01:15:05.258872 508400 start.go:297] selected driver: docker
I0924 01:15:05.258891 508400 start.go:901] validating driver "docker" against &{Name:no-preload-558135 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-558135 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false Mount
String:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:15:05.259076 508400 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0924 01:15:05.259794 508400 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0924 01:15:05.308138 508400 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:36 OomKillDisable:true NGoroutines:52 SystemTime:2024-09-24 01:15:05.299019674 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0924 01:15:05.308517 508400 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0924 01:15:05.308548 508400 cni.go:84] Creating CNI manager for ""
I0924 01:15:05.308590 508400 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0924 01:15:05.308646 508400 start.go:340] cluster config:
{Name:no-preload-558135 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-558135 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker Moun
tIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:15:05.311569 508400 out.go:177] * Starting "no-preload-558135" primary control-plane node in "no-preload-558135" cluster
I0924 01:15:05.313629 508400 cache.go:121] Beginning downloading kic base image for docker with containerd
I0924 01:15:05.315880 508400 out.go:177] * Pulling base image v0.0.45-1727108449-19696 ...
I0924 01:15:05.317647 508400 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0924 01:15:05.317742 508400 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local docker daemon
I0924 01:15:05.317840 508400 profile.go:143] Saving config to /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/config.json ...
I0924 01:15:05.318231 508400 cache.go:107] acquiring lock: {Name:mk398fc6f821486820763f92f690857ee3a862a1 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318342 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0924 01:15:05.318354 508400 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 142.236µs
I0924 01:15:05.318368 508400 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0924 01:15:05.318386 508400 cache.go:107] acquiring lock: {Name:mk564505433d8160c6cf8d9034c077c3eb692643 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318399 508400 cache.go:107] acquiring lock: {Name:mk9958e93ea76798dc57faa97724f0d182438d07 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318428 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0924 01:15:05.318434 508400 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 49.412µs
I0924 01:15:05.318440 508400 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0924 01:15:05.318461 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 exists
I0924 01:15:05.318453 508400 cache.go:107] acquiring lock: {Name:mk8cd370c169857c73fddb24a05ce8c68e33a758 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318470 508400 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.31.1" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1" took 79.393µs
I0924 01:15:05.318478 508400 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.31.1 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 succeeded
I0924 01:15:05.318490 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 exists
I0924 01:15:05.318489 508400 cache.go:107] acquiring lock: {Name:mkec047b96edc009c73a2992a5921497c2568ff3 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318499 508400 cache.go:96] cache image "registry.k8s.io/etcd:3.5.15-0" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0" took 46.81µs
I0924 01:15:05.318506 508400 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.15-0 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 succeeded
I0924 01:15:05.318514 508400 cache.go:107] acquiring lock: {Name:mk59eeb3e7494fb41bb2b8f96dd347829135888a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318534 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 exists
I0924 01:15:05.318540 508400 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.31.1" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1" took 53.022µs
I0924 01:15:05.318546 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 exists
I0924 01:15:05.318546 508400 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.31.1 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 succeeded
I0924 01:15:05.318552 508400 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.3" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3" took 38.31µs
I0924 01:15:05.318560 508400 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.3 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 succeeded
I0924 01:15:05.318561 508400 cache.go:107] acquiring lock: {Name:mkeec81a65d2cbb29f6fa6cf5445016cf9d93eeb Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318579 508400 cache.go:107] acquiring lock: {Name:mk0fe11db21e83645ea1593e52f6cd6eba0dcf3b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.318592 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 exists
I0924 01:15:05.318598 508400 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.31.1" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1" took 38.901µs
I0924 01:15:05.318604 508400 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.31.1 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 succeeded
I0924 01:15:05.318649 508400 cache.go:115] /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 exists
I0924 01:15:05.318660 508400 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.31.1" -> "/home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1" took 85.112µs
I0924 01:15:05.318668 508400 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.31.1 -> /home/jenkins/minikube-integration/19696-296322/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 succeeded
I0924 01:15:05.318678 508400 cache.go:87] Successfully saved all images to host disk.
I0924 01:15:05.337850 508400 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 in local docker daemon, skipping pull
I0924 01:15:05.337874 508400 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 exists in daemon, skipping load
I0924 01:15:05.337895 508400 cache.go:194] Successfully downloaded all kic artifacts
I0924 01:15:05.337921 508400 start.go:360] acquireMachinesLock for no-preload-558135: {Name:mkc2977897c44e467ec83444dc746b792fae07d9 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0924 01:15:05.337981 508400 start.go:364] duration metric: took 39.31µs to acquireMachinesLock for "no-preload-558135"
I0924 01:15:05.338006 508400 start.go:96] Skipping create...Using existing machine configuration
I0924 01:15:05.338015 508400 fix.go:54] fixHost starting:
I0924 01:15:05.338285 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:05.358943 508400 fix.go:112] recreateIfNeeded on no-preload-558135: state=Stopped err=<nil>
W0924 01:15:05.358983 508400 fix.go:138] unexpected machine state, will restart: <nil>
I0924 01:15:05.361525 508400 out.go:177] * Restarting existing docker container for "no-preload-558135" ...
I0924 01:15:05.180516 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:07.675923 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:05.363889 508400 cli_runner.go:164] Run: docker start no-preload-558135
I0924 01:15:05.736516 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:05.757501 508400 kic.go:430] container "no-preload-558135" state is running.
I0924 01:15:05.757890 508400 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-558135
I0924 01:15:05.778260 508400 profile.go:143] Saving config to /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/config.json ...
I0924 01:15:05.778488 508400 machine.go:93] provisionDockerMachine start ...
I0924 01:15:05.778554 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:05.802347 508400 main.go:141] libmachine: Using SSH client type: native
I0924 01:15:05.802607 508400 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33440 <nil> <nil>}
I0924 01:15:05.802617 508400 main.go:141] libmachine: About to run SSH command:
hostname
I0924 01:15:05.804594 508400 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0924 01:15:08.938826 508400 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-558135
I0924 01:15:08.938852 508400 ubuntu.go:169] provisioning hostname "no-preload-558135"
I0924 01:15:08.938996 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:08.958136 508400 main.go:141] libmachine: Using SSH client type: native
I0924 01:15:08.958387 508400 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33440 <nil> <nil>}
I0924 01:15:08.958405 508400 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-558135 && echo "no-preload-558135" | sudo tee /etc/hostname
I0924 01:15:09.105960 508400 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-558135
I0924 01:15:09.106090 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:09.128028 508400 main.go:141] libmachine: Using SSH client type: native
I0924 01:15:09.128267 508400 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 33440 <nil> <nil>}
I0924 01:15:09.128291 508400 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-558135' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-558135/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-558135' | sudo tee -a /etc/hosts;
fi
fi
I0924 01:15:09.266951 508400 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0924 01:15:09.266978 508400 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19696-296322/.minikube CaCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19696-296322/.minikube}
I0924 01:15:09.267017 508400 ubuntu.go:177] setting up certificates
I0924 01:15:09.267033 508400 provision.go:84] configureAuth start
I0924 01:15:09.267099 508400 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-558135
I0924 01:15:09.284152 508400 provision.go:143] copyHostCerts
I0924 01:15:09.284227 508400 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem, removing ...
I0924 01:15:09.284238 508400 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem
I0924 01:15:09.284316 508400 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/ca.pem (1078 bytes)
I0924 01:15:09.284432 508400 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem, removing ...
I0924 01:15:09.284445 508400 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem
I0924 01:15:09.284474 508400 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/cert.pem (1123 bytes)
I0924 01:15:09.284556 508400 exec_runner.go:144] found /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem, removing ...
I0924 01:15:09.284565 508400 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem
I0924 01:15:09.284591 508400 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19696-296322/.minikube/key.pem (1675 bytes)
I0924 01:15:09.284734 508400 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem org=jenkins.no-preload-558135 san=[127.0.0.1 192.168.85.2 localhost minikube no-preload-558135]
I0924 01:15:10.018175 508400 provision.go:177] copyRemoteCerts
I0924 01:15:10.019485 508400 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0924 01:15:10.019602 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:10.041738 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:10.140907 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0924 01:15:10.178786 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0924 01:15:10.212058 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0924 01:15:10.237740 508400 provision.go:87] duration metric: took 970.69088ms to configureAuth
I0924 01:15:10.237766 508400 ubuntu.go:193] setting minikube options for container-runtime
I0924 01:15:10.238000 508400 config.go:182] Loaded profile config "no-preload-558135": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0924 01:15:10.238015 508400 machine.go:96] duration metric: took 4.459517521s to provisionDockerMachine
I0924 01:15:10.238025 508400 start.go:293] postStartSetup for "no-preload-558135" (driver="docker")
I0924 01:15:10.238041 508400 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0924 01:15:10.238102 508400 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0924 01:15:10.238151 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:10.255479 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:10.352314 508400 ssh_runner.go:195] Run: cat /etc/os-release
I0924 01:15:10.355654 508400 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0924 01:15:10.355690 508400 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0924 01:15:10.355701 508400 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0924 01:15:10.355708 508400 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0924 01:15:10.355719 508400 filesync.go:126] Scanning /home/jenkins/minikube-integration/19696-296322/.minikube/addons for local assets ...
I0924 01:15:10.355786 508400 filesync.go:126] Scanning /home/jenkins/minikube-integration/19696-296322/.minikube/files for local assets ...
I0924 01:15:10.355887 508400 filesync.go:149] local asset: /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem -> 3017112.pem in /etc/ssl/certs
I0924 01:15:10.355996 508400 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0924 01:15:10.365118 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem --> /etc/ssl/certs/3017112.pem (1708 bytes)
I0924 01:15:10.389873 508400 start.go:296] duration metric: took 151.826309ms for postStartSetup
I0924 01:15:10.389975 508400 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0924 01:15:10.390018 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:10.409667 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:10.500566 508400 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0924 01:15:10.505315 508400 fix.go:56] duration metric: took 5.167290636s for fixHost
I0924 01:15:10.505341 508400 start.go:83] releasing machines lock for "no-preload-558135", held for 5.167346874s
I0924 01:15:10.505424 508400 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-558135
I0924 01:15:10.521705 508400 ssh_runner.go:195] Run: cat /version.json
I0924 01:15:10.521764 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:10.521997 508400 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0924 01:15:10.522062 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:10.540019 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:10.552549 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:10.790515 508400 ssh_runner.go:195] Run: systemctl --version
I0924 01:15:10.796651 508400 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0924 01:15:10.803677 508400 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0924 01:15:10.844599 508400 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0924 01:15:10.844847 508400 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0924 01:15:10.861488 508400 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0924 01:15:10.861600 508400 start.go:495] detecting cgroup driver to use...
I0924 01:15:10.861696 508400 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0924 01:15:10.861795 508400 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0924 01:15:10.887174 508400 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0924 01:15:10.907361 508400 docker.go:217] disabling cri-docker service (if available) ...
I0924 01:15:10.907561 508400 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0924 01:15:10.929651 508400 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0924 01:15:10.947799 508400 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0924 01:15:11.059808 508400 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0924 01:15:11.162718 508400 docker.go:233] disabling docker service ...
I0924 01:15:11.162800 508400 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0924 01:15:11.181938 508400 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0924 01:15:11.195555 508400 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0924 01:15:11.287672 508400 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0924 01:15:11.393952 508400 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0924 01:15:11.408889 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0924 01:15:11.427716 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0924 01:15:11.442786 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0924 01:15:11.455172 508400 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0924 01:15:11.455251 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0924 01:15:11.467063 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0924 01:15:11.477798 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0924 01:15:11.489050 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0924 01:15:11.500803 508400 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0924 01:15:11.510659 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0924 01:15:11.521564 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0924 01:15:11.532878 508400 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0924 01:15:11.543600 508400 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0924 01:15:11.554407 508400 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0924 01:15:11.563320 508400 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:15:11.654842 508400 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0924 01:15:11.823903 508400 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0924 01:15:11.824023 508400 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0924 01:15:11.827847 508400 start.go:563] Will wait 60s for crictl version
I0924 01:15:11.827952 508400 ssh_runner.go:195] Run: which crictl
I0924 01:15:11.832381 508400 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0924 01:15:11.874255 508400 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0924 01:15:11.874357 508400 ssh_runner.go:195] Run: containerd --version
I0924 01:15:11.897928 508400 ssh_runner.go:195] Run: containerd --version
I0924 01:15:11.928649 508400 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I0924 01:15:11.930268 508400 cli_runner.go:164] Run: docker network inspect no-preload-558135 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0924 01:15:11.946880 508400 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0924 01:15:11.950892 508400 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0924 01:15:11.962490 508400 kubeadm.go:883] updating cluster {Name:no-preload-558135 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-558135 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenk
ins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0924 01:15:11.962611 508400 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0924 01:15:11.962665 508400 ssh_runner.go:195] Run: sudo crictl images --output json
I0924 01:15:12.017958 508400 containerd.go:627] all images are preloaded for containerd runtime.
I0924 01:15:12.017983 508400 cache_images.go:84] Images are preloaded, skipping loading
I0924 01:15:12.017992 508400 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.31.1 containerd true true} ...
I0924 01:15:12.018118 508400 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=no-preload-558135 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:no-preload-558135 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0924 01:15:12.018196 508400 ssh_runner.go:195] Run: sudo crictl info
I0924 01:15:12.062391 508400 cni.go:84] Creating CNI manager for ""
I0924 01:15:12.062420 508400 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0924 01:15:12.062432 508400 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0924 01:15:12.062457 508400 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-558135 NodeName:no-preload-558135 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP
ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0924 01:15:12.062592 508400 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "no-preload-558135"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0924 01:15:12.062669 508400 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0924 01:15:12.073705 508400 binaries.go:44] Found k8s binaries, skipping transfer
I0924 01:15:12.073812 508400 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0924 01:15:12.083842 508400 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (321 bytes)
I0924 01:15:12.105532 508400 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0924 01:15:12.126359 508400 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2171 bytes)
I0924 01:15:12.146476 508400 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0924 01:15:12.150008 508400 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0924 01:15:12.161522 508400 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:15:12.254881 508400 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0924 01:15:12.270877 508400 certs.go:68] Setting up /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135 for IP: 192.168.85.2
I0924 01:15:12.271004 508400 certs.go:194] generating shared ca certs ...
I0924 01:15:12.271035 508400 certs.go:226] acquiring lock for ca certs: {Name:mk4a6ab65221805436b06c42ec4fde316fe470ee Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:15:12.271211 508400 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19696-296322/.minikube/ca.key
I0924 01:15:12.271285 508400 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.key
I0924 01:15:12.271317 508400 certs.go:256] generating profile certs ...
I0924 01:15:12.271450 508400 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/client.key
I0924 01:15:12.271558 508400 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/apiserver.key.75068203
I0924 01:15:12.271647 508400 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/proxy-client.key
I0924 01:15:12.271809 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711.pem (1338 bytes)
W0924 01:15:12.271862 508400 certs.go:480] ignoring /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711_empty.pem, impossibly tiny 0 bytes
I0924 01:15:12.271894 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca-key.pem (1675 bytes)
I0924 01:15:12.271935 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/ca.pem (1078 bytes)
I0924 01:15:12.271993 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/cert.pem (1123 bytes)
I0924 01:15:12.272054 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/certs/key.pem (1675 bytes)
I0924 01:15:12.272128 508400 certs.go:484] found cert: /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem (1708 bytes)
I0924 01:15:12.272933 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0924 01:15:12.298049 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0924 01:15:12.323028 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0924 01:15:12.348775 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0924 01:15:12.374146 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0924 01:15:12.404654 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0924 01:15:12.436704 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0924 01:15:12.491604 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/profiles/no-preload-558135/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0924 01:15:12.548152 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0924 01:15:12.577121 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/certs/301711.pem --> /usr/share/ca-certificates/301711.pem (1338 bytes)
I0924 01:15:12.602655 508400 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19696-296322/.minikube/files/etc/ssl/certs/3017112.pem --> /usr/share/ca-certificates/3017112.pem (1708 bytes)
I0924 01:15:12.649693 508400 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0924 01:15:12.668885 508400 ssh_runner.go:195] Run: openssl version
I0924 01:15:12.677035 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0924 01:15:12.688416 508400 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0924 01:15:12.692149 508400 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 24 00:24 /usr/share/ca-certificates/minikubeCA.pem
I0924 01:15:12.692236 508400 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0924 01:15:12.699183 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0924 01:15:12.708539 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/301711.pem && ln -fs /usr/share/ca-certificates/301711.pem /etc/ssl/certs/301711.pem"
I0924 01:15:12.720284 508400 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/301711.pem
I0924 01:15:12.724531 508400 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 24 00:35 /usr/share/ca-certificates/301711.pem
I0924 01:15:12.724630 508400 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/301711.pem
I0924 01:15:12.732015 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/301711.pem /etc/ssl/certs/51391683.0"
I0924 01:15:12.741377 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/3017112.pem && ln -fs /usr/share/ca-certificates/3017112.pem /etc/ssl/certs/3017112.pem"
I0924 01:15:12.751653 508400 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/3017112.pem
I0924 01:15:12.755496 508400 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 24 00:35 /usr/share/ca-certificates/3017112.pem
I0924 01:15:12.755622 508400 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/3017112.pem
I0924 01:15:12.762797 508400 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/3017112.pem /etc/ssl/certs/3ec20f2e.0"
I0924 01:15:12.772068 508400 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0924 01:15:12.776077 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0924 01:15:12.787317 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0924 01:15:12.795166 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0924 01:15:12.804311 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0924 01:15:12.814561 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0924 01:15:12.822597 508400 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0924 01:15:12.831931 508400 kubeadm.go:392] StartCluster: {Name:no-preload-558135 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1727108449-19696@sha256:c662152d8855bc4c62a3b5786a68adf99e04794e7f8f374a3859703004ef1d21 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-558135 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0924 01:15:12.832042 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0924 01:15:12.832109 508400 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0924 01:15:12.875453 508400 cri.go:89] found id: "898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:15:12.875521 508400 cri.go:89] found id: "a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:15:12.875542 508400 cri.go:89] found id: "e70ec5008d041010bfaf45bf28c1a2e1aafc37e8bd46d9aec69986eb01bd4376"
I0924 01:15:12.875562 508400 cri.go:89] found id: "9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:15:12.875594 508400 cri.go:89] found id: "49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:15:12.875600 508400 cri.go:89] found id: "6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:15:12.875604 508400 cri.go:89] found id: "1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:15:12.875608 508400 cri.go:89] found id: "51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:15:12.875611 508400 cri.go:89] found id: ""
I0924 01:15:12.875679 508400 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0924 01:15:12.888402 508400 cri.go:116] JSON = null
W0924 01:15:12.888452 508400 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0924 01:15:12.888518 508400 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0924 01:15:12.897670 508400 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0924 01:15:12.897731 508400 kubeadm.go:593] restartPrimaryControlPlane start ...
I0924 01:15:12.897808 508400 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0924 01:15:12.907186 508400 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0924 01:15:12.907817 508400 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-558135" does not appear in /home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:15:12.908072 508400 kubeconfig.go:62] /home/jenkins/minikube-integration/19696-296322/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-558135" cluster setting kubeconfig missing "no-preload-558135" context setting]
I0924 01:15:12.908534 508400 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19696-296322/kubeconfig: {Name:mk12cf5f8c4244466c827b22ce4fe2341553290d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:15:12.910635 508400 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0924 01:15:12.920473 508400 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0924 01:15:12.920507 508400 kubeadm.go:597] duration metric: took 22.755876ms to restartPrimaryControlPlane
I0924 01:15:12.920517 508400 kubeadm.go:394] duration metric: took 88.595128ms to StartCluster
I0924 01:15:12.920532 508400 settings.go:142] acquiring lock: {Name:mk1b01c5281da0b61714a1aa76e5632af5b39da5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:15:12.920616 508400 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19696-296322/kubeconfig
I0924 01:15:12.921565 508400 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19696-296322/kubeconfig: {Name:mk12cf5f8c4244466c827b22ce4fe2341553290d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0924 01:15:12.921789 508400 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0924 01:15:12.922084 508400 config.go:182] Loaded profile config "no-preload-558135": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0924 01:15:12.922131 508400 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0924 01:15:12.922212 508400 addons.go:69] Setting storage-provisioner=true in profile "no-preload-558135"
I0924 01:15:12.922231 508400 addons.go:234] Setting addon storage-provisioner=true in "no-preload-558135"
W0924 01:15:12.922237 508400 addons.go:243] addon storage-provisioner should already be in state true
I0924 01:15:12.922267 508400 host.go:66] Checking if "no-preload-558135" exists ...
I0924 01:15:12.922313 508400 addons.go:69] Setting default-storageclass=true in profile "no-preload-558135"
I0924 01:15:12.922334 508400 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-558135"
I0924 01:15:12.922637 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:12.922943 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:12.923205 508400 addons.go:69] Setting dashboard=true in profile "no-preload-558135"
I0924 01:15:12.923226 508400 addons.go:234] Setting addon dashboard=true in "no-preload-558135"
W0924 01:15:12.923233 508400 addons.go:243] addon dashboard should already be in state true
I0924 01:15:12.923259 508400 host.go:66] Checking if "no-preload-558135" exists ...
I0924 01:15:12.923684 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:12.926046 508400 out.go:177] * Verifying Kubernetes components...
I0924 01:15:12.926415 508400 addons.go:69] Setting metrics-server=true in profile "no-preload-558135"
I0924 01:15:12.926473 508400 addons.go:234] Setting addon metrics-server=true in "no-preload-558135"
W0924 01:15:12.926496 508400 addons.go:243] addon metrics-server should already be in state true
I0924 01:15:12.926558 508400 host.go:66] Checking if "no-preload-558135" exists ...
I0924 01:15:12.927313 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:12.928647 508400 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0924 01:15:12.964848 508400 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0924 01:15:12.966551 508400 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:15:12.966579 508400 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0924 01:15:12.966644 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:12.995962 508400 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0924 01:15:12.998979 508400 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0924 01:15:12.999076 508400 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0924 01:15:12.999168 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:13.011136 508400 addons.go:234] Setting addon default-storageclass=true in "no-preload-558135"
W0924 01:15:13.011165 508400 addons.go:243] addon default-storageclass should already be in state true
I0924 01:15:13.011193 508400 host.go:66] Checking if "no-preload-558135" exists ...
I0924 01:15:13.011627 508400 cli_runner.go:164] Run: docker container inspect no-preload-558135 --format={{.State.Status}}
I0924 01:15:13.040532 508400 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0924 01:15:13.042583 508400 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0924 01:15:09.676736 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:12.177175 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:14.177211 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:13.044299 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0924 01:15:13.044326 508400 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0924 01:15:13.044402 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:13.054370 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:13.063951 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:13.082463 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:13.084948 508400 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0924 01:15:13.084968 508400 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0924 01:15:13.085036 508400 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-558135
I0924 01:15:13.127082 508400 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33440 SSHKeyPath:/home/jenkins/minikube-integration/19696-296322/.minikube/machines/no-preload-558135/id_rsa Username:docker}
I0924 01:15:13.143073 508400 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0924 01:15:13.223155 508400 node_ready.go:35] waiting up to 6m0s for node "no-preload-558135" to be "Ready" ...
I0924 01:15:13.247833 508400 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0924 01:15:13.247908 508400 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0924 01:15:13.316283 508400 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0924 01:15:13.316369 508400 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0924 01:15:13.350751 508400 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0924 01:15:13.350774 508400 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0924 01:15:13.361840 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0924 01:15:13.432285 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:15:13.460305 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0924 01:15:13.460381 508400 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0924 01:15:13.521063 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0924 01:15:13.538871 508400 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.539034 508400 retry.go:31] will retry after 322.555897ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.589463 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0924 01:15:13.589543 508400 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0924 01:15:13.664940 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0924 01:15:13.664978 508400 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
W0924 01:15:13.768937 508400 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.768964 508400 retry.go:31] will retry after 315.177653ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.834994 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0924 01:15:13.835015 508400 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0924 01:15:13.862460 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0924 01:15:13.904352 508400 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.904388 508400 retry.go:31] will retry after 330.235998ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0924 01:15:13.926984 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0924 01:15:13.927012 508400 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0924 01:15:14.073525 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0924 01:15:14.073551 508400 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0924 01:15:14.084785 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0924 01:15:14.137007 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0924 01:15:14.137032 508400 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0924 01:15:14.234794 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0924 01:15:14.250191 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0924 01:15:14.250218 508400 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0924 01:15:14.396589 508400 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:15:14.396616 508400 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0924 01:15:14.531117 508400 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0924 01:15:16.182786 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:18.677465 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:17.871041 508400 node_ready.go:49] node "no-preload-558135" has status "Ready":"True"
I0924 01:15:17.871071 508400 node_ready.go:38] duration metric: took 4.647836834s for node "no-preload-558135" to be "Ready" ...
I0924 01:15:17.871081 508400 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0924 01:15:17.892084 508400 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-rq7m2" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.907548 508400 pod_ready.go:93] pod "coredns-7c65d6cfc9-rq7m2" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:17.907579 508400 pod_ready.go:82] duration metric: took 15.455843ms for pod "coredns-7c65d6cfc9-rq7m2" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.907592 508400 pod_ready.go:79] waiting up to 6m0s for pod "etcd-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.920863 508400 pod_ready.go:93] pod "etcd-no-preload-558135" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:17.920889 508400 pod_ready.go:82] duration metric: took 13.28849ms for pod "etcd-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.920907 508400 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.931582 508400 pod_ready.go:93] pod "kube-apiserver-no-preload-558135" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:17.931607 508400 pod_ready.go:82] duration metric: took 10.692551ms for pod "kube-apiserver-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.931620 508400 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.942482 508400 pod_ready.go:93] pod "kube-controller-manager-no-preload-558135" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:17.942508 508400 pod_ready.go:82] duration metric: took 10.880957ms for pod "kube-controller-manager-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:17.942522 508400 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-krnb9" in "kube-system" namespace to be "Ready" ...
I0924 01:15:18.076063 508400 pod_ready.go:93] pod "kube-proxy-krnb9" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:18.076092 508400 pod_ready.go:82] duration metric: took 133.561749ms for pod "kube-proxy-krnb9" in "kube-system" namespace to be "Ready" ...
I0924 01:15:18.076106 508400 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:18.132262 508400 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (4.269764527s)
I0924 01:15:18.479635 508400 pod_ready.go:93] pod "kube-scheduler-no-preload-558135" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:18.479664 508400 pod_ready.go:82] duration metric: took 403.550305ms for pod "kube-scheduler-no-preload-558135" in "kube-system" namespace to be "Ready" ...
I0924 01:15:18.479676 508400 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace to be "Ready" ...
I0924 01:15:20.492005 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:21.281804 508400 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (7.196976401s)
I0924 01:15:21.350090 508400 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.115255399s)
I0924 01:15:21.350125 508400 addons.go:475] Verifying addon metrics-server=true in "no-preload-558135"
I0924 01:15:21.748451 508400 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.217288701s)
I0924 01:15:21.750205 508400 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-558135 addons enable metrics-server
I0924 01:15:21.751913 508400 out.go:177] * Enabled addons: default-storageclass, storage-provisioner, metrics-server, dashboard
I0924 01:15:20.677676 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:23.177277 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:21.754200 508400 addons.go:510] duration metric: took 8.83205871s for enable addons: enabled=[default-storageclass storage-provisioner metrics-server dashboard]
I0924 01:15:22.987357 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:25.178088 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:27.678473 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:25.492114 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:27.992267 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:30.178376 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:32.713650 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:30.487533 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:32.987567 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:35.177286 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:37.177700 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:35.486508 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:37.491829 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:39.987739 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:39.675978 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:41.681287 503471 pod_ready.go:103] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:43.177206 503471 pod_ready.go:93] pod "etcd-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:43.177231 503471 pod_ready.go:82] duration metric: took 1m20.506853276s for pod "etcd-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.177248 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.182450 503471 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:43.182476 503471 pod_ready.go:82] duration metric: took 5.21957ms for pod "kube-apiserver-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:43.182507 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:42.485670 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:44.987149 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:45.191626 503471 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:47.190352 503471 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.190379 503471 pod_ready.go:82] duration metric: took 4.00786197s for pod "kube-controller-manager-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.190392 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-dctnp" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.195753 503471 pod_ready.go:93] pod "kube-proxy-dctnp" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.195776 503471 pod_ready.go:82] duration metric: took 5.355496ms for pod "kube-proxy-dctnp" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.195787 503471 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.201308 503471 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace has status "Ready":"True"
I0924 01:15:47.201337 503471 pod_ready.go:82] duration metric: took 5.540809ms for pod "kube-scheduler-old-k8s-version-654890" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.201351 503471 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace to be "Ready" ...
I0924 01:15:47.487928 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:49.986122 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:49.208658 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:51.707222 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:53.708077 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:52.485989 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:54.486343 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:56.209641 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:58.707466 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:56.986648 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:15:59.486329 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:00.708419 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:03.207876 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:01.986345 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:04.487715 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:05.208533 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:07.707911 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:06.987377 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:09.486389 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:09.707959 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:12.207936 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:11.986562 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:13.988735 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:14.708194 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:17.208113 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:16.485844 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:18.486353 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:19.208473 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:21.710594 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:20.486455 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:22.487120 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:24.487852 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:24.208195 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:26.209405 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:28.713846 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:26.986761 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:29.486659 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:31.208503 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:33.707734 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:31.988539 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:34.486675 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:35.708337 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:38.207839 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:36.986563 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:39.486197 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:40.213566 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:42.709635 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:41.985760 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:43.985931 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:45.211860 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:47.707193 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:45.986189 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:48.486878 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:49.707944 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:51.710202 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:53.712325 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:50.986005 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:52.986296 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:56.212429 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:58.708373 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:55.485852 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:57.488641 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:16:59.985527 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:01.208889 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:03.227933 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:01.989793 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:03.990725 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:05.709089 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:08.208394 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:06.488664 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:08.986286 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:10.208508 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:12.707726 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:11.485880 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:13.490174 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:15.208067 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:17.208790 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:15.986288 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:17.986364 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:19.209149 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:21.708671 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:20.485872 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:22.486441 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:24.486873 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:24.216251 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:26.707770 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:26.985850 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:29.005750 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:29.208529 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:31.709364 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:33.761534 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:31.486011 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:33.486893 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:36.208565 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:38.707551 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:35.985760 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:37.986094 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:39.986371 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:40.707768 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:42.707814 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:41.987251 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:43.987616 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:44.707848 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:46.708038 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:48.708177 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:46.485566 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:48.486342 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:51.207438 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:53.208301 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:50.985619 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:52.986354 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:55.707839 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:58.208516 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:55.486824 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:57.489396 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:17:59.985641 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:00.235354 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:02.707695 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:01.987756 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:04.486204 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:05.208024 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:07.707243 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:06.486823 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:08.985919 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:09.707408 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:11.707886 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:10.986437 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:13.485686 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:14.207930 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:16.707304 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:15.486824 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:17.487910 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:19.985698 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:19.209677 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:21.708310 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:21.985762 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:23.986376 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:24.207628 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:26.208190 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:28.707753 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:26.485064 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:28.485541 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:31.207884 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:33.208047 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:30.486377 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:32.986106 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:34.987626 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:35.208663 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:37.707832 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:37.488431 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:39.986291 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:40.207918 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:42.209391 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:41.986776 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:44.485574 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:44.710598 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:47.207389 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:46.486185 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:48.486314 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:49.207968 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:51.208089 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:53.215727 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:50.985833 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:52.986186 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:55.707466 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:57.707846 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:55.486493 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:57.487971 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:18:59.985819 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:00.226503 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:02.708170 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:02.485654 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:04.487225 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:05.207792 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:07.208012 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:06.985291 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:08.985854 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:09.208719 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:11.707807 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:13.707883 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:10.986372 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:12.986504 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:16.208372 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:18.208583 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:15.485785 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:17.490242 508400 pod_ready.go:103] pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:18.485829 508400 pod_ready.go:82] duration metric: took 4m0.006138258s for pod "metrics-server-6867b74b74-46xh4" in "kube-system" namespace to be "Ready" ...
E0924 01:19:18.485857 508400 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0924 01:19:18.485867 508400 pod_ready.go:39] duration metric: took 4m0.614774773s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0924 01:19:18.485890 508400 api_server.go:52] waiting for apiserver process to appear ...
I0924 01:19:18.485932 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:18.486000 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:18.529126 508400 cri.go:89] found id: "cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:18.529150 508400 cri.go:89] found id: "49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:18.529155 508400 cri.go:89] found id: ""
I0924 01:19:18.529163 508400 logs.go:276] 2 containers: [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec]
I0924 01:19:18.529221 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.533367 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.537119 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:18.537199 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:18.576445 508400 cri.go:89] found id: "c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:18.576465 508400 cri.go:89] found id: "1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:18.576470 508400 cri.go:89] found id: ""
I0924 01:19:18.576477 508400 logs.go:276] 2 containers: [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b]
I0924 01:19:18.576536 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.580457 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.584094 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:18.584170 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:18.624141 508400 cri.go:89] found id: "34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:18.624167 508400 cri.go:89] found id: "898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:18.624171 508400 cri.go:89] found id: ""
I0924 01:19:18.624178 508400 logs.go:276] 2 containers: [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6]
I0924 01:19:18.624235 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.629332 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.633865 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:18.633952 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:18.678615 508400 cri.go:89] found id: "5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:18.678638 508400 cri.go:89] found id: "6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:18.678645 508400 cri.go:89] found id: ""
I0924 01:19:18.678652 508400 logs.go:276] 2 containers: [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4]
I0924 01:19:18.678718 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.682263 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.685672 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:18.685749 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:18.727441 508400 cri.go:89] found id: "9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:18.727461 508400 cri.go:89] found id: "9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:18.727466 508400 cri.go:89] found id: ""
I0924 01:19:18.727473 508400 logs.go:276] 2 containers: [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9]
I0924 01:19:18.727530 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.731228 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.734832 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:18.735020 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:18.785451 508400 cri.go:89] found id: "23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:18.785475 508400 cri.go:89] found id: "51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:18.785481 508400 cri.go:89] found id: ""
I0924 01:19:18.785488 508400 logs.go:276] 2 containers: [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f]
I0924 01:19:18.785574 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.791026 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.794884 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:18.794998 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:18.835569 508400 cri.go:89] found id: "84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:18.835642 508400 cri.go:89] found id: "a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:18.835661 508400 cri.go:89] found id: ""
I0924 01:19:18.835701 508400 logs.go:276] 2 containers: [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7]
I0924 01:19:18.835774 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.839386 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.843218 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:18.843290 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:18.884277 508400 cri.go:89] found id: "8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:18.884343 508400 cri.go:89] found id: ""
I0924 01:19:18.884367 508400 logs.go:276] 1 containers: [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f]
I0924 01:19:18.884452 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.888639 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:18.888754 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:18.931032 508400 cri.go:89] found id: "e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:18.931103 508400 cri.go:89] found id: "9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:18.931123 508400 cri.go:89] found id: ""
I0924 01:19:18.931147 508400 logs.go:276] 2 containers: [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e]
I0924 01:19:18.931241 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.940238 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:18.944062 508400 logs.go:123] Gathering logs for container status ...
I0924 01:19:18.944089 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:19:19.001414 508400 logs.go:123] Gathering logs for describe nodes ...
I0924 01:19:19.001468 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:19:19.147929 508400 logs.go:123] Gathering logs for kube-proxy [9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9] ...
I0924 01:19:19.147958 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:19.200173 508400 logs.go:123] Gathering logs for kindnet [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb] ...
I0924 01:19:19.200208 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:19.243136 508400 logs.go:123] Gathering logs for kindnet [a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7] ...
I0924 01:19:19.243208 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:19.283229 508400 logs.go:123] Gathering logs for storage-provisioner [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc] ...
I0924 01:19:19.283259 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:19.336221 508400 logs.go:123] Gathering logs for containerd ...
I0924 01:19:19.336249 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:19:19.407651 508400 logs.go:123] Gathering logs for dmesg ...
I0924 01:19:19.407689 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:19:19.426384 508400 logs.go:123] Gathering logs for etcd [1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b] ...
I0924 01:19:19.426421 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:19.482387 508400 logs.go:123] Gathering logs for kube-proxy [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff] ...
I0924 01:19:19.482418 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:19.524196 508400 logs.go:123] Gathering logs for kube-controller-manager [51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f] ...
I0924 01:19:19.524226 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:19.586996 508400 logs.go:123] Gathering logs for kubernetes-dashboard [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f] ...
I0924 01:19:19.587031 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:19.629351 508400 logs.go:123] Gathering logs for storage-provisioner [9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e] ...
I0924 01:19:19.629388 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:19.684132 508400 logs.go:123] Gathering logs for etcd [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6] ...
I0924 01:19:19.684160 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:19.733733 508400 logs.go:123] Gathering logs for kube-scheduler [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b] ...
I0924 01:19:19.733769 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:19.779385 508400 logs.go:123] Gathering logs for kube-scheduler [6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4] ...
I0924 01:19:19.779418 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:19.828325 508400 logs.go:123] Gathering logs for kube-controller-manager [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23] ...
I0924 01:19:19.828358 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:19.896856 508400 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:19.896894 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:19.946874 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:19.947161 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:19.978126 508400 logs.go:123] Gathering logs for kube-apiserver [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd] ...
I0924 01:19:19.978159 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:20.047655 508400 logs.go:123] Gathering logs for kube-apiserver [49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec] ...
I0924 01:19:20.047695 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:20.100126 508400 logs.go:123] Gathering logs for coredns [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1] ...
I0924 01:19:20.100164 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:20.707950 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:22.711943 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:20.146416 508400 logs.go:123] Gathering logs for coredns [898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6] ...
I0924 01:19:20.146455 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:20.188342 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:20.188407 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:19:20.188462 508400 out.go:270] X Problems detected in kubelet:
W0924 01:19:20.188473 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:20.188481 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:20.188492 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:20.188503 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:19:24.759385 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:27.207971 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:29.209588 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:31.709364 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:30.189595 508400 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0924 01:19:30.203444 508400 api_server.go:72] duration metric: took 4m17.281610313s to wait for apiserver process to appear ...
I0924 01:19:30.203472 508400 api_server.go:88] waiting for apiserver healthz status ...
I0924 01:19:30.203510 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:30.203572 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:30.250621 508400 cri.go:89] found id: "cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:30.250649 508400 cri.go:89] found id: "49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:30.250655 508400 cri.go:89] found id: ""
I0924 01:19:30.250662 508400 logs.go:276] 2 containers: [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec]
I0924 01:19:30.250731 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.254985 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.258840 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:30.258976 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:30.304331 508400 cri.go:89] found id: "c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:30.304366 508400 cri.go:89] found id: "1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:30.304372 508400 cri.go:89] found id: ""
I0924 01:19:30.304380 508400 logs.go:276] 2 containers: [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b]
I0924 01:19:30.304452 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.308595 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.312424 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:30.312522 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:30.355973 508400 cri.go:89] found id: "34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:30.355995 508400 cri.go:89] found id: "898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:30.356000 508400 cri.go:89] found id: ""
I0924 01:19:30.356007 508400 logs.go:276] 2 containers: [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6]
I0924 01:19:30.356066 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.360054 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.363820 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:30.363908 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:30.403855 508400 cri.go:89] found id: "5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:30.403878 508400 cri.go:89] found id: "6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:30.403882 508400 cri.go:89] found id: ""
I0924 01:19:30.403891 508400 logs.go:276] 2 containers: [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4]
I0924 01:19:30.403950 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.407797 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.411816 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:30.411927 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:30.452160 508400 cri.go:89] found id: "9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:30.452182 508400 cri.go:89] found id: "9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:30.452186 508400 cri.go:89] found id: ""
I0924 01:19:30.452194 508400 logs.go:276] 2 containers: [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9]
I0924 01:19:30.452253 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.456160 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.461191 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:30.461297 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:30.501612 508400 cri.go:89] found id: "23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:30.501638 508400 cri.go:89] found id: "51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:30.501643 508400 cri.go:89] found id: ""
I0924 01:19:30.501651 508400 logs.go:276] 2 containers: [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f]
I0924 01:19:30.501727 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.505950 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.511621 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:30.511745 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:30.555419 508400 cri.go:89] found id: "84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:30.555482 508400 cri.go:89] found id: "a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:30.555501 508400 cri.go:89] found id: ""
I0924 01:19:30.555514 508400 logs.go:276] 2 containers: [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7]
I0924 01:19:30.555577 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.559761 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.579349 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:30.579430 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:30.625537 508400 cri.go:89] found id: "e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:30.625563 508400 cri.go:89] found id: "9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:30.625569 508400 cri.go:89] found id: ""
I0924 01:19:30.625576 508400 logs.go:276] 2 containers: [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e]
I0924 01:19:30.625636 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.639169 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.643075 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:30.643173 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:30.684254 508400 cri.go:89] found id: "8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:30.684320 508400 cri.go:89] found id: ""
I0924 01:19:30.684335 508400 logs.go:276] 1 containers: [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f]
I0924 01:19:30.684408 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:30.689179 508400 logs.go:123] Gathering logs for kubernetes-dashboard [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f] ...
I0924 01:19:30.689202 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:30.747384 508400 logs.go:123] Gathering logs for container status ...
I0924 01:19:30.747456 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:19:30.812660 508400 logs.go:123] Gathering logs for dmesg ...
I0924 01:19:30.812692 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:19:30.830597 508400 logs.go:123] Gathering logs for etcd [1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b] ...
I0924 01:19:30.830635 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:30.884335 508400 logs.go:123] Gathering logs for kindnet [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb] ...
I0924 01:19:30.884367 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:30.931500 508400 logs.go:123] Gathering logs for storage-provisioner [9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e] ...
I0924 01:19:30.931534 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:30.975380 508400 logs.go:123] Gathering logs for kube-scheduler [6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4] ...
I0924 01:19:30.975409 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:31.033898 508400 logs.go:123] Gathering logs for kube-controller-manager [51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f] ...
I0924 01:19:31.033931 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:31.107884 508400 logs.go:123] Gathering logs for containerd ...
I0924 01:19:31.107924 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:19:31.179670 508400 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:31.179718 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:31.228684 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:31.228936 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:31.260822 508400 logs.go:123] Gathering logs for kube-apiserver [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd] ...
I0924 01:19:31.260864 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:31.315545 508400 logs.go:123] Gathering logs for etcd [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6] ...
I0924 01:19:31.315582 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:31.367154 508400 logs.go:123] Gathering logs for coredns [898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6] ...
I0924 01:19:31.367188 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:31.407305 508400 logs.go:123] Gathering logs for coredns [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1] ...
I0924 01:19:31.407336 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:31.450595 508400 logs.go:123] Gathering logs for kube-controller-manager [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23] ...
I0924 01:19:31.450624 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:31.519493 508400 logs.go:123] Gathering logs for kindnet [a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7] ...
I0924 01:19:31.519530 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:31.565836 508400 logs.go:123] Gathering logs for storage-provisioner [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc] ...
I0924 01:19:31.565870 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:31.608057 508400 logs.go:123] Gathering logs for kube-proxy [9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9] ...
I0924 01:19:31.608094 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:31.658026 508400 logs.go:123] Gathering logs for describe nodes ...
I0924 01:19:31.658055 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:19:31.809123 508400 logs.go:123] Gathering logs for kube-apiserver [49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec] ...
I0924 01:19:31.809220 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:31.865852 508400 logs.go:123] Gathering logs for kube-scheduler [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b] ...
I0924 01:19:31.865890 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:31.907118 508400 logs.go:123] Gathering logs for kube-proxy [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff] ...
I0924 01:19:31.907148 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:31.947363 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:31.947389 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:19:31.947464 508400 out.go:270] X Problems detected in kubelet:
W0924 01:19:31.947482 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:31.947510 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:31.947533 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:31.947544 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:19:34.212724 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:36.708375 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:39.208776 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:41.708313 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:43.708560 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:41.949138 508400 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0924 01:19:41.957620 508400 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0924 01:19:41.958688 508400 api_server.go:141] control plane version: v1.31.1
I0924 01:19:41.958713 508400 api_server.go:131] duration metric: took 11.755233225s to wait for apiserver health ...
I0924 01:19:41.958721 508400 system_pods.go:43] waiting for kube-system pods to appear ...
I0924 01:19:41.958744 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:41.958811 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:41.999106 508400 cri.go:89] found id: "cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:41.999127 508400 cri.go:89] found id: "49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:41.999133 508400 cri.go:89] found id: ""
I0924 01:19:41.999140 508400 logs.go:276] 2 containers: [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec]
I0924 01:19:41.999204 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.009402 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.030811 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:42.030898 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:42.074536 508400 cri.go:89] found id: "c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:42.074567 508400 cri.go:89] found id: "1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:42.074576 508400 cri.go:89] found id: ""
I0924 01:19:42.074585 508400 logs.go:276] 2 containers: [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b]
I0924 01:19:42.074654 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.079721 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.085064 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:42.085211 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:42.138851 508400 cri.go:89] found id: "34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:42.138971 508400 cri.go:89] found id: "898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:42.139001 508400 cri.go:89] found id: ""
I0924 01:19:42.139031 508400 logs.go:276] 2 containers: [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6]
I0924 01:19:42.139132 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.144225 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.148925 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:42.149016 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:42.227052 508400 cri.go:89] found id: "5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:42.227083 508400 cri.go:89] found id: "6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:42.227090 508400 cri.go:89] found id: ""
I0924 01:19:42.227099 508400 logs.go:276] 2 containers: [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4]
I0924 01:19:42.227226 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.232021 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.236592 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:42.236678 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:42.284520 508400 cri.go:89] found id: "9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:42.284548 508400 cri.go:89] found id: "9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:42.284553 508400 cri.go:89] found id: ""
I0924 01:19:42.284562 508400 logs.go:276] 2 containers: [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9]
I0924 01:19:42.284649 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.289013 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.293491 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:42.293622 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:42.338701 508400 cri.go:89] found id: "23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:42.338725 508400 cri.go:89] found id: "51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:42.338731 508400 cri.go:89] found id: ""
I0924 01:19:42.338738 508400 logs.go:276] 2 containers: [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f]
I0924 01:19:42.338797 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.343336 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.348109 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:42.348258 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:42.389487 508400 cri.go:89] found id: "84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:42.389565 508400 cri.go:89] found id: "a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:42.389584 508400 cri.go:89] found id: ""
I0924 01:19:42.389609 508400 logs.go:276] 2 containers: [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7]
I0924 01:19:42.389690 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.393712 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.397552 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:42.397629 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:42.436236 508400 cri.go:89] found id: "8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:42.436260 508400 cri.go:89] found id: ""
I0924 01:19:42.436268 508400 logs.go:276] 1 containers: [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f]
I0924 01:19:42.436355 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.440309 508400 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:42.440433 508400 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:42.489097 508400 cri.go:89] found id: "e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:42.489119 508400 cri.go:89] found id: "9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:42.489136 508400 cri.go:89] found id: ""
I0924 01:19:42.489143 508400 logs.go:276] 2 containers: [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e]
I0924 01:19:42.489201 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.493130 508400 ssh_runner.go:195] Run: which crictl
I0924 01:19:42.496655 508400 logs.go:123] Gathering logs for dmesg ...
I0924 01:19:42.496680 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:19:42.514763 508400 logs.go:123] Gathering logs for coredns [898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6] ...
I0924 01:19:42.514788 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 898641ce3e6f61272d8acf7eb77c1646f8b44e4f93aa22994aa8df7184e73fd6"
I0924 01:19:42.560280 508400 logs.go:123] Gathering logs for kube-proxy [9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9] ...
I0924 01:19:42.560319 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9698bd1ae5c4cb7085eb93df40c4a78c0e947b6b6096887cb32d3d3b3f84f3d9"
I0924 01:19:42.602776 508400 logs.go:123] Gathering logs for kindnet [a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7] ...
I0924 01:19:42.602808 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a051a2b449ff0ba4bdc77398857a62cef81b55d77c316d3c89f44d0d2b0880a7"
I0924 01:19:42.650233 508400 logs.go:123] Gathering logs for kubernetes-dashboard [8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f] ...
I0924 01:19:42.650263 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8577d1f6eee27ab28182821d47b50b803412dec67c963927e9a74c5760656c8f"
I0924 01:19:42.693972 508400 logs.go:123] Gathering logs for storage-provisioner [e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc] ...
I0924 01:19:42.694007 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e2fc67748993981d4c6eb2f308b5b4c9936e8992daabaca65541da638dc7bafc"
I0924 01:19:42.741731 508400 logs.go:123] Gathering logs for containerd ...
I0924 01:19:42.741758 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:19:42.801318 508400 logs.go:123] Gathering logs for kube-scheduler [5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b] ...
I0924 01:19:42.801355 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5ee7283b07ea8dd92daaeb3fe86c762c21998f8488e35eb45abc804e28da5a1b"
I0924 01:19:42.838788 508400 logs.go:123] Gathering logs for kube-scheduler [6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4] ...
I0924 01:19:42.838856 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6a91941381cb2fbd806c1cdd6b71ff46da40752cc4a426d4d6d1e3244a59a5d4"
I0924 01:19:42.903844 508400 logs.go:123] Gathering logs for kindnet [84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb] ...
I0924 01:19:42.903881 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84c427884fc8400e3e9c0d9609d187b64209b2ed3311860691b883ec72df07eb"
I0924 01:19:42.954874 508400 logs.go:123] Gathering logs for container status ...
I0924 01:19:42.954913 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:19:43.060093 508400 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:43.060121 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:43.115558 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:43.115823 508400 logs.go:138] Found kubelet problem: Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:43.148304 508400 logs.go:123] Gathering logs for describe nodes ...
I0924 01:19:43.148343 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:19:43.284610 508400 logs.go:123] Gathering logs for kube-apiserver [49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec] ...
I0924 01:19:43.284641 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 49fe7dc079e9bff501f95307c0034839334c0e54affc0c752936b8e78ad4cbec"
I0924 01:19:43.350548 508400 logs.go:123] Gathering logs for etcd [c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6] ...
I0924 01:19:43.350579 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1e10e0c55d8c40ab07efefe182096b7d34a0c512646431ef6ffb097fd31a1a6"
I0924 01:19:43.436344 508400 logs.go:123] Gathering logs for etcd [1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b] ...
I0924 01:19:43.436381 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1cb94a371664ea3677fa31112c560d578dac0f9675204f133025c555395cdf7b"
I0924 01:19:43.493282 508400 logs.go:123] Gathering logs for storage-provisioner [9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e] ...
I0924 01:19:43.493313 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cf16f88c6bf005843393498b031408abccf86b16b40f53e85b3479bb4dfc17e"
I0924 01:19:43.535602 508400 logs.go:123] Gathering logs for kube-apiserver [cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd] ...
I0924 01:19:43.535684 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbade5a359066cba74a1f6bc775533cc3c5cb2c6d38523b102789f5bece0f7cd"
I0924 01:19:43.592948 508400 logs.go:123] Gathering logs for coredns [34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1] ...
I0924 01:19:43.592982 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 34ec1669638ee25702af042937a534774b4fb1cb2f3ca4df2ce689ca6b58dbd1"
I0924 01:19:43.647804 508400 logs.go:123] Gathering logs for kube-proxy [9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff] ...
I0924 01:19:43.647836 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9c1c31ea42d18dcf69a39e06236994a78de89e270308135e53ca3886ca3120ff"
I0924 01:19:43.686861 508400 logs.go:123] Gathering logs for kube-controller-manager [23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23] ...
I0924 01:19:43.686891 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 23bb6483ab495904a5558786abb963f34754345c899d60d23a4dba22add17c23"
I0924 01:19:43.760213 508400 logs.go:123] Gathering logs for kube-controller-manager [51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f] ...
I0924 01:19:43.760249 508400 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 51be5c6b298bd9996efc77eff4f6ddceeee75d629fefc08a26771c02220a8a8f"
I0924 01:19:43.822376 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:43.822405 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:19:43.822484 508400 out.go:270] X Problems detected in kubelet:
W0924 01:19:43.822498 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: W0924 01:15:21.587778 657 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-558135" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-558135' and this object
W0924 01:19:43.822525 508400 out.go:270] Sep 24 01:15:21 no-preload-558135 kubelet[657]: E0924 01:15:21.587830 657 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-558135\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-558135' and this object" logger="UnhandledError"
I0924 01:19:43.822542 508400 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:43.822549 508400 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:19:46.207038 503471 pod_ready.go:103] pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace has status "Ready":"False"
I0924 01:19:47.207950 503471 pod_ready.go:82] duration metric: took 4m0.006584819s for pod "metrics-server-9975d5f86-5qvnr" in "kube-system" namespace to be "Ready" ...
E0924 01:19:47.207975 503471 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0924 01:19:47.207985 503471 pod_ready.go:39] duration metric: took 5m24.781817041s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0924 01:19:47.207999 503471 api_server.go:52] waiting for apiserver process to appear ...
I0924 01:19:47.208031 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:47.208103 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:47.248170 503471 cri.go:89] found id: "0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:47.248237 503471 cri.go:89] found id: "a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:47.248255 503471 cri.go:89] found id: ""
I0924 01:19:47.248294 503471 logs.go:276] 2 containers: [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c]
I0924 01:19:47.248373 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.252444 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.255922 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:47.256037 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:47.299255 503471 cri.go:89] found id: "1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:47.299279 503471 cri.go:89] found id: "4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:47.299284 503471 cri.go:89] found id: ""
I0924 01:19:47.299291 503471 logs.go:276] 2 containers: [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2]
I0924 01:19:47.299363 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.303065 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.307165 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:47.307239 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:47.344657 503471 cri.go:89] found id: "726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:47.344678 503471 cri.go:89] found id: "ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:47.344683 503471 cri.go:89] found id: ""
I0924 01:19:47.344690 503471 logs.go:276] 2 containers: [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c]
I0924 01:19:47.344774 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.348345 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.352584 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:47.352658 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:47.397310 503471 cri.go:89] found id: "11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:47.397331 503471 cri.go:89] found id: "92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:47.397336 503471 cri.go:89] found id: ""
I0924 01:19:47.397343 503471 logs.go:276] 2 containers: [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3]
I0924 01:19:47.397400 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.401198 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.404571 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:47.404647 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:47.450078 503471 cri.go:89] found id: "a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:47.450102 503471 cri.go:89] found id: "a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:47.450107 503471 cri.go:89] found id: ""
I0924 01:19:47.450114 503471 logs.go:276] 2 containers: [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c]
I0924 01:19:47.450195 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.454086 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.457899 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:47.457973 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:47.505274 503471 cri.go:89] found id: "14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:47.505344 503471 cri.go:89] found id: "840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:47.505363 503471 cri.go:89] found id: ""
I0924 01:19:47.505390 503471 logs.go:276] 2 containers: [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55]
I0924 01:19:47.505489 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.509368 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.513105 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:47.513224 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:47.552883 503471 cri.go:89] found id: "c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:47.552915 503471 cri.go:89] found id: "321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:47.552922 503471 cri.go:89] found id: ""
I0924 01:19:47.552930 503471 logs.go:276] 2 containers: [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee]
I0924 01:19:47.553023 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.556760 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.560250 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:47.560322 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:47.601478 503471 cri.go:89] found id: "ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:47.601521 503471 cri.go:89] found id: ""
I0924 01:19:47.601530 503471 logs.go:276] 1 containers: [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63]
I0924 01:19:47.601588 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.605414 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:47.605493 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:47.662058 503471 cri.go:89] found id: "fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:47.662083 503471 cri.go:89] found id: "fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:47.662088 503471 cri.go:89] found id: ""
I0924 01:19:47.662096 503471 logs.go:276] 2 containers: [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d]
I0924 01:19:47.662156 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.666136 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:47.669638 503471 logs.go:123] Gathering logs for coredns [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9] ...
I0924 01:19:47.669676 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:47.710636 503471 logs.go:123] Gathering logs for kube-scheduler [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57] ...
I0924 01:19:47.710669 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:47.753288 503471 logs.go:123] Gathering logs for container status ...
I0924 01:19:47.753320 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:19:47.800841 503471 logs.go:123] Gathering logs for dmesg ...
I0924 01:19:47.800872 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:19:47.817994 503471 logs.go:123] Gathering logs for describe nodes ...
I0924 01:19:47.818024 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:19:47.981580 503471 logs.go:123] Gathering logs for kube-apiserver [a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c] ...
I0924 01:19:47.981616 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:48.052176 503471 logs.go:123] Gathering logs for containerd ...
I0924 01:19:48.052216 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:19:48.118198 503471 logs.go:123] Gathering logs for kube-proxy [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d] ...
I0924 01:19:48.118240 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:48.160574 503471 logs.go:123] Gathering logs for kubernetes-dashboard [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63] ...
I0924 01:19:48.160607 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:48.205909 503471 logs.go:123] Gathering logs for storage-provisioner [fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d] ...
I0924 01:19:48.205939 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:48.245163 503471 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:48.245190 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:48.298751 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431464 666 reflector.go:138] object-"kube-system"/"kube-proxy-token-6n88c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-6n88c" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299014 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431566 666 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299243 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431638 666 reflector.go:138] object-"kube-system"/"kindnet-token-jt6n9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jt6n9" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299478 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431704 666 reflector.go:138] object-"kube-system"/"storage-provisioner-token-g5gtv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-g5gtv" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299693 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431771 666 reflector.go:138] object-"default"/"default-token-2t7hj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-2t7hj" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.299915 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431832 666 reflector.go:138] object-"kube-system"/"metrics-server-token-dpjw8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-dpjw8" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.300143 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433069 666 reflector.go:138] object-"kube-system"/"coredns-token-djfwt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-djfwt" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.300363 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433138 666 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:48.307885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:24 old-k8s-version-654890 kubelet[666]: E0924 01:14:24.244333 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.309456 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:25 old-k8s-version-654890 kubelet[666]: E0924 01:14:25.186030 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.312301 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:39 old-k8s-version-654890 kubelet[666]: E0924 01:14:39.793083 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.314771 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:51 old-k8s-version-654890 kubelet[666]: E0924 01:14:51.304991 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.315114 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:52 old-k8s-version-654890 kubelet[666]: E0924 01:14:52.318075 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.315302 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:53 old-k8s-version-654890 kubelet[666]: E0924 01:14:53.784377 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.315745 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:55 old-k8s-version-654890 kubelet[666]: E0924 01:14:55.344748 666 pod_workers.go:191] Error syncing pod c12ca6a0-fd9b-45bf-9da0-2ec1193cce32 ("storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"
W0924 01:19:48.316678 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:01 old-k8s-version-654890 kubelet[666]: E0924 01:15:01.366255 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.319205 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:07 old-k8s-version-654890 kubelet[666]: E0924 01:15:07.792722 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.319672 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:11 old-k8s-version-654890 kubelet[666]: E0924 01:15:11.008383 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.319860 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:20 old-k8s-version-654890 kubelet[666]: E0924 01:15:20.784619 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.320481 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:26 old-k8s-version-654890 kubelet[666]: E0924 01:15:26.452520 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.320821 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:30 old-k8s-version-654890 kubelet[666]: E0924 01:15:30.986615 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.321011 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:33 old-k8s-version-654890 kubelet[666]: E0924 01:15:33.783850 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.321345 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.783545 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.321534 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.784908 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.321873 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.784164 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.324342 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.792225 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.324660 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:14 old-k8s-version-654890 kubelet[666]: E0924 01:16:14.789130 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.325123 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:15 old-k8s-version-654890 kubelet[666]: E0924 01:16:15.580475 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.325467 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:20 old-k8s-version-654890 kubelet[666]: E0924 01:16:20.992161 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.325652 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:29 old-k8s-version-654890 kubelet[666]: E0924 01:16:29.787709 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.326017 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:34 old-k8s-version-654890 kubelet[666]: E0924 01:16:34.784335 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.326235 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:42 old-k8s-version-654890 kubelet[666]: E0924 01:16:42.784427 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.326575 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:45 old-k8s-version-654890 kubelet[666]: E0924 01:16:45.783505 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.326768 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:53 old-k8s-version-654890 kubelet[666]: E0924 01:16:53.784108 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.327107 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:57 old-k8s-version-654890 kubelet[666]: E0924 01:16:57.783801 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.327294 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:06 old-k8s-version-654890 kubelet[666]: E0924 01:17:06.784121 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.327631 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:11 old-k8s-version-654890 kubelet[666]: E0924 01:17:11.783481 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.327817 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:18 old-k8s-version-654890 kubelet[666]: E0924 01:17:18.783911 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.328150 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:24 old-k8s-version-654890 kubelet[666]: E0924 01:17:24.784083 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.330620 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:31 old-k8s-version-654890 kubelet[666]: E0924 01:17:31.792392 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:48.331220 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:39 old-k8s-version-654890 kubelet[666]: E0924 01:17:39.856284 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.331553 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:40 old-k8s-version-654890 kubelet[666]: E0924 01:17:40.985923 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.331738 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:45 old-k8s-version-654890 kubelet[666]: E0924 01:17:45.784024 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.332067 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:55 old-k8s-version-654890 kubelet[666]: E0924 01:17:55.783900 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.332252 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:59 old-k8s-version-654890 kubelet[666]: E0924 01:17:59.784186 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.332585 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:06 old-k8s-version-654890 kubelet[666]: E0924 01:18:06.788365 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.332774 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:12 old-k8s-version-654890 kubelet[666]: E0924 01:18:12.783859 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.333106 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:18 old-k8s-version-654890 kubelet[666]: E0924 01:18:18.783560 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.333291 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:27 old-k8s-version-654890 kubelet[666]: E0924 01:18:27.783904 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.333627 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:30 old-k8s-version-654890 kubelet[666]: E0924 01:18:30.783887 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.333814 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:41 old-k8s-version-654890 kubelet[666]: E0924 01:18:41.784268 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.334149 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: E0924 01:18:44.783947 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.334334 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:55 old-k8s-version-654890 kubelet[666]: E0924 01:18:55.783966 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.334664 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: E0924 01:18:58.784580 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.334851 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:09 old-k8s-version-654890 kubelet[666]: E0924 01:19:09.783890 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.335185 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: E0924 01:19:11.783671 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.335370 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.335699 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.336028 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.336213 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.336399 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0924 01:19:48.336409 503471 logs.go:123] Gathering logs for kube-controller-manager [840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55] ...
I0924 01:19:48.336425 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:48.394895 503471 logs.go:123] Gathering logs for kindnet [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335] ...
I0924 01:19:48.394935 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:48.445555 503471 logs.go:123] Gathering logs for coredns [ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c] ...
I0924 01:19:48.445586 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:48.484819 503471 logs.go:123] Gathering logs for kube-scheduler [92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3] ...
I0924 01:19:48.484886 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:48.531995 503471 logs.go:123] Gathering logs for kube-proxy [a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c] ...
I0924 01:19:48.532082 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:48.573118 503471 logs.go:123] Gathering logs for kube-controller-manager [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b] ...
I0924 01:19:48.573189 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:48.633525 503471 logs.go:123] Gathering logs for kindnet [321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee] ...
I0924 01:19:48.633563 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:48.680372 503471 logs.go:123] Gathering logs for kube-apiserver [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f] ...
I0924 01:19:48.680403 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:48.742350 503471 logs.go:123] Gathering logs for etcd [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e] ...
I0924 01:19:48.742384 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:48.797001 503471 logs.go:123] Gathering logs for etcd [4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2] ...
I0924 01:19:48.797035 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:48.847657 503471 logs.go:123] Gathering logs for storage-provisioner [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2] ...
I0924 01:19:48.847687 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:48.891111 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:48.891138 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:19:48.891192 503471 out.go:270] X Problems detected in kubelet:
W0924 01:19:48.891209 503471 out.go:270] Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.891225 503471 out.go:270] Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.891256 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:48.891265 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:48.891275 503471 out.go:270] Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0924 01:19:48.891281 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:19:48.891290 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:19:53.831041 508400 system_pods.go:59] 9 kube-system pods found
I0924 01:19:53.831084 508400 system_pods.go:61] "coredns-7c65d6cfc9-rq7m2" [3466c720-c315-49c5-ab82-124e89533aef] Running
I0924 01:19:53.831092 508400 system_pods.go:61] "etcd-no-preload-558135" [ff66b09a-46f2-4b17-88b0-afb588b773c6] Running
I0924 01:19:53.831096 508400 system_pods.go:61] "kindnet-f8qbt" [53c3dc9b-cea2-4ff8-b553-dd0c30157e23] Running
I0924 01:19:53.831198 508400 system_pods.go:61] "kube-apiserver-no-preload-558135" [db3f8c5f-20a4-4c42-b78c-8c27470be367] Running
I0924 01:19:53.831204 508400 system_pods.go:61] "kube-controller-manager-no-preload-558135" [dfc5a382-d975-4dc1-b20b-90cfd175b47c] Running
I0924 01:19:53.831209 508400 system_pods.go:61] "kube-proxy-krnb9" [186e9c5d-3693-48a6-9657-955801ac448d] Running
I0924 01:19:53.831213 508400 system_pods.go:61] "kube-scheduler-no-preload-558135" [890e8fae-d914-46ad-bb5a-7778684d10c8] Running
I0924 01:19:53.831220 508400 system_pods.go:61] "metrics-server-6867b74b74-46xh4" [4e3e7ba3-4efd-4a9b-9682-443e9112afba] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0924 01:19:53.831231 508400 system_pods.go:61] "storage-provisioner" [7ba817a8-e447-48d8-9b19-14a59da5d457] Running
I0924 01:19:53.831240 508400 system_pods.go:74] duration metric: took 11.872511683s to wait for pod list to return data ...
I0924 01:19:53.831264 508400 default_sa.go:34] waiting for default service account to be created ...
I0924 01:19:53.834369 508400 default_sa.go:45] found service account: "default"
I0924 01:19:53.834401 508400 default_sa.go:55] duration metric: took 3.110534ms for default service account to be created ...
I0924 01:19:53.834412 508400 system_pods.go:116] waiting for k8s-apps to be running ...
I0924 01:19:53.841050 508400 system_pods.go:86] 9 kube-system pods found
I0924 01:19:53.841085 508400 system_pods.go:89] "coredns-7c65d6cfc9-rq7m2" [3466c720-c315-49c5-ab82-124e89533aef] Running
I0924 01:19:53.841093 508400 system_pods.go:89] "etcd-no-preload-558135" [ff66b09a-46f2-4b17-88b0-afb588b773c6] Running
I0924 01:19:53.841098 508400 system_pods.go:89] "kindnet-f8qbt" [53c3dc9b-cea2-4ff8-b553-dd0c30157e23] Running
I0924 01:19:53.841102 508400 system_pods.go:89] "kube-apiserver-no-preload-558135" [db3f8c5f-20a4-4c42-b78c-8c27470be367] Running
I0924 01:19:53.841108 508400 system_pods.go:89] "kube-controller-manager-no-preload-558135" [dfc5a382-d975-4dc1-b20b-90cfd175b47c] Running
I0924 01:19:53.841112 508400 system_pods.go:89] "kube-proxy-krnb9" [186e9c5d-3693-48a6-9657-955801ac448d] Running
I0924 01:19:53.841116 508400 system_pods.go:89] "kube-scheduler-no-preload-558135" [890e8fae-d914-46ad-bb5a-7778684d10c8] Running
I0924 01:19:53.841124 508400 system_pods.go:89] "metrics-server-6867b74b74-46xh4" [4e3e7ba3-4efd-4a9b-9682-443e9112afba] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0924 01:19:53.841128 508400 system_pods.go:89] "storage-provisioner" [7ba817a8-e447-48d8-9b19-14a59da5d457] Running
I0924 01:19:53.841137 508400 system_pods.go:126] duration metric: took 6.718849ms to wait for k8s-apps to be running ...
I0924 01:19:53.841150 508400 system_svc.go:44] waiting for kubelet service to be running ....
I0924 01:19:53.841253 508400 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0924 01:19:53.854173 508400 system_svc.go:56] duration metric: took 13.012968ms WaitForService to wait for kubelet
I0924 01:19:53.854203 508400 kubeadm.go:582] duration metric: took 4m40.932372039s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0924 01:19:53.854224 508400 node_conditions.go:102] verifying NodePressure condition ...
I0924 01:19:53.857708 508400 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0924 01:19:53.857738 508400 node_conditions.go:123] node cpu capacity is 2
I0924 01:19:53.857750 508400 node_conditions.go:105] duration metric: took 3.519855ms to run NodePressure ...
I0924 01:19:53.857761 508400 start.go:241] waiting for startup goroutines ...
I0924 01:19:53.857768 508400 start.go:246] waiting for cluster config update ...
I0924 01:19:53.857779 508400 start.go:255] writing updated cluster config ...
I0924 01:19:53.858104 508400 ssh_runner.go:195] Run: rm -f paused
I0924 01:19:53.919100 508400 start.go:600] kubectl: 1.31.1, cluster: 1.31.1 (minor skew: 0)
I0924 01:19:53.921339 508400 out.go:177] * Done! kubectl is now configured to use "no-preload-558135" cluster and "default" namespace by default
I0924 01:19:58.892677 503471 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0924 01:19:58.904580 503471 api_server.go:72] duration metric: took 5m55.910809038s to wait for apiserver process to appear ...
I0924 01:19:58.904607 503471 api_server.go:88] waiting for apiserver healthz status ...
I0924 01:19:58.904644 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0924 01:19:58.904701 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0924 01:19:58.944094 503471 cri.go:89] found id: "0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:58.944123 503471 cri.go:89] found id: "a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:58.944129 503471 cri.go:89] found id: ""
I0924 01:19:58.944140 503471 logs.go:276] 2 containers: [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c]
I0924 01:19:58.944210 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.948097 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.952102 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0924 01:19:58.952189 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0924 01:19:58.990627 503471 cri.go:89] found id: "1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:58.990651 503471 cri.go:89] found id: "4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:19:58.990657 503471 cri.go:89] found id: ""
I0924 01:19:58.990664 503471 logs.go:276] 2 containers: [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2]
I0924 01:19:58.990744 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.994962 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:58.998358 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0924 01:19:58.998428 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0924 01:19:59.039355 503471 cri.go:89] found id: "726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:59.039379 503471 cri.go:89] found id: "ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:59.039384 503471 cri.go:89] found id: ""
I0924 01:19:59.039391 503471 logs.go:276] 2 containers: [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c]
I0924 01:19:59.039451 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.043628 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.047352 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0924 01:19:59.047432 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0924 01:19:59.088932 503471 cri.go:89] found id: "11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:59.088957 503471 cri.go:89] found id: "92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:59.088963 503471 cri.go:89] found id: ""
I0924 01:19:59.088970 503471 logs.go:276] 2 containers: [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3]
I0924 01:19:59.089029 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.093313 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.096780 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0924 01:19:59.096850 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0924 01:19:59.137471 503471 cri.go:89] found id: "a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:59.137492 503471 cri.go:89] found id: "a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:59.137497 503471 cri.go:89] found id: ""
I0924 01:19:59.137505 503471 logs.go:276] 2 containers: [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c]
I0924 01:19:59.137584 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.141423 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.144785 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0924 01:19:59.144903 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0924 01:19:59.183946 503471 cri.go:89] found id: "14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:19:59.183968 503471 cri.go:89] found id: "840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:19:59.183973 503471 cri.go:89] found id: ""
I0924 01:19:59.183980 503471 logs.go:276] 2 containers: [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55]
I0924 01:19:59.184038 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.187604 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.191086 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0924 01:19:59.191163 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0924 01:19:59.233371 503471 cri.go:89] found id: "c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:19:59.233394 503471 cri.go:89] found id: "321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:59.233399 503471 cri.go:89] found id: ""
I0924 01:19:59.233407 503471 logs.go:276] 2 containers: [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee]
I0924 01:19:59.233487 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.237332 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.241220 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0924 01:19:59.241332 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0924 01:19:59.284694 503471 cri.go:89] found id: "fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:59.284770 503471 cri.go:89] found id: "fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:19:59.284783 503471 cri.go:89] found id: ""
I0924 01:19:59.284791 503471 logs.go:276] 2 containers: [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d]
I0924 01:19:59.284904 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.288841 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.292850 503471 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0924 01:19:59.292961 503471 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0924 01:19:59.342825 503471 cri.go:89] found id: "ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:19:59.342879 503471 cri.go:89] found id: ""
I0924 01:19:59.342902 503471 logs.go:276] 1 containers: [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63]
I0924 01:19:59.343028 503471 ssh_runner.go:195] Run: which crictl
I0924 01:19:59.346892 503471 logs.go:123] Gathering logs for kindnet [321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee] ...
I0924 01:19:59.346956 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee"
I0924 01:19:59.390245 503471 logs.go:123] Gathering logs for storage-provisioner [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2] ...
I0924 01:19:59.390298 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2"
I0924 01:19:59.430145 503471 logs.go:123] Gathering logs for coredns [ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c] ...
I0924 01:19:59.430171 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c"
I0924 01:19:59.477526 503471 logs.go:123] Gathering logs for kube-scheduler [92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3] ...
I0924 01:19:59.477553 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3"
I0924 01:19:59.522254 503471 logs.go:123] Gathering logs for kube-proxy [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d] ...
I0924 01:19:59.522285 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d"
I0924 01:19:59.578762 503471 logs.go:123] Gathering logs for etcd [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e] ...
I0924 01:19:59.578860 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e"
I0924 01:19:59.621417 503471 logs.go:123] Gathering logs for kubelet ...
I0924 01:19:59.621447 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0924 01:19:59.677632 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431464 666 reflector.go:138] object-"kube-system"/"kube-proxy-token-6n88c": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-6n88c" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.677885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431566 666 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678107 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431638 666 reflector.go:138] object-"kube-system"/"kindnet-token-jt6n9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jt6n9" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678337 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431704 666 reflector.go:138] object-"kube-system"/"storage-provisioner-token-g5gtv": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-g5gtv" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678549 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431771 666 reflector.go:138] object-"default"/"default-token-2t7hj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-2t7hj" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.678769 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.431832 666 reflector.go:138] object-"kube-system"/"metrics-server-token-dpjw8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-dpjw8" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.679001 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433069 666 reflector.go:138] object-"kube-system"/"coredns-token-djfwt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-djfwt" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.679205 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:22 old-k8s-version-654890 kubelet[666]: E0924 01:14:22.433138 666 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-654890" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-654890' and this object
W0924 01:19:59.686595 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:24 old-k8s-version-654890 kubelet[666]: E0924 01:14:24.244333 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.688182 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:25 old-k8s-version-654890 kubelet[666]: E0924 01:14:25.186030 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.690962 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:39 old-k8s-version-654890 kubelet[666]: E0924 01:14:39.793083 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.693368 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:51 old-k8s-version-654890 kubelet[666]: E0924 01:14:51.304991 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.693697 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:52 old-k8s-version-654890 kubelet[666]: E0924 01:14:52.318075 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.693885 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:53 old-k8s-version-654890 kubelet[666]: E0924 01:14:53.784377 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.694358 503471 logs.go:138] Found kubelet problem: Sep 24 01:14:55 old-k8s-version-654890 kubelet[666]: E0924 01:14:55.344748 666 pod_workers.go:191] Error syncing pod c12ca6a0-fd9b-45bf-9da0-2ec1193cce32 ("storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(c12ca6a0-fd9b-45bf-9da0-2ec1193cce32)"
W0924 01:19:59.695337 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:01 old-k8s-version-654890 kubelet[666]: E0924 01:15:01.366255 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.697828 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:07 old-k8s-version-654890 kubelet[666]: E0924 01:15:07.792722 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.698297 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:11 old-k8s-version-654890 kubelet[666]: E0924 01:15:11.008383 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.698482 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:20 old-k8s-version-654890 kubelet[666]: E0924 01:15:20.784619 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.699079 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:26 old-k8s-version-654890 kubelet[666]: E0924 01:15:26.452520 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.699407 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:30 old-k8s-version-654890 kubelet[666]: E0924 01:15:30.986615 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.699593 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:33 old-k8s-version-654890 kubelet[666]: E0924 01:15:33.783850 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.699929 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.783545 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.700115 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:44 old-k8s-version-654890 kubelet[666]: E0924 01:15:44.784908 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.700443 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.784164 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.702897 503471 logs.go:138] Found kubelet problem: Sep 24 01:15:59 old-k8s-version-654890 kubelet[666]: E0924 01:15:59.792225 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.703245 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:14 old-k8s-version-654890 kubelet[666]: E0924 01:16:14.789130 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.703708 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:15 old-k8s-version-654890 kubelet[666]: E0924 01:16:15.580475 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704038 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:20 old-k8s-version-654890 kubelet[666]: E0924 01:16:20.992161 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704222 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:29 old-k8s-version-654890 kubelet[666]: E0924 01:16:29.787709 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.704549 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:34 old-k8s-version-654890 kubelet[666]: E0924 01:16:34.784335 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.704734 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:42 old-k8s-version-654890 kubelet[666]: E0924 01:16:42.784427 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.705065 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:45 old-k8s-version-654890 kubelet[666]: E0924 01:16:45.783505 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.705249 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:53 old-k8s-version-654890 kubelet[666]: E0924 01:16:53.784108 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.705575 503471 logs.go:138] Found kubelet problem: Sep 24 01:16:57 old-k8s-version-654890 kubelet[666]: E0924 01:16:57.783801 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.705759 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:06 old-k8s-version-654890 kubelet[666]: E0924 01:17:06.784121 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.706092 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:11 old-k8s-version-654890 kubelet[666]: E0924 01:17:11.783481 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.706279 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:18 old-k8s-version-654890 kubelet[666]: E0924 01:17:18.783911 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.706609 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:24 old-k8s-version-654890 kubelet[666]: E0924 01:17:24.784083 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.709045 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:31 old-k8s-version-654890 kubelet[666]: E0924 01:17:31.792392 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0924 01:19:59.709642 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:39 old-k8s-version-654890 kubelet[666]: E0924 01:17:39.856284 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.709971 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:40 old-k8s-version-654890 kubelet[666]: E0924 01:17:40.985923 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.710166 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:45 old-k8s-version-654890 kubelet[666]: E0924 01:17:45.784024 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.710493 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:55 old-k8s-version-654890 kubelet[666]: E0924 01:17:55.783900 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.710677 503471 logs.go:138] Found kubelet problem: Sep 24 01:17:59 old-k8s-version-654890 kubelet[666]: E0924 01:17:59.784186 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.711009 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:06 old-k8s-version-654890 kubelet[666]: E0924 01:18:06.788365 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.711196 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:12 old-k8s-version-654890 kubelet[666]: E0924 01:18:12.783859 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.711569 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:18 old-k8s-version-654890 kubelet[666]: E0924 01:18:18.783560 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.711755 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:27 old-k8s-version-654890 kubelet[666]: E0924 01:18:27.783904 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.712084 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:30 old-k8s-version-654890 kubelet[666]: E0924 01:18:30.783887 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.712268 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:41 old-k8s-version-654890 kubelet[666]: E0924 01:18:41.784268 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.712597 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: E0924 01:18:44.783947 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.712784 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:55 old-k8s-version-654890 kubelet[666]: E0924 01:18:55.783966 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.713114 503471 logs.go:138] Found kubelet problem: Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: E0924 01:18:58.784580 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.713298 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:09 old-k8s-version-654890 kubelet[666]: E0924 01:19:09.783890 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.713628 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: E0924 01:19:11.783671 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.713812 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.714144 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.714487 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:19:59.714673 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.714857 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:19:59.715195 503471 logs.go:138] Found kubelet problem: Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
I0924 01:19:59.715208 503471 logs.go:123] Gathering logs for kube-apiserver [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f] ...
I0924 01:19:59.715224 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f"
I0924 01:19:59.775878 503471 logs.go:123] Gathering logs for kube-apiserver [a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c] ...
I0924 01:19:59.775913 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c"
I0924 01:19:59.848947 503471 logs.go:123] Gathering logs for coredns [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9] ...
I0924 01:19:59.848982 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9"
I0924 01:19:59.893787 503471 logs.go:123] Gathering logs for kube-scheduler [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57] ...
I0924 01:19:59.893817 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57"
I0924 01:19:59.934822 503471 logs.go:123] Gathering logs for kube-proxy [a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c] ...
I0924 01:19:59.934854 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c"
I0924 01:19:59.975700 503471 logs.go:123] Gathering logs for kube-controller-manager [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b] ...
I0924 01:19:59.975727 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b"
I0924 01:20:00.150686 503471 logs.go:123] Gathering logs for containerd ...
I0924 01:20:00.150774 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0924 01:20:00.348822 503471 logs.go:123] Gathering logs for dmesg ...
I0924 01:20:00.348953 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0924 01:20:00.421331 503471 logs.go:123] Gathering logs for describe nodes ...
I0924 01:20:00.421432 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0924 01:20:00.860083 503471 logs.go:123] Gathering logs for etcd [4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2] ...
I0924 01:20:00.860117 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2"
I0924 01:20:00.933708 503471 logs.go:123] Gathering logs for kubernetes-dashboard [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63] ...
I0924 01:20:00.933741 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63"
I0924 01:20:00.983639 503471 logs.go:123] Gathering logs for container status ...
I0924 01:20:00.983672 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0924 01:20:01.031385 503471 logs.go:123] Gathering logs for kube-controller-manager [840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55] ...
I0924 01:20:01.031544 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55"
I0924 01:20:01.119721 503471 logs.go:123] Gathering logs for kindnet [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335] ...
I0924 01:20:01.119762 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335"
I0924 01:20:01.189736 503471 logs.go:123] Gathering logs for storage-provisioner [fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d] ...
I0924 01:20:01.189781 503471 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d"
I0924 01:20:01.236887 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:20:01.236929 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0924 01:20:01.236990 503471 out.go:270] X Problems detected in kubelet:
W0924 01:20:01.237011 503471 out.go:270] Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:20:01.237022 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
W0924 01:20:01.237031 503471 out.go:270] Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:20:01.237044 503471 out.go:270] Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0924 01:20:01.237050 503471 out.go:270] Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
I0924 01:20:01.237057 503471 out.go:358] Setting ErrFile to fd 2...
I0924 01:20:01.237068 503471 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0924 01:20:11.238141 503471 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0924 01:20:11.249366 503471 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0924 01:20:11.251766 503471 out.go:201]
W0924 01:20:11.253416 503471 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0924 01:20:11.253457 503471 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0924 01:20:11.253478 503471 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0924 01:20:11.253487 503471 out.go:270] *
W0924 01:20:11.254517 503471 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0924 01:20:11.256620 503471 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
dbc349201d122 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 88126bedeb6e3 dashboard-metrics-scraper-8d5bb5db8-99rbp
fb460cedc7032 ba04bb24b9575 5 minutes ago Running storage-provisioner 3 5e24378a3bb45 storage-provisioner
ffc127edc7c13 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 f06d76e7c81f4 kubernetes-dashboard-cd95d586-h24mv
726b9b637e708 db91994f4ee8f 5 minutes ago Running coredns 1 5c0836d67030d coredns-74ff55c5b-4h5vb
a3efc4d0adcbc 25a5233254979 5 minutes ago Running kube-proxy 1 2971d9181326c kube-proxy-dctnp
c1d6f4b139eb3 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 bb0ecc9201018 kindnet-nlwr9
c9d42857f2854 1611cd07b61d5 5 minutes ago Running busybox 1 08ed38edff5f4 busybox
fd25616712b12 ba04bb24b9575 5 minutes ago Exited storage-provisioner 2 5e24378a3bb45 storage-provisioner
11cd4f142a26c e7605f88f17d6 6 minutes ago Running kube-scheduler 1 e667d66c4f0fa kube-scheduler-old-k8s-version-654890
14974f31e17ef 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 e54d68aa9f6cf kube-controller-manager-old-k8s-version-654890
1a02c274b765e 05b738aa1bc63 6 minutes ago Running etcd 1 b7520c800888c etcd-old-k8s-version-654890
0d53a8585147b 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 d4238bde7ae3f kube-apiserver-old-k8s-version-654890
ab7e937daae24 1611cd07b61d5 6 minutes ago Exited busybox 0 1739a553cdd41 busybox
ef14acfdf46ff db91994f4ee8f 8 minutes ago Exited coredns 0 acd6649c873a8 coredns-74ff55c5b-4h5vb
321cc991e8aee 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 d54bd258985f3 kindnet-nlwr9
a216776df4a76 25a5233254979 8 minutes ago Exited kube-proxy 0 e7534ab746538 kube-proxy-dctnp
a3f73fa00d878 2c08bbbc02d3a 9 minutes ago Exited kube-apiserver 0 1cc1f10e426da kube-apiserver-old-k8s-version-654890
92e903457a976 e7605f88f17d6 9 minutes ago Exited kube-scheduler 0 c65917054c52e kube-scheduler-old-k8s-version-654890
840c0a636700f 1df8a2b116bd1 9 minutes ago Exited kube-controller-manager 0 e4410cb979bab kube-controller-manager-old-k8s-version-654890
4b3e46c31d87b 05b738aa1bc63 9 minutes ago Exited etcd 0 ee5ed1b99e768 etcd-old-k8s-version-654890
==> containerd <==
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.819256262Z" level=info msg="CreateContainer within sandbox \"88126bedeb6e3b952fa5fb4dcaba221d7815e6fb9331135f05a02c2901fd56e2\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b\""
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.821771010Z" level=info msg="StartContainer for \"673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b\""
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.891316541Z" level=info msg="StartContainer for \"673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b\" returns successfully"
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.924728953Z" level=info msg="shim disconnected" id=673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b namespace=k8s.io
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.924951181Z" level=warning msg="cleaning up after shim disconnected" id=673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b namespace=k8s.io
Sep 24 01:16:14 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:14.925030984Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 24 01:16:15 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:15.600267470Z" level=info msg="RemoveContainer for \"b4d87e032e2da4676d5aad7b4fee7e24821a7bc42626369c5fefd58fc67981e2\""
Sep 24 01:16:15 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:16:15.608576139Z" level=info msg="RemoveContainer for \"b4d87e032e2da4676d5aad7b4fee7e24821a7bc42626369c5fefd58fc67981e2\" returns successfully"
Sep 24 01:17:31 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:31.784472043Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:17:31 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:31.789911194Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Sep 24 01:17:31 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:31.791711644Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 24 01:17:31 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:31.791761802Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.786893937Z" level=info msg="CreateContainer within sandbox \"88126bedeb6e3b952fa5fb4dcaba221d7815e6fb9331135f05a02c2901fd56e2\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.803529502Z" level=info msg="CreateContainer within sandbox \"88126bedeb6e3b952fa5fb4dcaba221d7815e6fb9331135f05a02c2901fd56e2\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8\""
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.804212251Z" level=info msg="StartContainer for \"dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8\""
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.894837484Z" level=info msg="StartContainer for \"dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8\" returns successfully"
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.921121806Z" level=info msg="shim disconnected" id=dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8 namespace=k8s.io
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.921188227Z" level=warning msg="cleaning up after shim disconnected" id=dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8 namespace=k8s.io
Sep 24 01:17:38 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:38.921201240Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 24 01:17:39 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:39.853280411Z" level=info msg="RemoveContainer for \"673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b\""
Sep 24 01:17:39 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:17:39.859426097Z" level=info msg="RemoveContainer for \"673c98d5ea337644569e7b6b17b7eff8df8749f9704e79eae5f49d2c87f5a07b\" returns successfully"
Sep 24 01:20:12 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:20:12.786365040Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:20:12 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:20:12.801404917Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Sep 24 01:20:12 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:20:12.803421250Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 24 01:20:12 old-k8s-version-654890 containerd[573]: time="2024-09-24T01:20:12.803537459Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [726b9b637e70873b8dcf15fbdf3dcedddcb9cd6aefaf87948d0de9b3e24e0da9] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:43974 - 30852 "HINFO IN 7371709559352309961.4617567100753704921. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.01302635s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0924 01:14:56.446636 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-24 01:14:26.446076485 +0000 UTC m=+0.047964985) (total time: 30.000445822s):
Trace[2019727887]: [30.000445822s] [30.000445822s] END
E0924 01:14:56.446675 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0924 01:14:56.446954 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-24 01:14:26.44661475 +0000 UTC m=+0.048503250) (total time: 30.000278749s):
Trace[939984059]: [30.000278749s] [30.000278749s] END
E0924 01:14:56.446998 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0924 01:14:56.447557 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-24 01:14:26.44684113 +0000 UTC m=+0.048729622) (total time: 30.000692755s):
Trace[911902081]: [30.000692755s] [30.000692755s] END
E0924 01:14:56.447578 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [ef14acfdf46ffeb7940e8ac4eaff81bfbeec196a22051bc2cb550f636b541f4c] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:34331 - 53042 "HINFO IN 8439755339388553706.260013902647775604. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.041521832s
==> describe nodes <==
Name: old-k8s-version-654890
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-654890
kubernetes.io/os=linux
minikube.k8s.io/commit=41795ff643dcbe39cdf81f27d064464d20ae8e7c
minikube.k8s.io/name=old-k8s-version-654890
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_24T01_11_18_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 24 Sep 2024 01:11:14 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-654890
AcquireTime: <unset>
RenewTime: Tue, 24 Sep 2024 01:20:05 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 24 Sep 2024 01:15:12 +0000 Tue, 24 Sep 2024 01:11:07 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 24 Sep 2024 01:15:12 +0000 Tue, 24 Sep 2024 01:11:07 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 24 Sep 2024 01:15:12 +0000 Tue, 24 Sep 2024 01:11:07 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 24 Sep 2024 01:15:12 +0000 Tue, 24 Sep 2024 01:11:33 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-654890
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
System Info:
Machine ID: 3dde889bde1140d582de9eb21e00c1b1
System UUID: e3111167-6378-4792-ab33-07385a8ccf74
Boot ID: e579fd69-d9d0-4441-8d26-00b8ee3b7574
Kernel Version: 5.15.0-1070-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m44s
kube-system coredns-74ff55c5b-4h5vb 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m40s
kube-system etcd-old-k8s-version-654890 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m47s
kube-system kindnet-nlwr9 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m40s
kube-system kube-apiserver-old-k8s-version-654890 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m47s
kube-system kube-controller-manager-old-k8s-version-654890 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m47s
kube-system kube-proxy-dctnp 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m40s
kube-system kube-scheduler-old-k8s-version-654890 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m47s
kube-system metrics-server-9975d5f86-5qvnr 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m33s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m39s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-99rbp 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m33s
kubernetes-dashboard kubernetes-dashboard-cd95d586-h24mv 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m33s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 9m7s (x4 over 9m7s) kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 9m7s (x4 over 9m7s) kubelet Node old-k8s-version-654890 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 9m7s (x4 over 9m7s) kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientPID
Normal Starting 8m48s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m48s kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m48s kubelet Node old-k8s-version-654890 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m48s kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m47s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m40s kubelet Node old-k8s-version-654890 status is now: NodeReady
Normal Starting 8m38s kube-proxy Starting kube-proxy.
Normal Starting 6m3s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-654890 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m3s (x7 over 6m3s) kubelet Node old-k8s-version-654890 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m3s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m47s kube-proxy Starting kube-proxy.
==> dmesg <==
[Sep23 23:56] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.
[Sep24 00:45] hrtimer: interrupt took 5498017 ns
==> etcd [1a02c274b765eec9b703f23f5251e302342e1238e7833b5bdd4c5284939dba4e] <==
2024-09-24 01:16:09.668255 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:16:19.668137 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:16:29.667959 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:16:39.668081 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:16:49.667891 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:16:59.668158 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:09.668114 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:19.668094 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:29.668046 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:39.667997 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:49.668415 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:17:59.668303 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:09.668035 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:19.668043 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:29.668094 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:39.668026 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:49.667943 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:18:59.668103 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:09.667943 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:19.668030 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:29.668136 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:39.667924 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:49.668105 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:19:59.668131 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:20:09.668354 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [4b3e46c31d87b9f4360a7d001b62bfbec9eaeb7fac41c03c7afdcf345bdc85d2] <==
raft2024/09/24 01:11:07 INFO: ea7e25599daad906 became leader at term 2
raft2024/09/24 01:11:07 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-09-24 01:11:07.631231 I | etcdserver: published {Name:old-k8s-version-654890 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-09-24 01:11:07.631941 I | embed: ready to serve client requests
2024-09-24 01:11:07.642711 I | embed: serving client requests on 192.168.76.2:2379
2024-09-24 01:11:07.642838 I | etcdserver: setting up the initial cluster version to 3.4
2024-09-24 01:11:07.643046 I | embed: ready to serve client requests
2024-09-24 01:11:07.644906 I | embed: serving client requests on 127.0.0.1:2379
2024-09-24 01:11:07.646991 N | etcdserver/membership: set the initial cluster version to 3.4
2024-09-24 01:11:07.647213 I | etcdserver/api: enabled capabilities for version 3.4
2024-09-24 01:11:16.972706 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:11:27.839879 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:11:35.734995 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:11:45.735657 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:11:55.735468 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:05.736045 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:15.735067 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:25.734887 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:35.735082 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:45.735076 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:12:55.735061 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:13:05.736010 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:13:15.734974 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:13:25.734860 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-24 01:13:35.735108 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
01:20:13 up 3:02, 0 users, load average: 0.58, 1.68, 2.41
Linux old-k8s-version-654890 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:48 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [321cc991e8aee83030aaf0fd4a0169e888f5b32b23247e48167abb3444235dee] <==
I0924 01:11:37.005177 1 controller.go:374] Syncing nftables rules
I0924 01:11:46.804422 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:11:46.804459 1 main.go:299] handling current node
I0924 01:11:56.804261 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:11:56.804298 1 main.go:299] handling current node
I0924 01:12:06.803690 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:06.803757 1 main.go:299] handling current node
I0924 01:12:16.811019 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:16.811053 1 main.go:299] handling current node
I0924 01:12:26.810733 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:26.810768 1 main.go:299] handling current node
I0924 01:12:36.803693 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:36.803728 1 main.go:299] handling current node
I0924 01:12:46.809427 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:46.809464 1 main.go:299] handling current node
I0924 01:12:56.803684 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:12:56.803724 1 main.go:299] handling current node
I0924 01:13:06.811086 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:13:06.811122 1 main.go:299] handling current node
I0924 01:13:16.812772 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:13:16.812831 1 main.go:299] handling current node
I0924 01:13:26.805061 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:13:26.805101 1 main.go:299] handling current node
I0924 01:13:36.804067 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:13:36.804100 1 main.go:299] handling current node
==> kindnet [c1d6f4b139eb39cb24db27d8618300757f678faca88fa20d24e066fa8a75c335] <==
I0924 01:18:05.624256 1 main.go:299] handling current node
I0924 01:18:15.626998 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:18:15.627191 1 main.go:299] handling current node
I0924 01:18:25.620358 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:18:25.620396 1 main.go:299] handling current node
I0924 01:18:35.624003 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:18:35.624035 1 main.go:299] handling current node
I0924 01:18:45.629084 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:18:45.629194 1 main.go:299] handling current node
I0924 01:18:55.629455 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:18:55.629550 1 main.go:299] handling current node
I0924 01:19:05.627281 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:05.627398 1 main.go:299] handling current node
I0924 01:19:15.629337 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:15.629437 1 main.go:299] handling current node
I0924 01:19:25.620904 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:25.620948 1 main.go:299] handling current node
I0924 01:19:35.627385 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:35.627434 1 main.go:299] handling current node
I0924 01:19:45.623070 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:45.623120 1 main.go:299] handling current node
I0924 01:19:55.627180 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:19:55.627299 1 main.go:299] handling current node
I0924 01:20:05.627387 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0924 01:20:05.627487 1 main.go:299] handling current node
==> kube-apiserver [0d53a8585147b008b66d8a2452e78b09c653dbd3ce70d1d015e55f15fe5bc97f] <==
I0924 01:16:50.075246 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:16:50.075257 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0924 01:17:24.884591 1 handler_proxy.go:102] no RequestInfo found in the context
E0924 01:17:24.884691 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0924 01:17:24.884712 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0924 01:17:32.916702 1 client.go:360] parsed scheme: "passthrough"
I0924 01:17:32.916772 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:17:32.916782 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0924 01:18:09.335315 1 client.go:360] parsed scheme: "passthrough"
I0924 01:18:09.335361 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:18:09.335545 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0924 01:18:52.999123 1 client.go:360] parsed scheme: "passthrough"
I0924 01:18:52.999170 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:18:52.999182 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0924 01:19:23.456574 1 handler_proxy.go:102] no RequestInfo found in the context
E0924 01:19:23.456655 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0924 01:19:23.456664 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0924 01:19:27.215933 1 client.go:360] parsed scheme: "passthrough"
I0924 01:19:27.215978 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:19:27.215987 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0924 01:20:01.868907 1 client.go:360] parsed scheme: "passthrough"
I0924 01:20:01.868951 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:20:01.868959 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [a3f73fa00d8780bfbae087dd100b5dea468ee8262086a416cf2f1a32baf45e2c] <==
I0924 01:11:15.342478 1 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0924 01:11:15.342561 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0924 01:11:15.371413 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0924 01:11:15.378448 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0924 01:11:15.378532 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0924 01:11:15.890404 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0924 01:11:15.939347 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0924 01:11:16.098730 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I0924 01:11:16.100105 1 controller.go:606] quota admission added evaluator for: endpoints
I0924 01:11:16.106707 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0924 01:11:17.073923 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0924 01:11:17.440540 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0924 01:11:17.498620 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0924 01:11:25.899471 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0924 01:11:33.171121 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0924 01:11:33.172351 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0924 01:11:53.705296 1 client.go:360] parsed scheme: "passthrough"
I0924 01:11:53.705343 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:11:53.705496 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0924 01:12:29.944745 1 client.go:360] parsed scheme: "passthrough"
I0924 01:12:29.944958 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:12:29.944976 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0924 01:13:08.100452 1 client.go:360] parsed scheme: "passthrough"
I0924 01:13:08.100496 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0924 01:13:08.100505 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [14974f31e17ef863e970aac9ec385dd3a7df85dabded10c6c4158fa5ae8bb04b] <==
W0924 01:15:45.719178 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:16:12.835620 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:16:17.369809 1 request.go:655] Throttling request took 1.048420999s, request: GET:https://192.168.76.2:8443/apis/authorization.k8s.io/v1?timeout=32s
W0924 01:16:18.221443 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:16:43.337760 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:16:49.872066 1 request.go:655] Throttling request took 1.048345831s, request: GET:https://192.168.76.2:8443/apis/authentication.k8s.io/v1beta1?timeout=32s
W0924 01:16:50.724123 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:17:13.839739 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:17:22.374644 1 request.go:655] Throttling request took 1.048419792s, request: GET:https://192.168.76.2:8443/apis/admissionregistration.k8s.io/v1?timeout=32s
W0924 01:17:23.226115 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:17:44.341746 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:17:54.876511 1 request.go:655] Throttling request took 1.048387215s, request: GET:https://192.168.76.2:8443/apis/policy/v1beta1?timeout=32s
W0924 01:17:55.728086 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:18:14.843877 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:18:27.378520 1 request.go:655] Throttling request took 1.048423036s, request: GET:https://192.168.76.2:8443/apis/batch/v1?timeout=32s
W0924 01:18:28.229934 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:18:45.346488 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:18:59.880377 1 request.go:655] Throttling request took 1.048485652s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0924 01:19:00.731947 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:19:15.848415 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:19:32.382500 1 request.go:655] Throttling request took 1.04822641s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0924 01:19:33.233807 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0924 01:19:46.350672 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0924 01:20:04.884268 1 request.go:655] Throttling request took 1.048447779s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0924 01:20:05.735757 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [840c0a636700f21e186c2b7d0a495ae8ba333ce02654b24940c178384884ac55] <==
E0924 01:11:33.103429 1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
I0924 01:11:33.140840 1 shared_informer.go:247] Caches are synced for stateful set
I0924 01:11:33.163609 1 shared_informer.go:247] Caches are synced for deployment
I0924 01:11:33.165375 1 shared_informer.go:247] Caches are synced for daemon sets
I0924 01:11:33.176363 1 shared_informer.go:247] Caches are synced for disruption
I0924 01:11:33.176536 1 disruption.go:339] Sending events to api server.
I0924 01:11:33.195077 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0924 01:11:33.200884 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-dtp6w"
I0924 01:11:33.227585 1 shared_informer.go:247] Caches are synced for resource quota
I0924 01:11:33.241250 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-nlwr9"
I0924 01:11:33.241783 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-dctnp"
I0924 01:11:33.242288 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-4h5vb"
I0924 01:11:33.267428 1 shared_informer.go:247] Caches are synced for resource quota
E0924 01:11:33.298134 1 daemon_controller.go:320] kube-system/kube-proxy failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-proxy", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"f33e8adb-f65a-4a76-bfa6-5d096706451a", ResourceVersion:"249", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63862737077, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubeadm", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001701dc0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001701de0)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.
LabelSelector)(0x4001701e00), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-proxy"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"kube-proxy", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Gl
usterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0x40018c0740), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001701
e20), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeS
ource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001701e40), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil),
AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kube-proxy", Image:"k8s.gcr.io/kube-proxy:v1.20.0", Command:[]string{"/usr/local/bin/kube-proxy", "--config=/var/lib/kube-proxy/config.conf", "--hostname-override=$(NODE_NAME)"}, Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001701e80)}}, Resources:v1.R
esourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList(nil)}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"kube-proxy", ReadOnly:false, MountPath:"/var/lib/kube-proxy", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x40016e8fc0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPo
licy:"Always", TerminationGracePeriodSeconds:(*int64)(0x40019e0718), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"kube-proxy", DeprecatedServiceAccount:"kube-proxy", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x40004cc460), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), Runtime
ClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x40002fbbc8)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x40019e0768)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again
I0924 01:11:33.390937 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0924 01:11:33.691120 1 shared_informer.go:247] Caches are synced for garbage collector
I0924 01:11:33.716924 1 shared_informer.go:247] Caches are synced for garbage collector
I0924 01:11:33.716952 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0924 01:11:34.428361 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0924 01:11:34.462041 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-dtp6w"
I0924 01:11:38.043649 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0924 01:13:39.799889 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
I0924 01:13:39.836822 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"metrics-server-9975d5f86-\" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount \"metrics-server\" not found"
E0924 01:13:39.887978 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with pods "metrics-server-9975d5f86-" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount "metrics-server" not found
E0924 01:13:39.960422 1 clusterroleaggregation_controller.go:181] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [a216776df4a7683d11f652ba6188ed06bc8f4d2aca1cfdfd75fd8caa734d5f7c] <==
I0924 01:11:35.486972 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0924 01:11:35.487060 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0924 01:11:35.517765 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0924 01:11:35.517859 1 server_others.go:185] Using iptables Proxier.
I0924 01:11:35.518069 1 server.go:650] Version: v1.20.0
I0924 01:11:35.518998 1 config.go:315] Starting service config controller
I0924 01:11:35.519013 1 shared_informer.go:240] Waiting for caches to sync for service config
I0924 01:11:35.519031 1 config.go:224] Starting endpoint slice config controller
I0924 01:11:35.519035 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0924 01:11:35.623506 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0924 01:11:35.623629 1 shared_informer.go:247] Caches are synced for service config
==> kube-proxy [a3efc4d0adcbc5ed4559be7029923280ee64507be7e1478e828a19882024959d] <==
I0924 01:14:26.465975 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0924 01:14:26.466242 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0924 01:14:26.484078 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0924 01:14:26.484174 1 server_others.go:185] Using iptables Proxier.
I0924 01:14:26.484395 1 server.go:650] Version: v1.20.0
I0924 01:14:26.485221 1 config.go:315] Starting service config controller
I0924 01:14:26.485243 1 shared_informer.go:240] Waiting for caches to sync for service config
I0924 01:14:26.485263 1 config.go:224] Starting endpoint slice config controller
I0924 01:14:26.485267 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0924 01:14:26.585413 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0924 01:14:26.585424 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [11cd4f142a26cd68a385f7146deef4ee88d1b79b18a062cef614b19d4789bc57] <==
I0924 01:14:17.021499 1 serving.go:331] Generated self-signed cert in-memory
W0924 01:14:22.381067 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0924 01:14:22.381109 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0924 01:14:22.381129 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0924 01:14:22.381137 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0924 01:14:22.535691 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0924 01:14:22.543705 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0924 01:14:22.543747 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0924 01:14:22.543766 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0924 01:14:22.647341 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [92e903457a9761a7af4f917c69d759a5da333110e45a058d985445733540fbd3] <==
W0924 01:11:14.579894 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0924 01:11:14.580118 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0924 01:11:14.580206 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0924 01:11:14.580279 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0924 01:11:14.625550 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0924 01:11:14.625720 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0924 01:11:14.625804 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0924 01:11:14.625975 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0924 01:11:14.632615 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0924 01:11:14.632730 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0924 01:11:14.643004 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0924 01:11:14.643632 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0924 01:11:14.643977 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0924 01:11:14.644258 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0924 01:11:14.644497 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0924 01:11:14.645415 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0924 01:11:14.645533 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0924 01:11:14.645871 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0924 01:11:14.646118 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0924 01:11:14.647374 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0924 01:11:15.540508 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0924 01:11:15.597715 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0924 01:11:15.627226 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0924 01:11:15.643360 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
I0924 01:11:16.226097 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Sep 24 01:18:41 old-k8s-version-654890 kubelet[666]: E0924 01:18:41.784268 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: I0924 01:18:44.783165 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:18:44 old-k8s-version-654890 kubelet[666]: E0924 01:18:44.783947 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:18:55 old-k8s-version-654890 kubelet[666]: E0924 01:18:55.783966 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: I0924 01:18:58.783801 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:18:58 old-k8s-version-654890 kubelet[666]: E0924 01:18:58.784580 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:09 old-k8s-version-654890 kubelet[666]: E0924 01:19:09.783890 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: I0924 01:19:11.783287 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:19:11 old-k8s-version-654890 kubelet[666]: E0924 01:19:11.783671 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:20 old-k8s-version-654890 kubelet[666]: E0924 01:19:20.784316 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: I0924 01:19:23.783172 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:19:23 old-k8s-version-654890 kubelet[666]: E0924 01:19:23.783518 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: I0924 01:19:34.787670 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.788045 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:34 old-k8s-version-654890 kubelet[666]: E0924 01:19:34.792957 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:45 old-k8s-version-654890 kubelet[666]: E0924 01:19:45.783770 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: I0924 01:19:49.783155 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:19:49 old-k8s-version-654890 kubelet[666]: E0924 01:19:49.783506 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:19:59 old-k8s-version-654890 kubelet[666]: E0924 01:19:59.784017 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 24 01:20:01 old-k8s-version-654890 kubelet[666]: I0924 01:20:01.783191 666 scope.go:95] [topologymanager] RemoveContainer - Container ID: dbc349201d12250e06795b49e35b576fac970cbc68e51834bf8c0b6b6e3ec5b8
Sep 24 01:20:01 old-k8s-version-654890 kubelet[666]: E0924 01:20:01.783590 666 pod_workers.go:191] Error syncing pod a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b ("dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-99rbp_kubernetes-dashboard(a14eb15f-c6ee-4a9e-ad4e-9b64b2037c7b)"
Sep 24 01:20:12 old-k8s-version-654890 kubelet[666]: E0924 01:20:12.803702 666 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 24 01:20:12 old-k8s-version-654890 kubelet[666]: E0924 01:20:12.803757 666 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 24 01:20:12 old-k8s-version-654890 kubelet[666]: E0924 01:20:12.803914 666 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-dpjw8,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-5qvnr_kube-system(38d3926
0-494a-4760-91bb-091e6afba5ca): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Sep 24 01:20:12 old-k8s-version-654890 kubelet[666]: E0924 01:20:12.803962 666 pod_workers.go:191] Error syncing pod 38d39260-494a-4760-91bb-091e6afba5ca ("metrics-server-9975d5f86-5qvnr_kube-system(38d39260-494a-4760-91bb-091e6afba5ca)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
==> kubernetes-dashboard [ffc127edc7c13cabaa59d04ea0a177c94e9e6bf5bfccbfda48fd77224f966e63] <==
2024/09/24 01:14:44 Using namespace: kubernetes-dashboard
2024/09/24 01:14:44 Using in-cluster config to connect to apiserver
2024/09/24 01:14:44 Using secret token for csrf signing
2024/09/24 01:14:44 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/09/24 01:14:44 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/09/24 01:14:44 Successful initial request to the apiserver, version: v1.20.0
2024/09/24 01:14:44 Generating JWE encryption key
2024/09/24 01:14:44 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/09/24 01:14:44 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/09/24 01:14:45 Initializing JWE encryption key from synchronized object
2024/09/24 01:14:45 Creating in-cluster Sidecar client
2024/09/24 01:14:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:14:45 Serving insecurely on HTTP port: 9090
2024/09/24 01:15:15 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:15:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:16:15 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:16:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:17:15 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:17:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:18:15 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:18:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:19:15 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:19:45 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/24 01:14:44 Starting overwatch
==> storage-provisioner [fb460cedc70328f07448f754198dcea04f2edccb76c1ffe5b7a6a0942e34a3f2] <==
I0924 01:15:10.007854 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0924 01:15:10.027208 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0924 01:15:10.030010 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0924 01:15:27.515761 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0924 01:15:27.517783 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-654890_6e1f6478-2303-410e-bcb2-e9fddb38566d!
I0924 01:15:27.518553 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"1c3ff6ee-6fcf-4f7b-8635-09c423e83901", APIVersion:"v1", ResourceVersion:"859", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-654890_6e1f6478-2303-410e-bcb2-e9fddb38566d became leader
I0924 01:15:27.619292 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-654890_6e1f6478-2303-410e-bcb2-e9fddb38566d!
==> storage-provisioner [fd25616712b12bdb0e36c34f8d44469da4cb256070f158c54dc08e1b4037a13d] <==
I0924 01:14:24.269942 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0924 01:14:54.272040 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-654890 -n old-k8s-version-654890
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-654890 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-5qvnr
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-654890 describe pod metrics-server-9975d5f86-5qvnr
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-654890 describe pod metrics-server-9975d5f86-5qvnr: exit status 1 (106.888759ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-5qvnr" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-654890 describe pod metrics-server-9975d5f86-5qvnr: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (381.67s)