=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-135957 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
E1009 19:35:09.874417 7596 cert_rotation.go:171] "Unhandled Error" err="key failed with : open /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/addons-514774/client.crt: no such file or directory" logger="UnhandledError"
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-135957 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m18.96111765s)
-- stdout --
* [old-k8s-version-135957] minikube v1.34.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19780
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19780-2290/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19780-2290/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-135957" primary control-plane node in "old-k8s-version-135957" cluster
* Pulling base image v0.0.45-1728382586-19774 ...
* Restarting existing docker container for "old-k8s-version-135957" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-135957 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, default-storageclass, dashboard
-- /stdout --
** stderr **
I1009 19:34:38.824896 212114 out.go:345] Setting OutFile to fd 1 ...
I1009 19:34:38.825158 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:34:38.825187 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:34:38.825217 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:34:38.825488 212114 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19780-2290/.minikube/bin
I1009 19:34:38.825995 212114 out.go:352] Setting JSON to false
I1009 19:34:38.827172 212114 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":4622,"bootTime":1728497857,"procs":210,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I1009 19:34:38.827282 212114 start.go:139] virtualization:
I1009 19:34:38.830915 212114 out.go:177] * [old-k8s-version-135957] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I1009 19:34:38.834334 212114 out.go:177] - MINIKUBE_LOCATION=19780
I1009 19:34:38.834396 212114 notify.go:220] Checking for updates...
I1009 19:34:38.840239 212114 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1009 19:34:38.842840 212114 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19780-2290/kubeconfig
I1009 19:34:38.845338 212114 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19780-2290/.minikube
I1009 19:34:38.847970 212114 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I1009 19:34:38.850995 212114 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I1009 19:34:38.854044 212114 config.go:182] Loaded profile config "old-k8s-version-135957": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I1009 19:34:38.857281 212114 out.go:177] * Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
I1009 19:34:38.859851 212114 driver.go:394] Setting default libvirt URI to qemu:///system
I1009 19:34:38.907551 212114 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I1009 19:34:38.907767 212114 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1009 19:34:39.002594 212114 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:42 OomKillDisable:true NGoroutines:52 SystemTime:2024-10-09 19:34:38.988911254 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I1009 19:34:39.002721 212114 docker.go:318] overlay module found
I1009 19:34:39.005460 212114 out.go:177] * Using the docker driver based on existing profile
I1009 19:34:39.008025 212114 start.go:297] selected driver: docker
I1009 19:34:39.008043 212114 start.go:901] validating driver "docker" against &{Name:old-k8s-version-135957 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-135957 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1009 19:34:39.008173 212114 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1009 19:34:39.008873 212114 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1009 19:34:39.117217 212114 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:42 OomKillDisable:true NGoroutines:52 SystemTime:2024-10-09 19:34:39.082793792 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I1009 19:34:39.117648 212114 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I1009 19:34:39.117677 212114 cni.go:84] Creating CNI manager for ""
I1009 19:34:39.117729 212114 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I1009 19:34:39.117771 212114 start.go:340] cluster config:
{Name:old-k8s-version-135957 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-135957 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1009 19:34:39.120854 212114 out.go:177] * Starting "old-k8s-version-135957" primary control-plane node in "old-k8s-version-135957" cluster
I1009 19:34:39.123484 212114 cache.go:121] Beginning downloading kic base image for docker with containerd
I1009 19:34:39.126122 212114 out.go:177] * Pulling base image v0.0.45-1728382586-19774 ...
I1009 19:34:39.128750 212114 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I1009 19:34:39.128806 212114 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I1009 19:34:39.128816 212114 cache.go:56] Caching tarball of preloaded images
I1009 19:34:39.128902 212114 preload.go:172] Found /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I1009 19:34:39.128912 212114 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I1009 19:34:39.129035 212114 profile.go:143] Saving config to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/config.json ...
I1009 19:34:39.129299 212114 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec in local docker daemon
I1009 19:34:39.149922 212114 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec in local docker daemon, skipping pull
I1009 19:34:39.149940 212114 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec exists in daemon, skipping load
I1009 19:34:39.149953 212114 cache.go:194] Successfully downloaded all kic artifacts
I1009 19:34:39.149985 212114 start.go:360] acquireMachinesLock for old-k8s-version-135957: {Name:mk2236540877b98c3a6f3f4fce961e1e3f6adc1d Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1009 19:34:39.150037 212114 start.go:364] duration metric: took 33.362µs to acquireMachinesLock for "old-k8s-version-135957"
I1009 19:34:39.150055 212114 start.go:96] Skipping create...Using existing machine configuration
I1009 19:34:39.150061 212114 fix.go:54] fixHost starting:
I1009 19:34:39.150335 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:39.167509 212114 fix.go:112] recreateIfNeeded on old-k8s-version-135957: state=Stopped err=<nil>
W1009 19:34:39.167535 212114 fix.go:138] unexpected machine state, will restart: <nil>
I1009 19:34:39.170390 212114 out.go:177] * Restarting existing docker container for "old-k8s-version-135957" ...
I1009 19:34:39.176781 212114 cli_runner.go:164] Run: docker start old-k8s-version-135957
I1009 19:34:39.606873 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:39.635617 212114 kic.go:430] container "old-k8s-version-135957" state is running.
I1009 19:34:39.635993 212114 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-135957
I1009 19:34:39.671098 212114 profile.go:143] Saving config to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/config.json ...
I1009 19:34:39.671351 212114 machine.go:93] provisionDockerMachine start ...
I1009 19:34:39.671424 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:39.714340 212114 main.go:141] libmachine: Using SSH client type: native
I1009 19:34:39.714612 212114 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I1009 19:34:39.714627 212114 main.go:141] libmachine: About to run SSH command:
hostname
I1009 19:34:39.715252 212114 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:36186->127.0.0.1:33063: read: connection reset by peer
I1009 19:34:42.852063 212114 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-135957
I1009 19:34:42.852087 212114 ubuntu.go:169] provisioning hostname "old-k8s-version-135957"
I1009 19:34:42.852154 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:42.870448 212114 main.go:141] libmachine: Using SSH client type: native
I1009 19:34:42.871536 212114 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I1009 19:34:42.871560 212114 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-135957 && echo "old-k8s-version-135957" | sudo tee /etc/hostname
I1009 19:34:43.021362 212114 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-135957
I1009 19:34:43.021563 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:43.039374 212114 main.go:141] libmachine: Using SSH client type: native
I1009 19:34:43.040894 212114 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33063 <nil> <nil>}
I1009 19:34:43.040924 212114 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-135957' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-135957/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-135957' | sudo tee -a /etc/hosts;
fi
fi
I1009 19:34:43.172830 212114 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1009 19:34:43.172855 212114 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19780-2290/.minikube CaCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19780-2290/.minikube}
I1009 19:34:43.172877 212114 ubuntu.go:177] setting up certificates
I1009 19:34:43.172886 212114 provision.go:84] configureAuth start
I1009 19:34:43.172943 212114 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-135957
I1009 19:34:43.192083 212114 provision.go:143] copyHostCerts
I1009 19:34:43.192141 212114 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem, removing ...
I1009 19:34:43.192149 212114 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem
I1009 19:34:43.192222 212114 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem (1078 bytes)
I1009 19:34:43.192326 212114 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem, removing ...
I1009 19:34:43.192331 212114 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem
I1009 19:34:43.192357 212114 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem (1123 bytes)
I1009 19:34:43.192416 212114 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem, removing ...
I1009 19:34:43.192420 212114 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem
I1009 19:34:43.192443 212114 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem (1679 bytes)
I1009 19:34:43.192498 212114 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-135957 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-135957]
I1009 19:34:44.009452 212114 provision.go:177] copyRemoteCerts
I1009 19:34:44.009592 212114 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1009 19:34:44.009660 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:44.027364 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:44.122220 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I1009 19:34:44.147437 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I1009 19:34:44.174172 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I1009 19:34:44.198790 212114 provision.go:87] duration metric: took 1.02589123s to configureAuth
I1009 19:34:44.198821 212114 ubuntu.go:193] setting minikube options for container-runtime
I1009 19:34:44.199020 212114 config.go:182] Loaded profile config "old-k8s-version-135957": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I1009 19:34:44.199033 212114 machine.go:96] duration metric: took 4.527665607s to provisionDockerMachine
I1009 19:34:44.199042 212114 start.go:293] postStartSetup for "old-k8s-version-135957" (driver="docker")
I1009 19:34:44.199057 212114 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1009 19:34:44.199117 212114 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1009 19:34:44.199163 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:44.216046 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:44.309502 212114 ssh_runner.go:195] Run: cat /etc/os-release
I1009 19:34:44.312455 212114 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1009 19:34:44.312488 212114 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I1009 19:34:44.312499 212114 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I1009 19:34:44.312509 212114 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I1009 19:34:44.312519 212114 filesync.go:126] Scanning /home/jenkins/minikube-integration/19780-2290/.minikube/addons for local assets ...
I1009 19:34:44.312578 212114 filesync.go:126] Scanning /home/jenkins/minikube-integration/19780-2290/.minikube/files for local assets ...
I1009 19:34:44.312710 212114 filesync.go:149] local asset: /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem -> 75962.pem in /etc/ssl/certs
I1009 19:34:44.312843 212114 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I1009 19:34:44.323252 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem --> /etc/ssl/certs/75962.pem (1708 bytes)
I1009 19:34:44.347818 212114 start.go:296] duration metric: took 148.758044ms for postStartSetup
I1009 19:34:44.347920 212114 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1009 19:34:44.347968 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:44.366159 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:44.457436 212114 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1009 19:34:44.461992 212114 fix.go:56] duration metric: took 5.311924827s for fixHost
I1009 19:34:44.462019 212114 start.go:83] releasing machines lock for "old-k8s-version-135957", held for 5.311974877s
I1009 19:34:44.462095 212114 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-135957
I1009 19:34:44.479858 212114 ssh_runner.go:195] Run: cat /version.json
I1009 19:34:44.479921 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:44.479981 212114 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I1009 19:34:44.480043 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:44.506352 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:44.519004 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:44.741022 212114 ssh_runner.go:195] Run: systemctl --version
I1009 19:34:44.745590 212114 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I1009 19:34:44.749827 212114 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I1009 19:34:44.768688 212114 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I1009 19:34:44.768775 212114 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I1009 19:34:44.779363 212114 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I1009 19:34:44.779401 212114 start.go:495] detecting cgroup driver to use...
I1009 19:34:44.779432 212114 detect.go:187] detected "cgroupfs" cgroup driver on host os
I1009 19:34:44.779483 212114 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I1009 19:34:44.794142 212114 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I1009 19:34:44.805817 212114 docker.go:217] disabling cri-docker service (if available) ...
I1009 19:34:44.805897 212114 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I1009 19:34:44.820731 212114 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I1009 19:34:44.833785 212114 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I1009 19:34:44.927874 212114 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I1009 19:34:45.026675 212114 docker.go:233] disabling docker service ...
I1009 19:34:45.026761 212114 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I1009 19:34:45.045240 212114 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I1009 19:34:45.061211 212114 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I1009 19:34:45.183174 212114 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I1009 19:34:45.295743 212114 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1009 19:34:45.309686 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I1009 19:34:45.333583 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I1009 19:34:45.344593 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I1009 19:34:45.356694 212114 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I1009 19:34:45.356777 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I1009 19:34:45.367813 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1009 19:34:45.380486 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I1009 19:34:45.391115 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1009 19:34:45.401316 212114 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I1009 19:34:45.410486 212114 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I1009 19:34:45.421874 212114 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I1009 19:34:45.430979 212114 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I1009 19:34:45.440107 212114 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1009 19:34:45.546320 212114 ssh_runner.go:195] Run: sudo systemctl restart containerd
I1009 19:34:45.730395 212114 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I1009 19:34:45.730473 212114 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I1009 19:34:45.734504 212114 start.go:563] Will wait 60s for crictl version
I1009 19:34:45.734581 212114 ssh_runner.go:195] Run: which crictl
I1009 19:34:45.738380 212114 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I1009 19:34:45.789088 212114 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I1009 19:34:45.789260 212114 ssh_runner.go:195] Run: containerd --version
I1009 19:34:45.812949 212114 ssh_runner.go:195] Run: containerd --version
I1009 19:34:45.843149 212114 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
I1009 19:34:45.845826 212114 cli_runner.go:164] Run: docker network inspect old-k8s-version-135957 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1009 19:34:45.862065 212114 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I1009 19:34:45.866055 212114 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1009 19:34:45.877349 212114 kubeadm.go:883] updating cluster {Name:old-k8s-version-135957 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-135957 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I1009 19:34:45.877466 212114 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I1009 19:34:45.877523 212114 ssh_runner.go:195] Run: sudo crictl images --output json
I1009 19:34:45.916633 212114 containerd.go:627] all images are preloaded for containerd runtime.
I1009 19:34:45.916706 212114 containerd.go:534] Images already preloaded, skipping extraction
I1009 19:34:45.916766 212114 ssh_runner.go:195] Run: sudo crictl images --output json
I1009 19:34:45.952768 212114 containerd.go:627] all images are preloaded for containerd runtime.
I1009 19:34:45.952791 212114 cache_images.go:84] Images are preloaded, skipping loading
I1009 19:34:45.952799 212114 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I1009 19:34:45.952913 212114 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-135957 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-135957 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I1009 19:34:45.952980 212114 ssh_runner.go:195] Run: sudo crictl info
I1009 19:34:45.998701 212114 cni.go:84] Creating CNI manager for ""
I1009 19:34:45.998777 212114 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I1009 19:34:45.998799 212114 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I1009 19:34:45.998856 212114 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-135957 NodeName:old-k8s-version-135957 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I1009 19:34:45.999021 212114 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-135957"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I1009 19:34:45.999115 212114 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I1009 19:34:46.008631 212114 binaries.go:44] Found k8s binaries, skipping transfer
I1009 19:34:46.008725 212114 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I1009 19:34:46.018997 212114 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I1009 19:34:46.052758 212114 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I1009 19:34:46.077413 212114 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I1009 19:34:46.103283 212114 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I1009 19:34:46.107746 212114 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1009 19:34:46.120143 212114 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1009 19:34:46.217936 212114 ssh_runner.go:195] Run: sudo systemctl start kubelet
I1009 19:34:46.236988 212114 certs.go:68] Setting up /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957 for IP: 192.168.76.2
I1009 19:34:46.237007 212114 certs.go:194] generating shared ca certs ...
I1009 19:34:46.237024 212114 certs.go:226] acquiring lock for ca certs: {Name:mke6990d9a3fb276a87991bc9cbf7d64b4192c7c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:34:46.237163 212114 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19780-2290/.minikube/ca.key
I1009 19:34:46.237218 212114 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.key
I1009 19:34:46.237231 212114 certs.go:256] generating profile certs ...
I1009 19:34:46.237316 212114 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/client.key
I1009 19:34:46.237387 212114 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/apiserver.key.54c639ff
I1009 19:34:46.237435 212114 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/proxy-client.key
I1009 19:34:46.237555 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596.pem (1338 bytes)
W1009 19:34:46.237593 212114 certs.go:480] ignoring /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596_empty.pem, impossibly tiny 0 bytes
I1009 19:34:46.237604 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem (1675 bytes)
I1009 19:34:46.237630 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem (1078 bytes)
I1009 19:34:46.237657 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem (1123 bytes)
I1009 19:34:46.237697 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem (1679 bytes)
I1009 19:34:46.237749 212114 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem (1708 bytes)
I1009 19:34:46.238457 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I1009 19:34:46.266829 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I1009 19:34:46.299498 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I1009 19:34:46.331364 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I1009 19:34:46.358901 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I1009 19:34:46.392073 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I1009 19:34:46.416427 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I1009 19:34:46.444209 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/old-k8s-version-135957/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I1009 19:34:46.470712 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem --> /usr/share/ca-certificates/75962.pem (1708 bytes)
I1009 19:34:46.498396 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I1009 19:34:46.528861 212114 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596.pem --> /usr/share/ca-certificates/7596.pem (1338 bytes)
I1009 19:34:46.556502 212114 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I1009 19:34:46.584443 212114 ssh_runner.go:195] Run: openssl version
I1009 19:34:46.595794 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/75962.pem && ln -fs /usr/share/ca-certificates/75962.pem /etc/ssl/certs/75962.pem"
I1009 19:34:46.605908 212114 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/75962.pem
I1009 19:34:46.609705 212114 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Oct 9 18:57 /usr/share/ca-certificates/75962.pem
I1009 19:34:46.609800 212114 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/75962.pem
I1009 19:34:46.616801 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/75962.pem /etc/ssl/certs/3ec20f2e.0"
I1009 19:34:46.626610 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I1009 19:34:46.636315 212114 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I1009 19:34:46.639941 212114 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Oct 9 18:47 /usr/share/ca-certificates/minikubeCA.pem
I1009 19:34:46.640053 212114 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I1009 19:34:46.646928 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I1009 19:34:46.656172 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/7596.pem && ln -fs /usr/share/ca-certificates/7596.pem /etc/ssl/certs/7596.pem"
I1009 19:34:46.667050 212114 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/7596.pem
I1009 19:34:46.671955 212114 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Oct 9 18:57 /usr/share/ca-certificates/7596.pem
I1009 19:34:46.672054 212114 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/7596.pem
I1009 19:34:46.679292 212114 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/7596.pem /etc/ssl/certs/51391683.0"
I1009 19:34:46.688332 212114 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I1009 19:34:46.691840 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I1009 19:34:46.698698 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I1009 19:34:46.705495 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I1009 19:34:46.712075 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I1009 19:34:46.718835 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I1009 19:34:46.726287 212114 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I1009 19:34:46.734304 212114 kubeadm.go:392] StartCluster: {Name:old-k8s-version-135957 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-135957 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1009 19:34:46.734424 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I1009 19:34:46.734517 212114 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I1009 19:34:46.781960 212114 cri.go:89] found id: "84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:34:46.781996 212114 cri.go:89] found id: "c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:34:46.782003 212114 cri.go:89] found id: "9d9a8b7cd2768bc41f3938d3f14efd940b216c4e8b9c194ba9c196dcca4a9bf1"
I1009 19:34:46.782007 212114 cri.go:89] found id: "e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:34:46.782032 212114 cri.go:89] found id: "d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:34:46.782040 212114 cri.go:89] found id: "187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:34:46.782044 212114 cri.go:89] found id: "1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:34:46.782047 212114 cri.go:89] found id: "4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:34:46.782050 212114 cri.go:89] found id: ""
I1009 19:34:46.782123 212114 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I1009 19:34:46.800985 212114 cri.go:116] JSON = null
W1009 19:34:46.801052 212114 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I1009 19:34:46.801143 212114 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I1009 19:34:46.811044 212114 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I1009 19:34:46.811118 212114 kubeadm.go:593] restartPrimaryControlPlane start ...
I1009 19:34:46.811195 212114 ssh_runner.go:195] Run: sudo test -d /data/minikube
I1009 19:34:46.829474 212114 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I1009 19:34:46.830011 212114 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-135957" does not appear in /home/jenkins/minikube-integration/19780-2290/kubeconfig
I1009 19:34:46.830195 212114 kubeconfig.go:62] /home/jenkins/minikube-integration/19780-2290/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-135957" cluster setting kubeconfig missing "old-k8s-version-135957" context setting]
I1009 19:34:46.830601 212114 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/kubeconfig: {Name:mk88e77ecd1f863276e8fadf431093322057a8c7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:34:46.832218 212114 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I1009 19:34:46.846178 212114 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I1009 19:34:46.846262 212114 kubeadm.go:597] duration metric: took 35.122596ms to restartPrimaryControlPlane
I1009 19:34:46.846285 212114 kubeadm.go:394] duration metric: took 111.990312ms to StartCluster
I1009 19:34:46.846313 212114 settings.go:142] acquiring lock: {Name:mkf94bbff2baa0ab7fd6f65328728d4b59af8d85 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:34:46.846411 212114 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19780-2290/kubeconfig
I1009 19:34:46.847127 212114 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/kubeconfig: {Name:mk88e77ecd1f863276e8fadf431093322057a8c7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:34:46.847400 212114 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I1009 19:34:46.847800 212114 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I1009 19:34:46.847930 212114 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-135957"
I1009 19:34:46.847953 212114 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-135957"
W1009 19:34:46.847960 212114 addons.go:243] addon storage-provisioner should already be in state true
I1009 19:34:46.847992 212114 host.go:66] Checking if "old-k8s-version-135957" exists ...
I1009 19:34:46.848033 212114 addons.go:69] Setting dashboard=true in profile "old-k8s-version-135957"
I1009 19:34:46.848062 212114 addons.go:234] Setting addon dashboard=true in "old-k8s-version-135957"
W1009 19:34:46.848099 212114 addons.go:243] addon dashboard should already be in state true
I1009 19:34:46.848135 212114 host.go:66] Checking if "old-k8s-version-135957" exists ...
I1009 19:34:46.848462 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:46.848884 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:46.847860 212114 config.go:182] Loaded profile config "old-k8s-version-135957": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I1009 19:34:46.852662 212114 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-135957"
I1009 19:34:46.852690 212114 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-135957"
W1009 19:34:46.852698 212114 addons.go:243] addon metrics-server should already be in state true
I1009 19:34:46.852731 212114 host.go:66] Checking if "old-k8s-version-135957" exists ...
I1009 19:34:46.852967 212114 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-135957"
I1009 19:34:46.853045 212114 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-135957"
I1009 19:34:46.853271 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:46.853922 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:46.854281 212114 out.go:177] * Verifying Kubernetes components...
I1009 19:34:46.857226 212114 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1009 19:34:46.911524 212114 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I1009 19:34:46.914317 212114 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I1009 19:34:46.914339 212114 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I1009 19:34:46.914420 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:46.921567 212114 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I1009 19:34:46.924800 212114 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I1009 19:34:46.924827 212114 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I1009 19:34:46.924895 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:46.932717 212114 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I1009 19:34:46.935741 212114 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I1009 19:34:46.938655 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I1009 19:34:46.938679 212114 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I1009 19:34:46.938750 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:46.948490 212114 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-135957"
W1009 19:34:46.948515 212114 addons.go:243] addon default-storageclass should already be in state true
I1009 19:34:46.948540 212114 host.go:66] Checking if "old-k8s-version-135957" exists ...
I1009 19:34:46.949009 212114 cli_runner.go:164] Run: docker container inspect old-k8s-version-135957 --format={{.State.Status}}
I1009 19:34:46.996938 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:47.005024 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:47.011131 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:47.014704 212114 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I1009 19:34:47.014725 212114 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I1009 19:34:47.014789 212114 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-135957
I1009 19:34:47.043307 212114 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33063 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/old-k8s-version-135957/id_rsa Username:docker}
I1009 19:34:47.116756 212114 ssh_runner.go:195] Run: sudo systemctl start kubelet
I1009 19:34:47.172133 212114 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-135957" to be "Ready" ...
I1009 19:34:47.236536 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I1009 19:34:47.236608 212114 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I1009 19:34:47.259084 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I1009 19:34:47.271183 212114 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I1009 19:34:47.271252 212114 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I1009 19:34:47.283754 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I1009 19:34:47.283826 212114 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I1009 19:34:47.289765 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I1009 19:34:47.317187 212114 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I1009 19:34:47.317264 212114 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I1009 19:34:47.342379 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I1009 19:34:47.342523 212114 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I1009 19:34:47.364364 212114 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:34:47.364442 212114 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I1009 19:34:47.400758 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:34:47.425465 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I1009 19:34:47.425537 212114 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W1009 19:34:47.463411 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.463508 212114 retry.go:31] will retry after 139.499051ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:47.512292 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.512375 212114 retry.go:31] will retry after 266.723085ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.528517 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I1009 19:34:47.528601 212114 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W1009 19:34:47.592971 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.593048 212114 retry.go:31] will retry after 137.850195ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.597785 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I1009 19:34:47.597852 212114 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I1009 19:34:47.604048 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I1009 19:34:47.625412 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I1009 19:34:47.625493 212114 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I1009 19:34:47.673486 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I1009 19:34:47.673505 212114 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I1009 19:34:47.732232 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:34:47.743310 212114 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I1009 19:34:47.743331 212114 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I1009 19:34:47.780030 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I1009 19:34:47.872167 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W1009 19:34:47.924243 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:47.924273 212114 retry.go:31] will retry after 227.608527ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:48.047832 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.047865 212114 retry.go:31] will retry after 384.361719ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:48.114785 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.114817 212114 retry.go:31] will retry after 314.128417ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.152061 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:48.202914 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.202996 212114 retry.go:31] will retry after 202.419151ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:48.315466 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.315496 212114 retry.go:31] will retry after 410.438566ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.405859 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I1009 19:34:48.429263 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I1009 19:34:48.432569 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W1009 19:34:48.609970 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.610001 212114 retry.go:31] will retry after 284.565711ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.726665 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:48.760102 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.760131 212114 retry.go:31] will retry after 805.81663ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:48.771788 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.771893 212114 retry.go:31] will retry after 592.703604ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:48.878317 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.878407 212114 retry.go:31] will retry after 926.390912ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:48.895594 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W1009 19:34:49.015844 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.015874 212114 retry.go:31] will retry after 430.309237ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.173360 212114 node_ready.go:53] error getting node "old-k8s-version-135957": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-135957": dial tcp 192.168.76.2:8443: connect: connection refused
I1009 19:34:49.364824 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:34:49.447244 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W1009 19:34:49.545128 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.545160 212114 retry.go:31] will retry after 489.616691ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.566482 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W1009 19:34:49.655213 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.655299 212114 retry.go:31] will retry after 1.241653882s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:49.729626 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.729666 212114 retry.go:31] will retry after 874.082366ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.804949 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:49.925300 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:49.925329 212114 retry.go:31] will retry after 956.334378ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.035573 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W1009 19:34:50.161633 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.161663 212114 retry.go:31] will retry after 653.534014ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.604305 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W1009 19:34:50.702037 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.702067 212114 retry.go:31] will retry after 1.765692181s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.815388 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:34:50.882740 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:50.895133 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.895240 212114 retry.go:31] will retry after 2.524115187s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.897440 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W1009 19:34:50.987091 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:50.987176 212114 retry.go:31] will retry after 1.119374933s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:51.003517 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:51.003598 212114 retry.go:31] will retry after 1.668770369s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:51.672897 212114 node_ready.go:53] error getting node "old-k8s-version-135957": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-135957": dial tcp 192.168.76.2:8443: connect: connection refused
I1009 19:34:52.107450 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:52.193587 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:52.193618 212114 retry.go:31] will retry after 2.973156936s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:52.467992 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W1009 19:34:52.543758 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:52.543792 212114 retry.go:31] will retry after 1.955838845s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:52.673052 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W1009 19:34:52.770888 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:52.770935 212114 retry.go:31] will retry after 2.658454725s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:53.419606 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W1009 19:34:53.490077 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:53.490107 212114 retry.go:31] will retry after 1.953527339s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:53.673709 212114 node_ready.go:53] error getting node "old-k8s-version-135957": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-135957": dial tcp 192.168.76.2:8443: connect: connection refused
I1009 19:34:54.500535 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W1009 19:34:54.577602 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:54.577635 212114 retry.go:31] will retry after 2.888783758s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:55.167879 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W1009 19:34:55.271762 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:55.271797 212114 retry.go:31] will retry after 5.209963544s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:55.430094 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I1009 19:34:55.444473 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W1009 19:34:55.660046 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:55.660091 212114 retry.go:31] will retry after 4.07372593s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W1009 19:34:55.742773 212114 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:55.742814 212114 retry.go:31] will retry after 6.05131176s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I1009 19:34:57.467363 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I1009 19:34:59.734246 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I1009 19:35:00.482173 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I1009 19:35:01.794270 212114 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I1009 19:35:03.145358 212114 node_ready.go:49] node "old-k8s-version-135957" has status "Ready":"True"
I1009 19:35:03.145441 212114 node_ready.go:38] duration metric: took 15.973101039s for node "old-k8s-version-135957" to be "Ready" ...
I1009 19:35:03.145469 212114 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I1009 19:35:03.331109 212114 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-txr44" in "kube-system" namespace to be "Ready" ...
I1009 19:35:03.422519 212114 pod_ready.go:93] pod "coredns-74ff55c5b-txr44" in "kube-system" namespace has status "Ready":"True"
I1009 19:35:03.422559 212114 pod_ready.go:82] duration metric: took 91.403457ms for pod "coredns-74ff55c5b-txr44" in "kube-system" namespace to be "Ready" ...
I1009 19:35:03.422576 212114 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:35:03.452947 212114 pod_ready.go:93] pod "etcd-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"True"
I1009 19:35:03.452976 212114 pod_ready.go:82] duration metric: took 30.392533ms for pod "etcd-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:35:03.453004 212114 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:35:04.130343 212114 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (6.66293545s)
I1009 19:35:04.384317 212114 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (3.902106009s)
I1009 19:35:04.384564 212114 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (2.590267312s)
I1009 19:35:04.384579 212114 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-135957"
I1009 19:35:04.384858 212114 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (4.650571893s)
I1009 19:35:04.391037 212114 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-135957 addons enable metrics-server
I1009 19:35:04.398048 212114 out.go:177] * Enabled addons: storage-provisioner, metrics-server, default-storageclass, dashboard
I1009 19:35:04.403665 212114 addons.go:510] duration metric: took 17.55586602s for enable addons: enabled=[storage-provisioner metrics-server default-storageclass dashboard]
I1009 19:35:05.460363 212114 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:07.959110 212114 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:09.959290 212114 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:10.459413 212114 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"True"
I1009 19:35:10.459449 212114 pod_ready.go:82] duration metric: took 7.006434734s for pod "kube-apiserver-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:35:10.459462 212114 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:35:12.465922 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:14.466766 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:16.966605 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:18.968826 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:21.465826 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:23.466520 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:25.467116 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:27.474872 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:30.010354 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:32.466074 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:34.466453 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:36.965976 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:38.966577 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:41.466744 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:43.966276 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:45.967502 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:47.970619 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:50.466689 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:52.965830 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:54.967028 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:56.967317 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:35:59.465959 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:01.466542 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:03.475543 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:05.966082 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:07.966953 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:09.967729 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:12.465358 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:14.465701 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:16.466029 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:18.468818 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:20.473427 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:22.965563 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:24.966658 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:27.465944 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:29.966658 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:32.465892 212114 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:32.966211 212114 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"True"
I1009 19:36:32.966236 212114 pod_ready.go:82] duration metric: took 1m22.506766871s for pod "kube-controller-manager-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:36:32.966247 212114 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-whqjp" in "kube-system" namespace to be "Ready" ...
I1009 19:36:32.971113 212114 pod_ready.go:93] pod "kube-proxy-whqjp" in "kube-system" namespace has status "Ready":"True"
I1009 19:36:32.971137 212114 pod_ready.go:82] duration metric: took 4.88284ms for pod "kube-proxy-whqjp" in "kube-system" namespace to be "Ready" ...
I1009 19:36:32.971149 212114 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:36:32.976121 212114 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-135957" in "kube-system" namespace has status "Ready":"True"
I1009 19:36:32.976146 212114 pod_ready.go:82] duration metric: took 4.990071ms for pod "kube-scheduler-old-k8s-version-135957" in "kube-system" namespace to be "Ready" ...
I1009 19:36:32.976161 212114 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace to be "Ready" ...
I1009 19:36:34.983013 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:36.983310 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:39.481985 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:41.482347 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:43.482830 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:45.982535 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:48.482688 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:50.983582 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:53.483128 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:55.982674 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:36:58.482852 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:00.483305 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:02.483460 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:04.982053 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:06.982923 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:09.482950 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:11.982032 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:13.982504 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:16.482515 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:18.482904 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:20.982286 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:23.481655 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:25.482400 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:27.983483 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:30.481842 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:32.482228 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:34.981881 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:36.983064 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:39.482979 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:41.982395 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:43.983182 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:45.985047 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:48.483002 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:50.982728 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:53.482282 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:55.982437 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:37:58.482599 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:00.984574 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:03.481827 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:05.482673 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:07.983340 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:10.482490 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:12.981861 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:14.981979 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:16.982490 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:18.982570 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:20.982816 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:23.482616 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:25.982556 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:28.481950 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:30.482563 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:32.483018 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:34.982127 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:36.982199 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:39.482903 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:41.982942 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:43.983027 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:46.482110 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:48.981592 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:50.983003 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:53.482477 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:55.482900 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:57.982613 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:38:59.983145 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:02.483201 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:04.484390 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:06.982213 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:09.482361 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:11.982789 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:14.482182 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:16.483677 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:18.982036 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:20.984014 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:23.482632 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:25.987516 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:28.491126 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:30.982136 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:33.482504 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:35.983022 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:37.983910 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:40.483859 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:42.982196 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:44.982447 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:47.482219 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:49.982340 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:51.982896 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:54.482386 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:56.482713 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:39:58.983853 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:01.487091 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:03.982593 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:05.984262 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:08.481984 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:10.482535 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:12.981573 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:14.983311 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:16.983856 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:18.986323 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:21.484022 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:23.983697 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:26.483244 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:28.484264 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:30.985198 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:32.983535 212114 pod_ready.go:82] duration metric: took 4m0.007360452s for pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace to be "Ready" ...
E1009 19:40:32.983561 212114 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I1009 19:40:32.983570 212114 pod_ready.go:39] duration metric: took 5m29.838077957s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I1009 19:40:32.983586 212114 api_server.go:52] waiting for apiserver process to appear ...
I1009 19:40:32.983616 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I1009 19:40:32.983677 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I1009 19:40:33.049530 212114 cri.go:89] found id: "a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:33.049553 212114 cri.go:89] found id: "4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:33.049558 212114 cri.go:89] found id: ""
I1009 19:40:33.049566 212114 logs.go:282] 2 containers: [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797]
I1009 19:40:33.049621 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.062880 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.069016 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I1009 19:40:33.069102 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I1009 19:40:33.148317 212114 cri.go:89] found id: "5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:33.148337 212114 cri.go:89] found id: "1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:33.148342 212114 cri.go:89] found id: ""
I1009 19:40:33.148356 212114 logs.go:282] 2 containers: [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f]
I1009 19:40:33.148419 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.152544 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.158280 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I1009 19:40:33.158385 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I1009 19:40:33.250632 212114 cri.go:89] found id: "63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:33.250656 212114 cri.go:89] found id: "84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:33.250662 212114 cri.go:89] found id: ""
I1009 19:40:33.250669 212114 logs.go:282] 2 containers: [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4]
I1009 19:40:33.250761 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.254707 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.258160 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I1009 19:40:33.258276 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I1009 19:40:33.329677 212114 cri.go:89] found id: "855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:33.329698 212114 cri.go:89] found id: "187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:33.329703 212114 cri.go:89] found id: ""
I1009 19:40:33.329711 212114 logs.go:282] 2 containers: [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e]
I1009 19:40:33.329770 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.335537 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.341688 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I1009 19:40:33.341770 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I1009 19:40:33.409335 212114 cri.go:89] found id: "4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:33.409361 212114 cri.go:89] found id: "e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:33.409365 212114 cri.go:89] found id: ""
I1009 19:40:33.409372 212114 logs.go:282] 2 containers: [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e]
I1009 19:40:33.409428 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.414175 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.418573 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I1009 19:40:33.418657 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I1009 19:40:33.480197 212114 cri.go:89] found id: "a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:33.480228 212114 cri.go:89] found id: "d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:33.480232 212114 cri.go:89] found id: ""
I1009 19:40:33.480240 212114 logs.go:282] 2 containers: [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4]
I1009 19:40:33.480322 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.484759 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.489087 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I1009 19:40:33.489206 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I1009 19:40:33.557450 212114 cri.go:89] found id: "c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:33.557473 212114 cri.go:89] found id: "c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:33.557479 212114 cri.go:89] found id: ""
I1009 19:40:33.557685 212114 logs.go:282] 2 containers: [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c]
I1009 19:40:33.557763 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.564818 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.574510 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I1009 19:40:33.574618 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I1009 19:40:33.671672 212114 cri.go:89] found id: "ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:33.671726 212114 cri.go:89] found id: ""
I1009 19:40:33.671736 212114 logs.go:282] 1 containers: [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60]
I1009 19:40:33.671806 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.677087 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I1009 19:40:33.677159 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I1009 19:40:33.763257 212114 cri.go:89] found id: "70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:33.763285 212114 cri.go:89] found id: "932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:33.763291 212114 cri.go:89] found id: ""
I1009 19:40:33.763298 212114 logs.go:282] 2 containers: [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f]
I1009 19:40:33.763356 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.767028 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.770406 212114 logs.go:123] Gathering logs for etcd [1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f] ...
I1009 19:40:33.770425 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:33.826020 212114 logs.go:123] Gathering logs for coredns [84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4] ...
I1009 19:40:33.826064 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:33.879106 212114 logs.go:123] Gathering logs for kube-scheduler [187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e] ...
I1009 19:40:33.879135 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:33.930392 212114 logs.go:123] Gathering logs for containerd ...
I1009 19:40:33.930428 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I1009 19:40:34.004770 212114 logs.go:123] Gathering logs for container status ...
I1009 19:40:34.004799 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I1009 19:40:34.056516 212114 logs.go:123] Gathering logs for kubelet ...
I1009 19:40:34.056599 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W1009 19:40:34.117909 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141088 655 reflector.go:138] object-"kube-system"/"kube-proxy-token-l28zw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l28zw" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118146 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141226 655 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118362 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141301 655 reflector.go:138] object-"kube-system"/"kindnet-token-ch425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ch425" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118562 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141420 655 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118773 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141526 655 reflector.go:138] object-"kube-system"/"coredns-token-x8mx9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-x8mx9" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118996 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.145947 655 reflector.go:138] object-"kube-system"/"metrics-server-token-n5d8s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-n5d8s" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.119211 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146058 655 reflector.go:138] object-"default"/"default-token-dtkng": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-dtkng" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.119436 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146124 655 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8fzd6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8fzd6" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.127887 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.568696 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.129333 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.908103 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.133313 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:18 old-k8s-version-135957 kubelet[655]: E1009 19:35:18.807162 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.135414 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:27 old-k8s-version-135957 kubelet[655]: E1009 19:35:27.017732 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.135765 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:28 old-k8s-version-135957 kubelet[655]: E1009 19:35:28.030088 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.135969 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:29 old-k8s-version-135957 kubelet[655]: E1009 19:35:29.798736 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.136376 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:34 old-k8s-version-135957 kubelet[655]: E1009 19:35:34.224840 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.137166 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:36 old-k8s-version-135957 kubelet[655]: E1009 19:35:36.070564 655 pod_workers.go:191] Error syncing pod dbfd3538-0cb4-4cf0-b208-e18c725f6d5d ("storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"
W1009 19:40:34.139585 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:40 old-k8s-version-135957 kubelet[655]: E1009 19:35:40.810507 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.140516 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:48 old-k8s-version-135957 kubelet[655]: E1009 19:35:48.113466 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.140979 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.225991 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.141166 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.805911 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.141347 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:07 old-k8s-version-135957 kubelet[655]: E1009 19:36:07.794321 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.141929 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:10 old-k8s-version-135957 kubelet[655]: E1009 19:36:10.183835 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.142254 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:14 old-k8s-version-135957 kubelet[655]: E1009 19:36:14.226874 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.142436 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:20 old-k8s-version-135957 kubelet[655]: E1009 19:36:20.794329 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.142759 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:29 old-k8s-version-135957 kubelet[655]: E1009 19:36:29.793939 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.145371 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:33 old-k8s-version-135957 kubelet[655]: E1009 19:36:33.803424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.145703 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:41 old-k8s-version-135957 kubelet[655]: E1009 19:36:41.794984 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.145888 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:44 old-k8s-version-135957 kubelet[655]: E1009 19:36:44.794752 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.146482 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.299505 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.146665 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.794264 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.146989 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:04 old-k8s-version-135957 kubelet[655]: E1009 19:37:04.225263 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.147171 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:10 old-k8s-version-135957 kubelet[655]: E1009 19:37:10.794424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.147494 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:19 old-k8s-version-135957 kubelet[655]: E1009 19:37:19.794014 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.147679 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:25 old-k8s-version-135957 kubelet[655]: E1009 19:37:25.794260 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.148013 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:33 old-k8s-version-135957 kubelet[655]: E1009 19:37:33.794463 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.148203 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:36 old-k8s-version-135957 kubelet[655]: E1009 19:37:36.794403 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.148534 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:46 old-k8s-version-135957 kubelet[655]: E1009 19:37:46.794633 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.148729 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:48 old-k8s-version-135957 kubelet[655]: E1009 19:37:48.796873 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.149055 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:59 old-k8s-version-135957 kubelet[655]: E1009 19:37:59.794479 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.151483 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:03 old-k8s-version-135957 kubelet[655]: E1009 19:38:03.802412 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.152171 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:12 old-k8s-version-135957 kubelet[655]: E1009 19:38:12.799928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.152410 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:15 old-k8s-version-135957 kubelet[655]: E1009 19:38:15.794598 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.153011 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:27 old-k8s-version-135957 kubelet[655]: E1009 19:38:27.530323 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.153194 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:28 old-k8s-version-135957 kubelet[655]: E1009 19:38:28.795022 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.153519 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:34 old-k8s-version-135957 kubelet[655]: E1009 19:38:34.226016 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.153707 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:39 old-k8s-version-135957 kubelet[655]: E1009 19:38:39.794229 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154038 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:49 old-k8s-version-135957 kubelet[655]: E1009 19:38:49.795059 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.154225 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:51 old-k8s-version-135957 kubelet[655]: E1009 19:38:51.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154565 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.793928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.154754 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.795042 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154941 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:16 old-k8s-version-135957 kubelet[655]: E1009 19:39:16.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.155340 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:18 old-k8s-version-135957 kubelet[655]: E1009 19:39:18.794102 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.155552 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:27 old-k8s-version-135957 kubelet[655]: E1009 19:39:27.794317 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.155882 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: E1009 19:39:30.794130 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156066 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:42 old-k8s-version-135957 kubelet[655]: E1009 19:39:42.794279 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.156436 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: E1009 19:39:43.794072 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156775 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.806354 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156959 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.812240 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157144 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157467 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.157651 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157977 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.158159 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:34.158169 212114 logs.go:123] Gathering logs for kube-apiserver [4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797] ...
I1009 19:40:34.158184 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:34.267905 212114 logs.go:123] Gathering logs for etcd [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a] ...
I1009 19:40:34.267941 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:34.313520 212114 logs.go:123] Gathering logs for kube-proxy [e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e] ...
I1009 19:40:34.313547 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:34.369410 212114 logs.go:123] Gathering logs for kube-controller-manager [d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4] ...
I1009 19:40:34.369434 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:34.435867 212114 logs.go:123] Gathering logs for coredns [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33] ...
I1009 19:40:34.435938 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:34.477407 212114 logs.go:123] Gathering logs for kube-scheduler [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd] ...
I1009 19:40:34.477432 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:34.525043 212114 logs.go:123] Gathering logs for kube-proxy [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454] ...
I1009 19:40:34.525071 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:34.579092 212114 logs.go:123] Gathering logs for kindnet [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9] ...
I1009 19:40:34.579120 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:34.645088 212114 logs.go:123] Gathering logs for kindnet [c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c] ...
I1009 19:40:34.645120 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:34.688041 212114 logs.go:123] Gathering logs for kubernetes-dashboard [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60] ...
I1009 19:40:34.688076 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:34.750518 212114 logs.go:123] Gathering logs for storage-provisioner [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c] ...
I1009 19:40:34.750548 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:34.792366 212114 logs.go:123] Gathering logs for storage-provisioner [932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f] ...
I1009 19:40:34.792401 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:34.844432 212114 logs.go:123] Gathering logs for dmesg ...
I1009 19:40:34.844506 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1009 19:40:34.862899 212114 logs.go:123] Gathering logs for kube-apiserver [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880] ...
I1009 19:40:34.862973 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:34.966604 212114 logs.go:123] Gathering logs for kube-controller-manager [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184] ...
I1009 19:40:34.966676 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:35.078882 212114 logs.go:123] Gathering logs for describe nodes ...
I1009 19:40:35.078925 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I1009 19:40:35.317341 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:35.317406 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
W1009 19:40:35.317493 212114 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W1009 19:40:35.317536 212114 out.go:270] Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:35.317592 212114 out.go:270] Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:35.317630 212114 out.go:270] Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:35.317662 212114 out.go:270] Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:35.317702 212114 out.go:270] Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:35.317738 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:35.317758 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:45.319566 212114 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I1009 19:40:45.334167 212114 api_server.go:72] duration metric: took 5m58.486701724s to wait for apiserver process to appear ...
I1009 19:40:45.334193 212114 api_server.go:88] waiting for apiserver healthz status ...
I1009 19:40:45.334230 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I1009 19:40:45.334291 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I1009 19:40:45.404349 212114 cri.go:89] found id: "a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:45.404374 212114 cri.go:89] found id: "4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:45.404379 212114 cri.go:89] found id: ""
I1009 19:40:45.404394 212114 logs.go:282] 2 containers: [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797]
I1009 19:40:45.404454 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.408983 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.413035 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I1009 19:40:45.413099 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I1009 19:40:45.466616 212114 cri.go:89] found id: "5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:45.466636 212114 cri.go:89] found id: "1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:45.466641 212114 cri.go:89] found id: ""
I1009 19:40:45.466651 212114 logs.go:282] 2 containers: [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f]
I1009 19:40:45.466707 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.470602 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.474342 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I1009 19:40:45.474415 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I1009 19:40:45.536508 212114 cri.go:89] found id: "63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:45.536533 212114 cri.go:89] found id: "84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:45.536539 212114 cri.go:89] found id: ""
I1009 19:40:45.536547 212114 logs.go:282] 2 containers: [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4]
I1009 19:40:45.536606 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.542255 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.546144 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I1009 19:40:45.546224 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I1009 19:40:45.610102 212114 cri.go:89] found id: "855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:45.610126 212114 cri.go:89] found id: "187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:45.610131 212114 cri.go:89] found id: ""
I1009 19:40:45.610138 212114 logs.go:282] 2 containers: [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e]
I1009 19:40:45.610196 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.614464 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.618470 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I1009 19:40:45.618545 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I1009 19:40:45.669697 212114 cri.go:89] found id: "4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:45.669720 212114 cri.go:89] found id: "e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:45.669726 212114 cri.go:89] found id: ""
I1009 19:40:45.669733 212114 logs.go:282] 2 containers: [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e]
I1009 19:40:45.669794 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.676366 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.680603 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I1009 19:40:45.680692 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I1009 19:40:45.749148 212114 cri.go:89] found id: "a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:45.749173 212114 cri.go:89] found id: "d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:45.749178 212114 cri.go:89] found id: ""
I1009 19:40:45.749185 212114 logs.go:282] 2 containers: [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4]
I1009 19:40:45.749242 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.753756 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.757836 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I1009 19:40:45.757914 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I1009 19:40:45.935969 212114 cri.go:89] found id: "c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:45.935996 212114 cri.go:89] found id: "c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:45.936001 212114 cri.go:89] found id: ""
I1009 19:40:45.936008 212114 logs.go:282] 2 containers: [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c]
I1009 19:40:45.936065 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.940437 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.953098 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I1009 19:40:45.953178 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I1009 19:40:46.012160 212114 cri.go:89] found id: "ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:46.012185 212114 cri.go:89] found id: ""
I1009 19:40:46.012193 212114 logs.go:282] 1 containers: [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60]
I1009 19:40:46.012253 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.016624 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I1009 19:40:46.016713 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I1009 19:40:46.083357 212114 cri.go:89] found id: "70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:46.083383 212114 cri.go:89] found id: "932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:46.083388 212114 cri.go:89] found id: ""
I1009 19:40:46.083396 212114 logs.go:282] 2 containers: [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f]
I1009 19:40:46.083456 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.087674 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.091613 212114 logs.go:123] Gathering logs for storage-provisioner [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c] ...
I1009 19:40:46.091642 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:46.144851 212114 logs.go:123] Gathering logs for etcd [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a] ...
I1009 19:40:46.144890 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:46.214756 212114 logs.go:123] Gathering logs for coredns [84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4] ...
I1009 19:40:46.214788 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:46.268020 212114 logs.go:123] Gathering logs for kube-scheduler [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd] ...
I1009 19:40:46.268058 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:46.318517 212114 logs.go:123] Gathering logs for kube-scheduler [187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e] ...
I1009 19:40:46.318547 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:46.374095 212114 logs.go:123] Gathering logs for kube-controller-manager [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184] ...
I1009 19:40:46.374136 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:46.460506 212114 logs.go:123] Gathering logs for kindnet [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9] ...
I1009 19:40:46.460541 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:46.524819 212114 logs.go:123] Gathering logs for kindnet [c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c] ...
I1009 19:40:46.524849 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:46.583620 212114 logs.go:123] Gathering logs for kubelet ...
I1009 19:40:46.583702 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W1009 19:40:46.644048 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141088 655 reflector.go:138] object-"kube-system"/"kube-proxy-token-l28zw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l28zw" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644351 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141226 655 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644618 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141301 655 reflector.go:138] object-"kube-system"/"kindnet-token-ch425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ch425" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644879 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141420 655 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645133 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141526 655 reflector.go:138] object-"kube-system"/"coredns-token-x8mx9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-x8mx9" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645400 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.145947 655 reflector.go:138] object-"kube-system"/"metrics-server-token-n5d8s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-n5d8s" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645645 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146058 655 reflector.go:138] object-"default"/"default-token-dtkng": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-dtkng" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645924 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146124 655 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8fzd6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8fzd6" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.654238 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.568696 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.655773 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.908103 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.658740 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:18 old-k8s-version-135957 kubelet[655]: E1009 19:35:18.807162 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.660983 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:27 old-k8s-version-135957 kubelet[655]: E1009 19:35:27.017732 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.661371 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:28 old-k8s-version-135957 kubelet[655]: E1009 19:35:28.030088 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.661645 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:29 old-k8s-version-135957 kubelet[655]: E1009 19:35:29.798736 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.662027 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:34 old-k8s-version-135957 kubelet[655]: E1009 19:35:34.224840 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.662893 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:36 old-k8s-version-135957 kubelet[655]: E1009 19:35:36.070564 655 pod_workers.go:191] Error syncing pod dbfd3538-0cb4-4cf0-b208-e18c725f6d5d ("storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"
W1009 19:40:46.665881 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:40 old-k8s-version-135957 kubelet[655]: E1009 19:35:40.810507 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.666940 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:48 old-k8s-version-135957 kubelet[655]: E1009 19:35:48.113466 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.667468 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.225991 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.667683 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.805911 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.667944 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:07 old-k8s-version-135957 kubelet[655]: E1009 19:36:07.794321 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.668632 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:10 old-k8s-version-135957 kubelet[655]: E1009 19:36:10.183835 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.669178 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:14 old-k8s-version-135957 kubelet[655]: E1009 19:36:14.226874 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.669376 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:20 old-k8s-version-135957 kubelet[655]: E1009 19:36:20.794329 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.669743 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:29 old-k8s-version-135957 kubelet[655]: E1009 19:36:29.793939 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.672283 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:33 old-k8s-version-135957 kubelet[655]: E1009 19:36:33.803424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.672629 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:41 old-k8s-version-135957 kubelet[655]: E1009 19:36:41.794984 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.672973 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:44 old-k8s-version-135957 kubelet[655]: E1009 19:36:44.794752 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.673626 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.299505 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.673854 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.794264 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.674216 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:04 old-k8s-version-135957 kubelet[655]: E1009 19:37:04.225263 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.674431 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:10 old-k8s-version-135957 kubelet[655]: E1009 19:37:10.794424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.674804 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:19 old-k8s-version-135957 kubelet[655]: E1009 19:37:19.794014 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.675040 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:25 old-k8s-version-135957 kubelet[655]: E1009 19:37:25.794260 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.675400 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:33 old-k8s-version-135957 kubelet[655]: E1009 19:37:33.794463 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.675661 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:36 old-k8s-version-135957 kubelet[655]: E1009 19:37:36.794403 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.676031 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:46 old-k8s-version-135957 kubelet[655]: E1009 19:37:46.794633 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.676261 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:48 old-k8s-version-135957 kubelet[655]: E1009 19:37:48.796873 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.676632 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:59 old-k8s-version-135957 kubelet[655]: E1009 19:37:59.794479 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.679482 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:03 old-k8s-version-135957 kubelet[655]: E1009 19:38:03.802412 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.679873 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:12 old-k8s-version-135957 kubelet[655]: E1009 19:38:12.799928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.680105 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:15 old-k8s-version-135957 kubelet[655]: E1009 19:38:15.794598 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.680775 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:27 old-k8s-version-135957 kubelet[655]: E1009 19:38:27.530323 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.680993 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:28 old-k8s-version-135957 kubelet[655]: E1009 19:38:28.795022 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.681361 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:34 old-k8s-version-135957 kubelet[655]: E1009 19:38:34.226016 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.681590 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:39 old-k8s-version-135957 kubelet[655]: E1009 19:38:39.794229 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.681999 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:49 old-k8s-version-135957 kubelet[655]: E1009 19:38:49.795059 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.682230 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:51 old-k8s-version-135957 kubelet[655]: E1009 19:38:51.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.682621 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.793928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.682835 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.795042 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.683069 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:16 old-k8s-version-135957 kubelet[655]: E1009 19:39:16.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.683468 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:18 old-k8s-version-135957 kubelet[655]: E1009 19:39:18.794102 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.683703 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:27 old-k8s-version-135957 kubelet[655]: E1009 19:39:27.794317 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.684106 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: E1009 19:39:30.794130 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.684324 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:42 old-k8s-version-135957 kubelet[655]: E1009 19:39:42.794279 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.684724 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: E1009 19:39:43.794072 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.685423 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.806354 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.686352 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.812240 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.687361 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.687730 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.687961 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.688346 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.688589 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.688999 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.692433 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:46.692450 212114 logs.go:123] Gathering logs for kube-apiserver [4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797] ...
I1009 19:40:46.692464 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:46.775228 212114 logs.go:123] Gathering logs for etcd [1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f] ...
I1009 19:40:46.775261 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:46.843804 212114 logs.go:123] Gathering logs for kube-controller-manager [d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4] ...
I1009 19:40:46.843886 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:46.910698 212114 logs.go:123] Gathering logs for containerd ...
I1009 19:40:46.910734 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I1009 19:40:46.996012 212114 logs.go:123] Gathering logs for dmesg ...
I1009 19:40:46.996052 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1009 19:40:47.015084 212114 logs.go:123] Gathering logs for kube-apiserver [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880] ...
I1009 19:40:47.015112 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:47.107848 212114 logs.go:123] Gathering logs for kubernetes-dashboard [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60] ...
I1009 19:40:47.107928 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:47.154334 212114 logs.go:123] Gathering logs for storage-provisioner [932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f] ...
I1009 19:40:47.154412 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:47.203396 212114 logs.go:123] Gathering logs for describe nodes ...
I1009 19:40:47.203474 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I1009 19:40:47.462255 212114 logs.go:123] Gathering logs for coredns [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33] ...
I1009 19:40:47.462327 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:47.515099 212114 logs.go:123] Gathering logs for kube-proxy [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454] ...
I1009 19:40:47.515182 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:47.569213 212114 logs.go:123] Gathering logs for kube-proxy [e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e] ...
I1009 19:40:47.569289 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:47.620464 212114 logs.go:123] Gathering logs for container status ...
I1009 19:40:47.620545 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I1009 19:40:47.675639 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:47.675797 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
W1009 19:40:47.675885 212114 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W1009 19:40:47.675951 212114 out.go:270] Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:47.675995 212114 out.go:270] Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:47.676047 212114 out.go:270] Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:47.676086 212114 out.go:270] Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:47.676137 212114 out.go:270] Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:47.676196 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:47.676228 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:57.677635 212114 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I1009 19:40:57.689852 212114 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I1009 19:40:57.692719 212114 out.go:201]
W1009 19:40:57.694057 212114 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W1009 19:40:57.694098 212114 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W1009 19:40:57.694120 212114 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W1009 19:40:57.694126 212114 out.go:270] *
*
W1009 19:40:57.695133 212114 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1009 19:40:57.697281 212114 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-135957 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-135957
helpers_test.go:235: (dbg) docker inspect old-k8s-version-135957:
-- stdout --
[
{
"Id": "9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4",
"Created": "2024-10-09T19:31:45.624893661Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 212309,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-10-09T19:34:39.330687036Z",
"FinishedAt": "2024-10-09T19:34:38.058767221Z"
},
"Image": "sha256:e5ca9b83e048da5ecbd9864892b13b9f06d661ec5eae41590141157c6fe62bf7",
"ResolvConfPath": "/var/lib/docker/containers/9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4/hostname",
"HostsPath": "/var/lib/docker/containers/9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4/hosts",
"LogPath": "/var/lib/docker/containers/9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4/9dcb067299db8a96cc9f7de24ff02d455c700da52eeecfd279d8e0b31b9f7cb4-json.log",
"Name": "/old-k8s-version-135957",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-135957:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-135957",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/2ce229899914a4c92a1d027b36098efdde96e93e1e39e3832306b4892ad80d83-init/diff:/var/lib/docker/overlay2/b874d444a15868350f8fd5f52e8f0ed756efd8ce6e723f3b60197aecd7f71b6f/diff",
"MergedDir": "/var/lib/docker/overlay2/2ce229899914a4c92a1d027b36098efdde96e93e1e39e3832306b4892ad80d83/merged",
"UpperDir": "/var/lib/docker/overlay2/2ce229899914a4c92a1d027b36098efdde96e93e1e39e3832306b4892ad80d83/diff",
"WorkDir": "/var/lib/docker/overlay2/2ce229899914a4c92a1d027b36098efdde96e93e1e39e3832306b4892ad80d83/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-135957",
"Source": "/var/lib/docker/volumes/old-k8s-version-135957/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-135957",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-135957",
"name.minikube.sigs.k8s.io": "old-k8s-version-135957",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "709597d7178708ad81e768ce3de65f41ef2090eb5f67240a5305d04d3e6ebd22",
"SandboxKey": "/var/run/docker/netns/709597d71787",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33063"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33064"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33067"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33065"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33066"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-135957": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "e0ecbcb012de88031158533d719b809fba94a66747c8e5e14b6a79021f77b92a",
"EndpointID": "5d11a65746d3263a1d48d80c054b74eb86ef0e29c0b6682413e444a50a1fffda",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-135957",
"9dcb067299db"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-135957 -n old-k8s-version-135957
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-135957 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-135957 logs -n 25: (2.364405202s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
| start | -p cert-expiration-383078 | cert-expiration-383078 | jenkins | v1.34.0 | 09 Oct 24 19:30 UTC | 09 Oct 24 19:31 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-872019 | force-systemd-env-872019 | jenkins | v1.34.0 | 09 Oct 24 19:30 UTC | 09 Oct 24 19:30 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-872019 | force-systemd-env-872019 | jenkins | v1.34.0 | 09 Oct 24 19:30 UTC | 09 Oct 24 19:31 UTC |
| start | -p cert-options-480357 | cert-options-480357 | jenkins | v1.34.0 | 09 Oct 24 19:31 UTC | 09 Oct 24 19:31 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-480357 ssh | cert-options-480357 | jenkins | v1.34.0 | 09 Oct 24 19:31 UTC | 09 Oct 24 19:31 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-480357 -- sudo | cert-options-480357 | jenkins | v1.34.0 | 09 Oct 24 19:31 UTC | 09 Oct 24 19:31 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-480357 | cert-options-480357 | jenkins | v1.34.0 | 09 Oct 24 19:31 UTC | 09 Oct 24 19:31 UTC |
| start | -p old-k8s-version-135957 | old-k8s-version-135957 | jenkins | v1.34.0 | 09 Oct 24 19:31 UTC | 09 Oct 24 19:34 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-383078 | cert-expiration-383078 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:34 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-383078 | cert-expiration-383078 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:34 UTC |
| start | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:35 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --apiserver-port=8444 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-135957 | old-k8s-version-135957 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:34 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-135957 | old-k8s-version-135957 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:34 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-135957 | old-k8s-version-135957 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | 09 Oct 24 19:34 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-135957 | old-k8s-version-135957 | jenkins | v1.34.0 | 09 Oct 24 19:34 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p default-k8s-diff-port-083200 | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:35 UTC | 09 Oct 24 19:35 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:35 UTC | 09 Oct 24 19:35 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p default-k8s-diff-port-083200 | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:35 UTC | 09 Oct 24 19:35 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:35 UTC | 09 Oct 24 19:40 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --apiserver-port=8444 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| image | default-k8s-diff-port-083200 | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | 09 Oct 24 19:40 UTC |
| | image list --format=json | | | | | |
| pause | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | 09 Oct 24 19:40 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | 09 Oct 24 19:40 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | 09 Oct 24 19:40 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| delete | -p | default-k8s-diff-port-083200 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | 09 Oct 24 19:40 UTC |
| | default-k8s-diff-port-083200 | | | | | |
| start | -p embed-certs-269650 | embed-certs-269650 | jenkins | v1.34.0 | 09 Oct 24 19:40 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
|---------|--------------------------------------------------------|------------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/10/09 19:40:26
Running on machine: ip-172-31-21-244
Binary: Built with gc go1.23.1 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I1009 19:40:26.489645 222294 out.go:345] Setting OutFile to fd 1 ...
I1009 19:40:26.489852 222294 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:26.489880 222294 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:26.489903 222294 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:26.490255 222294 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19780-2290/.minikube/bin
I1009 19:40:26.490768 222294 out.go:352] Setting JSON to false
I1009 19:40:26.492042 222294 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":4970,"bootTime":1728497857,"procs":228,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I1009 19:40:26.492137 222294 start.go:139] virtualization:
I1009 19:40:26.494342 222294 out.go:177] * [embed-certs-269650] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I1009 19:40:26.496663 222294 out.go:177] - MINIKUBE_LOCATION=19780
I1009 19:40:26.496791 222294 notify.go:220] Checking for updates...
I1009 19:40:26.499942 222294 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I1009 19:40:26.501946 222294 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19780-2290/kubeconfig
I1009 19:40:26.504521 222294 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19780-2290/.minikube
I1009 19:40:26.506264 222294 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I1009 19:40:26.507754 222294 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I1009 19:40:26.510295 222294 config.go:182] Loaded profile config "old-k8s-version-135957": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I1009 19:40:26.510392 222294 driver.go:394] Setting default libvirt URI to qemu:///system
I1009 19:40:26.530205 222294 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I1009 19:40:26.530337 222294 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1009 19:40:26.591066 222294 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-10-09 19:40:26.579122767 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I1009 19:40:26.591180 222294 docker.go:318] overlay module found
I1009 19:40:26.592869 222294 out.go:177] * Using the docker driver based on user configuration
I1009 19:40:26.595035 222294 start.go:297] selected driver: docker
I1009 19:40:26.595053 222294 start.go:901] validating driver "docker" against <nil>
I1009 19:40:26.595067 222294 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I1009 19:40:26.595699 222294 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I1009 19:40:26.646049 222294 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-10-09 19:40:26.6364872 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: bridge
-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I1009 19:40:26.646265 222294 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I1009 19:40:26.646490 222294 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I1009 19:40:26.648166 222294 out.go:177] * Using Docker driver with root privileges
I1009 19:40:26.650208 222294 cni.go:84] Creating CNI manager for ""
I1009 19:40:26.650286 222294 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I1009 19:40:26.650303 222294 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I1009 19:40:26.650385 222294 start.go:340] cluster config:
{Name:embed-certs-269650 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-269650 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stati
cIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1009 19:40:26.652181 222294 out.go:177] * Starting "embed-certs-269650" primary control-plane node in "embed-certs-269650" cluster
I1009 19:40:26.653916 222294 cache.go:121] Beginning downloading kic base image for docker with containerd
I1009 19:40:26.656391 222294 out.go:177] * Pulling base image v0.0.45-1728382586-19774 ...
I1009 19:40:26.658267 222294 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I1009 19:40:26.658311 222294 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec in local docker daemon
I1009 19:40:26.658319 222294 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4
I1009 19:40:26.658329 222294 cache.go:56] Caching tarball of preloaded images
I1009 19:40:26.658403 222294 preload.go:172] Found /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I1009 19:40:26.658412 222294 cache.go:59] Finished verifying existence of preloaded tar for v1.31.1 on containerd
I1009 19:40:26.658520 222294 profile.go:143] Saving config to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/config.json ...
I1009 19:40:26.658537 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/config.json: {Name:mk225ea6130476915b385ad0571afe8e67d7e653 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:26.677955 222294 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec in local docker daemon, skipping pull
I1009 19:40:26.677977 222294 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec exists in daemon, skipping load
I1009 19:40:26.677991 222294 cache.go:194] Successfully downloaded all kic artifacts
I1009 19:40:26.678013 222294 start.go:360] acquireMachinesLock for embed-certs-269650: {Name:mk3d1962565abb0952d504a83857c81bcdfb71a6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1009 19:40:26.678117 222294 start.go:364] duration metric: took 83.88µs to acquireMachinesLock for "embed-certs-269650"
I1009 19:40:26.678148 222294 start.go:93] Provisioning new machine with config: &{Name:embed-certs-269650 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-269650 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableM
etrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I1009 19:40:26.678225 222294 start.go:125] createHost starting for "" (driver="docker")
I1009 19:40:23.983697 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:26.483244 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:28.484264 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:26.680907 222294 out.go:235] * Creating docker container (CPUs=2, Memory=2200MB) ...
I1009 19:40:26.681170 222294 start.go:159] libmachine.API.Create for "embed-certs-269650" (driver="docker")
I1009 19:40:26.681206 222294 client.go:168] LocalClient.Create starting
I1009 19:40:26.681283 222294 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem
I1009 19:40:26.681321 222294 main.go:141] libmachine: Decoding PEM data...
I1009 19:40:26.681339 222294 main.go:141] libmachine: Parsing certificate...
I1009 19:40:26.681390 222294 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem
I1009 19:40:26.681411 222294 main.go:141] libmachine: Decoding PEM data...
I1009 19:40:26.681427 222294 main.go:141] libmachine: Parsing certificate...
I1009 19:40:26.681803 222294 cli_runner.go:164] Run: docker network inspect embed-certs-269650 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W1009 19:40:26.700520 222294 cli_runner.go:211] docker network inspect embed-certs-269650 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I1009 19:40:26.700599 222294 network_create.go:284] running [docker network inspect embed-certs-269650] to gather additional debugging logs...
I1009 19:40:26.700620 222294 cli_runner.go:164] Run: docker network inspect embed-certs-269650
W1009 19:40:26.715591 222294 cli_runner.go:211] docker network inspect embed-certs-269650 returned with exit code 1
I1009 19:40:26.715622 222294 network_create.go:287] error running [docker network inspect embed-certs-269650]: docker network inspect embed-certs-269650: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-269650 not found
I1009 19:40:26.715634 222294 network_create.go:289] output of [docker network inspect embed-certs-269650]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-269650 not found
** /stderr **
I1009 19:40:26.715722 222294 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1009 19:40:26.731726 222294 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-bda550f8dcd8 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:31:98:1f:4a} reservation:<nil>}
I1009 19:40:26.732143 222294 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-4f797a1a661b IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:fc:32:f4:94} reservation:<nil>}
I1009 19:40:26.732552 222294 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-3956c1bd1495 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:32:83:c1:05} reservation:<nil>}
I1009 19:40:26.732959 222294 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-e0ecbcb012de IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:02:42:b9:ec:0f:ad} reservation:<nil>}
I1009 19:40:26.733476 222294 network.go:206] using free private subnet 192.168.85.0/24: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4001a2e0d0}
I1009 19:40:26.733516 222294 network_create.go:124] attempt to create docker network embed-certs-269650 192.168.85.0/24 with gateway 192.168.85.1 and MTU of 1500 ...
I1009 19:40:26.733580 222294 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.85.0/24 --gateway=192.168.85.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-269650 embed-certs-269650
I1009 19:40:26.809451 222294 network_create.go:108] docker network embed-certs-269650 192.168.85.0/24 created
I1009 19:40:26.809484 222294 kic.go:121] calculated static IP "192.168.85.2" for the "embed-certs-269650" container
I1009 19:40:26.809556 222294 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I1009 19:40:26.824742 222294 cli_runner.go:164] Run: docker volume create embed-certs-269650 --label name.minikube.sigs.k8s.io=embed-certs-269650 --label created_by.minikube.sigs.k8s.io=true
I1009 19:40:26.842771 222294 oci.go:103] Successfully created a docker volume embed-certs-269650
I1009 19:40:26.842869 222294 cli_runner.go:164] Run: docker run --rm --name embed-certs-269650-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-269650 --entrypoint /usr/bin/test -v embed-certs-269650:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec -d /var/lib
I1009 19:40:27.479996 222294 oci.go:107] Successfully prepared a docker volume embed-certs-269650
I1009 19:40:27.480049 222294 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I1009 19:40:27.480081 222294 kic.go:194] Starting extracting preloaded images to volume ...
I1009 19:40:27.480152 222294 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-269650:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec -I lz4 -xf /preloaded.tar -C /extractDir
I1009 19:40:30.985198 212114 pod_ready.go:103] pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace has status "Ready":"False"
I1009 19:40:32.983535 212114 pod_ready.go:82] duration metric: took 4m0.007360452s for pod "metrics-server-9975d5f86-jcsl5" in "kube-system" namespace to be "Ready" ...
E1009 19:40:32.983561 212114 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I1009 19:40:32.983570 212114 pod_ready.go:39] duration metric: took 5m29.838077957s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I1009 19:40:32.983586 212114 api_server.go:52] waiting for apiserver process to appear ...
I1009 19:40:32.983616 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I1009 19:40:32.983677 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I1009 19:40:33.049530 212114 cri.go:89] found id: "a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:33.049553 212114 cri.go:89] found id: "4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:33.049558 212114 cri.go:89] found id: ""
I1009 19:40:33.049566 212114 logs.go:282] 2 containers: [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797]
I1009 19:40:33.049621 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.062880 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.069016 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I1009 19:40:33.069102 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I1009 19:40:33.148317 212114 cri.go:89] found id: "5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:33.148337 212114 cri.go:89] found id: "1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:33.148342 212114 cri.go:89] found id: ""
I1009 19:40:33.148356 212114 logs.go:282] 2 containers: [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f]
I1009 19:40:33.148419 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.152544 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.158280 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I1009 19:40:33.158385 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I1009 19:40:33.250632 212114 cri.go:89] found id: "63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:33.250656 212114 cri.go:89] found id: "84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:33.250662 212114 cri.go:89] found id: ""
I1009 19:40:33.250669 212114 logs.go:282] 2 containers: [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4]
I1009 19:40:33.250761 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.254707 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.258160 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I1009 19:40:33.258276 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I1009 19:40:33.329677 212114 cri.go:89] found id: "855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:33.329698 212114 cri.go:89] found id: "187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:33.329703 212114 cri.go:89] found id: ""
I1009 19:40:33.329711 212114 logs.go:282] 2 containers: [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e]
I1009 19:40:33.329770 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.335537 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.341688 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I1009 19:40:33.341770 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I1009 19:40:33.409335 212114 cri.go:89] found id: "4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:33.409361 212114 cri.go:89] found id: "e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:33.409365 212114 cri.go:89] found id: ""
I1009 19:40:33.409372 212114 logs.go:282] 2 containers: [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e]
I1009 19:40:33.409428 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.414175 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.418573 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I1009 19:40:33.418657 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I1009 19:40:33.480197 212114 cri.go:89] found id: "a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:33.480228 212114 cri.go:89] found id: "d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:33.480232 212114 cri.go:89] found id: ""
I1009 19:40:33.480240 212114 logs.go:282] 2 containers: [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4]
I1009 19:40:33.480322 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.484759 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.489087 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I1009 19:40:33.489206 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I1009 19:40:33.557450 212114 cri.go:89] found id: "c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:33.557473 212114 cri.go:89] found id: "c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:33.557479 212114 cri.go:89] found id: ""
I1009 19:40:33.557685 212114 logs.go:282] 2 containers: [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c]
I1009 19:40:33.557763 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.564818 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.574510 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I1009 19:40:33.574618 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I1009 19:40:33.671672 212114 cri.go:89] found id: "ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:33.671726 212114 cri.go:89] found id: ""
I1009 19:40:33.671736 212114 logs.go:282] 1 containers: [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60]
I1009 19:40:33.671806 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.677087 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I1009 19:40:33.677159 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I1009 19:40:33.763257 212114 cri.go:89] found id: "70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:33.763285 212114 cri.go:89] found id: "932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:33.763291 212114 cri.go:89] found id: ""
I1009 19:40:33.763298 212114 logs.go:282] 2 containers: [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f]
I1009 19:40:33.763356 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.767028 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:33.770406 212114 logs.go:123] Gathering logs for etcd [1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f] ...
I1009 19:40:33.770425 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:31.834322 222294 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19780-2290/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.1-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-269650:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec -I lz4 -xf /preloaded.tar -C /extractDir: (4.354107961s)
I1009 19:40:31.834351 222294 kic.go:203] duration metric: took 4.354267842s to extract preloaded images to volume ...
W1009 19:40:31.834488 222294 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I1009 19:40:31.834623 222294 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I1009 19:40:31.886353 222294 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-269650 --name embed-certs-269650 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-269650 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-269650 --network embed-certs-269650 --ip 192.168.85.2 --volume embed-certs-269650:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec
I1009 19:40:32.220890 222294 cli_runner.go:164] Run: docker container inspect embed-certs-269650 --format={{.State.Running}}
I1009 19:40:32.241979 222294 cli_runner.go:164] Run: docker container inspect embed-certs-269650 --format={{.State.Status}}
I1009 19:40:32.264831 222294 cli_runner.go:164] Run: docker exec embed-certs-269650 stat /var/lib/dpkg/alternatives/iptables
I1009 19:40:32.344747 222294 oci.go:144] the created container "embed-certs-269650" has a running status.
I1009 19:40:32.344774 222294 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa...
I1009 19:40:32.684175 222294 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I1009 19:40:32.710696 222294 cli_runner.go:164] Run: docker container inspect embed-certs-269650 --format={{.State.Status}}
I1009 19:40:32.739326 222294 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I1009 19:40:32.739345 222294 kic_runner.go:114] Args: [docker exec --privileged embed-certs-269650 chown docker:docker /home/docker/.ssh/authorized_keys]
I1009 19:40:32.825699 222294 cli_runner.go:164] Run: docker container inspect embed-certs-269650 --format={{.State.Status}}
I1009 19:40:32.855636 222294 machine.go:93] provisionDockerMachine start ...
I1009 19:40:32.855735 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:32.887428 222294 main.go:141] libmachine: Using SSH client type: native
I1009 19:40:32.887689 222294 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I1009 19:40:32.887699 222294 main.go:141] libmachine: About to run SSH command:
hostname
I1009 19:40:33.073172 222294 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-269650
I1009 19:40:33.073199 222294 ubuntu.go:169] provisioning hostname "embed-certs-269650"
I1009 19:40:33.073273 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:33.097412 222294 main.go:141] libmachine: Using SSH client type: native
I1009 19:40:33.097725 222294 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I1009 19:40:33.097747 222294 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-269650 && echo "embed-certs-269650" | sudo tee /etc/hostname
I1009 19:40:33.312776 222294 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-269650
I1009 19:40:33.312867 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:33.339405 222294 main.go:141] libmachine: Using SSH client type: native
I1009 19:40:33.339726 222294 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413e90] 0x4166d0 <nil> [] 0s} 127.0.0.1 33073 <nil> <nil>}
I1009 19:40:33.339751 222294 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-269650' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-269650/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-269650' | sudo tee -a /etc/hosts;
fi
fi
I1009 19:40:33.517148 222294 main.go:141] libmachine: SSH cmd err, output: <nil>:
I1009 19:40:33.517176 222294 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19780-2290/.minikube CaCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19780-2290/.minikube}
I1009 19:40:33.517197 222294 ubuntu.go:177] setting up certificates
I1009 19:40:33.517205 222294 provision.go:84] configureAuth start
I1009 19:40:33.517266 222294 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-269650
I1009 19:40:33.547901 222294 provision.go:143] copyHostCerts
I1009 19:40:33.547970 222294 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem, removing ...
I1009 19:40:33.547983 222294 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem
I1009 19:40:33.548060 222294 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/ca.pem (1078 bytes)
I1009 19:40:33.548158 222294 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem, removing ...
I1009 19:40:33.548170 222294 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem
I1009 19:40:33.548198 222294 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/cert.pem (1123 bytes)
I1009 19:40:33.548269 222294 exec_runner.go:144] found /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem, removing ...
I1009 19:40:33.548286 222294 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem
I1009 19:40:33.548313 222294 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19780-2290/.minikube/key.pem (1679 bytes)
I1009 19:40:33.548377 222294 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem org=jenkins.embed-certs-269650 san=[127.0.0.1 192.168.85.2 embed-certs-269650 localhost minikube]
I1009 19:40:33.951433 222294 provision.go:177] copyRemoteCerts
I1009 19:40:33.951571 222294 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I1009 19:40:33.951635 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:33.991075 222294 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa Username:docker}
I1009 19:40:34.102412 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I1009 19:40:34.130620 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I1009 19:40:34.162804 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I1009 19:40:34.187772 222294 provision.go:87] duration metric: took 670.545374ms to configureAuth
I1009 19:40:34.187805 222294 ubuntu.go:193] setting minikube options for container-runtime
I1009 19:40:34.188002 222294 config.go:182] Loaded profile config "embed-certs-269650": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I1009 19:40:34.188016 222294 machine.go:96] duration metric: took 1.332359878s to provisionDockerMachine
I1009 19:40:34.188023 222294 client.go:171] duration metric: took 7.506806066s to LocalClient.Create
I1009 19:40:34.188037 222294 start.go:167] duration metric: took 7.506867044s to libmachine.API.Create "embed-certs-269650"
I1009 19:40:34.188048 222294 start.go:293] postStartSetup for "embed-certs-269650" (driver="docker")
I1009 19:40:34.188058 222294 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I1009 19:40:34.188121 222294 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I1009 19:40:34.188164 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:34.216674 222294 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa Username:docker}
I1009 19:40:34.322105 222294 ssh_runner.go:195] Run: cat /etc/os-release
I1009 19:40:34.325752 222294 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I1009 19:40:34.325794 222294 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I1009 19:40:34.325806 222294 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I1009 19:40:34.325813 222294 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I1009 19:40:34.325837 222294 filesync.go:126] Scanning /home/jenkins/minikube-integration/19780-2290/.minikube/addons for local assets ...
I1009 19:40:34.325893 222294 filesync.go:126] Scanning /home/jenkins/minikube-integration/19780-2290/.minikube/files for local assets ...
I1009 19:40:34.325977 222294 filesync.go:149] local asset: /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem -> 75962.pem in /etc/ssl/certs
I1009 19:40:34.326089 222294 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I1009 19:40:34.336247 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem --> /etc/ssl/certs/75962.pem (1708 bytes)
I1009 19:40:34.362874 222294 start.go:296] duration metric: took 174.812137ms for postStartSetup
I1009 19:40:34.363231 222294 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-269650
I1009 19:40:34.383351 222294 profile.go:143] Saving config to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/config.json ...
I1009 19:40:34.383640 222294 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I1009 19:40:34.383694 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:34.405686 222294 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa Username:docker}
I1009 19:40:34.498491 222294 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I1009 19:40:34.504800 222294 start.go:128] duration metric: took 7.82655928s to createHost
I1009 19:40:34.504826 222294 start.go:83] releasing machines lock for "embed-certs-269650", held for 7.826694777s
I1009 19:40:34.504907 222294 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-269650
I1009 19:40:34.528454 222294 ssh_runner.go:195] Run: cat /version.json
I1009 19:40:34.528510 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:34.528755 222294 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I1009 19:40:34.528829 222294 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-269650
I1009 19:40:34.562978 222294 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa Username:docker}
I1009 19:40:34.570239 222294 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33073 SSHKeyPath:/home/jenkins/minikube-integration/19780-2290/.minikube/machines/embed-certs-269650/id_rsa Username:docker}
I1009 19:40:34.822009 222294 ssh_runner.go:195] Run: systemctl --version
I1009 19:40:34.827198 222294 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I1009 19:40:34.831717 222294 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I1009 19:40:34.860499 222294 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I1009 19:40:34.860593 222294 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I1009 19:40:34.896792 222294 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I1009 19:40:34.896833 222294 start.go:495] detecting cgroup driver to use...
I1009 19:40:34.896866 222294 detect.go:187] detected "cgroupfs" cgroup driver on host os
I1009 19:40:34.896937 222294 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I1009 19:40:34.912474 222294 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I1009 19:40:34.926652 222294 docker.go:217] disabling cri-docker service (if available) ...
I1009 19:40:34.926718 222294 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I1009 19:40:34.952875 222294 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I1009 19:40:34.971606 222294 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I1009 19:40:35.106013 222294 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I1009 19:40:35.229188 222294 docker.go:233] disabling docker service ...
I1009 19:40:35.229267 222294 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I1009 19:40:35.264716 222294 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I1009 19:40:35.282980 222294 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I1009 19:40:35.382544 222294 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I1009 19:40:35.470655 222294 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I1009 19:40:35.483274 222294 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I1009 19:40:35.504686 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I1009 19:40:35.516122 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I1009 19:40:35.526917 222294 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I1009 19:40:35.527031 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I1009 19:40:35.537952 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1009 19:40:35.549796 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I1009 19:40:35.559488 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I1009 19:40:35.569451 222294 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I1009 19:40:35.578931 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I1009 19:40:35.589121 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I1009 19:40:35.599155 222294 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I1009 19:40:35.609509 222294 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I1009 19:40:35.618577 222294 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I1009 19:40:35.627383 222294 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1009 19:40:35.707742 222294 ssh_runner.go:195] Run: sudo systemctl restart containerd
I1009 19:40:35.869739 222294 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I1009 19:40:35.869820 222294 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I1009 19:40:35.874069 222294 start.go:563] Will wait 60s for crictl version
I1009 19:40:35.874136 222294 ssh_runner.go:195] Run: which crictl
I1009 19:40:35.877784 222294 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I1009 19:40:35.925914 222294 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I1009 19:40:35.925984 222294 ssh_runner.go:195] Run: containerd --version
I1009 19:40:35.952223 222294 ssh_runner.go:195] Run: containerd --version
I1009 19:40:35.976093 222294 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I1009 19:40:35.977128 222294 cli_runner.go:164] Run: docker network inspect embed-certs-269650 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I1009 19:40:35.993221 222294 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I1009 19:40:35.996973 222294 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1009 19:40:36.007949 222294 kubeadm.go:883] updating cluster {Name:embed-certs-269650 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-269650 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:
false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I1009 19:40:36.008066 222294 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I1009 19:40:36.008126 222294 ssh_runner.go:195] Run: sudo crictl images --output json
I1009 19:40:36.052360 222294 containerd.go:627] all images are preloaded for containerd runtime.
I1009 19:40:36.052386 222294 containerd.go:534] Images already preloaded, skipping extraction
I1009 19:40:36.052454 222294 ssh_runner.go:195] Run: sudo crictl images --output json
I1009 19:40:36.092037 222294 containerd.go:627] all images are preloaded for containerd runtime.
I1009 19:40:36.092061 222294 cache_images.go:84] Images are preloaded, skipping loading
I1009 19:40:36.092070 222294 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.31.1 containerd true true} ...
I1009 19:40:36.092166 222294 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-269650 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:embed-certs-269650 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I1009 19:40:36.092240 222294 ssh_runner.go:195] Run: sudo crictl info
I1009 19:40:36.134983 222294 cni.go:84] Creating CNI manager for ""
I1009 19:40:36.135010 222294 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I1009 19:40:36.135020 222294 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I1009 19:40:36.135045 222294 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-269650 NodeName:embed-certs-269650 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I1009 19:40:36.135190 222294 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-269650"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I1009 19:40:36.135271 222294 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I1009 19:40:36.146145 222294 binaries.go:44] Found k8s binaries, skipping transfer
I1009 19:40:36.146285 222294 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I1009 19:40:36.155234 222294 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I1009 19:40:36.174369 222294 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I1009 19:40:36.194645 222294 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I1009 19:40:36.217413 222294 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I1009 19:40:36.220896 222294 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I1009 19:40:36.232146 222294 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I1009 19:40:36.323823 222294 ssh_runner.go:195] Run: sudo systemctl start kubelet
I1009 19:40:36.340475 222294 certs.go:68] Setting up /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650 for IP: 192.168.85.2
I1009 19:40:36.340497 222294 certs.go:194] generating shared ca certs ...
I1009 19:40:36.340513 222294 certs.go:226] acquiring lock for ca certs: {Name:mke6990d9a3fb276a87991bc9cbf7d64b4192c7c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:36.340732 222294 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19780-2290/.minikube/ca.key
I1009 19:40:36.340805 222294 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.key
I1009 19:40:36.340820 222294 certs.go:256] generating profile certs ...
I1009 19:40:36.340901 222294 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.key
I1009 19:40:36.340928 222294 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.crt with IP's: []
I1009 19:40:33.826020 212114 logs.go:123] Gathering logs for coredns [84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4] ...
I1009 19:40:33.826064 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:33.879106 212114 logs.go:123] Gathering logs for kube-scheduler [187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e] ...
I1009 19:40:33.879135 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:33.930392 212114 logs.go:123] Gathering logs for containerd ...
I1009 19:40:33.930428 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I1009 19:40:34.004770 212114 logs.go:123] Gathering logs for container status ...
I1009 19:40:34.004799 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I1009 19:40:34.056516 212114 logs.go:123] Gathering logs for kubelet ...
I1009 19:40:34.056599 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W1009 19:40:34.117909 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141088 655 reflector.go:138] object-"kube-system"/"kube-proxy-token-l28zw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l28zw" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118146 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141226 655 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118362 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141301 655 reflector.go:138] object-"kube-system"/"kindnet-token-ch425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ch425" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118562 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141420 655 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118773 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141526 655 reflector.go:138] object-"kube-system"/"coredns-token-x8mx9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-x8mx9" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.118996 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.145947 655 reflector.go:138] object-"kube-system"/"metrics-server-token-n5d8s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-n5d8s" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.119211 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146058 655 reflector.go:138] object-"default"/"default-token-dtkng": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-dtkng" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.119436 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146124 655 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8fzd6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8fzd6" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:34.127887 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.568696 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.129333 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.908103 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.133313 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:18 old-k8s-version-135957 kubelet[655]: E1009 19:35:18.807162 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.135414 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:27 old-k8s-version-135957 kubelet[655]: E1009 19:35:27.017732 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.135765 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:28 old-k8s-version-135957 kubelet[655]: E1009 19:35:28.030088 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.135969 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:29 old-k8s-version-135957 kubelet[655]: E1009 19:35:29.798736 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.136376 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:34 old-k8s-version-135957 kubelet[655]: E1009 19:35:34.224840 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.137166 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:36 old-k8s-version-135957 kubelet[655]: E1009 19:35:36.070564 655 pod_workers.go:191] Error syncing pod dbfd3538-0cb4-4cf0-b208-e18c725f6d5d ("storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"
W1009 19:40:34.139585 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:40 old-k8s-version-135957 kubelet[655]: E1009 19:35:40.810507 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.140516 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:48 old-k8s-version-135957 kubelet[655]: E1009 19:35:48.113466 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.140979 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.225991 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.141166 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.805911 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.141347 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:07 old-k8s-version-135957 kubelet[655]: E1009 19:36:07.794321 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.141929 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:10 old-k8s-version-135957 kubelet[655]: E1009 19:36:10.183835 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.142254 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:14 old-k8s-version-135957 kubelet[655]: E1009 19:36:14.226874 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.142436 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:20 old-k8s-version-135957 kubelet[655]: E1009 19:36:20.794329 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.142759 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:29 old-k8s-version-135957 kubelet[655]: E1009 19:36:29.793939 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.145371 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:33 old-k8s-version-135957 kubelet[655]: E1009 19:36:33.803424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.145703 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:41 old-k8s-version-135957 kubelet[655]: E1009 19:36:41.794984 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.145888 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:44 old-k8s-version-135957 kubelet[655]: E1009 19:36:44.794752 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.146482 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.299505 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.146665 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.794264 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.146989 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:04 old-k8s-version-135957 kubelet[655]: E1009 19:37:04.225263 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.147171 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:10 old-k8s-version-135957 kubelet[655]: E1009 19:37:10.794424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.147494 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:19 old-k8s-version-135957 kubelet[655]: E1009 19:37:19.794014 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.147679 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:25 old-k8s-version-135957 kubelet[655]: E1009 19:37:25.794260 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.148013 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:33 old-k8s-version-135957 kubelet[655]: E1009 19:37:33.794463 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.148203 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:36 old-k8s-version-135957 kubelet[655]: E1009 19:37:36.794403 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.148534 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:46 old-k8s-version-135957 kubelet[655]: E1009 19:37:46.794633 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.148729 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:48 old-k8s-version-135957 kubelet[655]: E1009 19:37:48.796873 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.149055 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:59 old-k8s-version-135957 kubelet[655]: E1009 19:37:59.794479 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.151483 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:03 old-k8s-version-135957 kubelet[655]: E1009 19:38:03.802412 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:34.152171 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:12 old-k8s-version-135957 kubelet[655]: E1009 19:38:12.799928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.152410 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:15 old-k8s-version-135957 kubelet[655]: E1009 19:38:15.794598 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.153011 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:27 old-k8s-version-135957 kubelet[655]: E1009 19:38:27.530323 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.153194 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:28 old-k8s-version-135957 kubelet[655]: E1009 19:38:28.795022 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.153519 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:34 old-k8s-version-135957 kubelet[655]: E1009 19:38:34.226016 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.153707 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:39 old-k8s-version-135957 kubelet[655]: E1009 19:38:39.794229 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154038 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:49 old-k8s-version-135957 kubelet[655]: E1009 19:38:49.795059 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.154225 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:51 old-k8s-version-135957 kubelet[655]: E1009 19:38:51.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154565 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.793928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.154754 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.795042 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.154941 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:16 old-k8s-version-135957 kubelet[655]: E1009 19:39:16.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.155340 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:18 old-k8s-version-135957 kubelet[655]: E1009 19:39:18.794102 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.155552 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:27 old-k8s-version-135957 kubelet[655]: E1009 19:39:27.794317 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.155882 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: E1009 19:39:30.794130 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156066 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:42 old-k8s-version-135957 kubelet[655]: E1009 19:39:42.794279 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.156436 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: E1009 19:39:43.794072 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156775 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.806354 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.156959 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.812240 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157144 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157467 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.157651 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:34.157977 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:34.158159 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:34.158169 212114 logs.go:123] Gathering logs for kube-apiserver [4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797] ...
I1009 19:40:34.158184 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:34.267905 212114 logs.go:123] Gathering logs for etcd [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a] ...
I1009 19:40:34.267941 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:34.313520 212114 logs.go:123] Gathering logs for kube-proxy [e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e] ...
I1009 19:40:34.313547 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:34.369410 212114 logs.go:123] Gathering logs for kube-controller-manager [d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4] ...
I1009 19:40:34.369434 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:34.435867 212114 logs.go:123] Gathering logs for coredns [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33] ...
I1009 19:40:34.435938 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:34.477407 212114 logs.go:123] Gathering logs for kube-scheduler [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd] ...
I1009 19:40:34.477432 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:34.525043 212114 logs.go:123] Gathering logs for kube-proxy [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454] ...
I1009 19:40:34.525071 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:34.579092 212114 logs.go:123] Gathering logs for kindnet [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9] ...
I1009 19:40:34.579120 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:34.645088 212114 logs.go:123] Gathering logs for kindnet [c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c] ...
I1009 19:40:34.645120 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:34.688041 212114 logs.go:123] Gathering logs for kubernetes-dashboard [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60] ...
I1009 19:40:34.688076 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:34.750518 212114 logs.go:123] Gathering logs for storage-provisioner [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c] ...
I1009 19:40:34.750548 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:34.792366 212114 logs.go:123] Gathering logs for storage-provisioner [932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f] ...
I1009 19:40:34.792401 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:34.844432 212114 logs.go:123] Gathering logs for dmesg ...
I1009 19:40:34.844506 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1009 19:40:34.862899 212114 logs.go:123] Gathering logs for kube-apiserver [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880] ...
I1009 19:40:34.862973 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:34.966604 212114 logs.go:123] Gathering logs for kube-controller-manager [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184] ...
I1009 19:40:34.966676 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:35.078882 212114 logs.go:123] Gathering logs for describe nodes ...
I1009 19:40:35.078925 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I1009 19:40:35.317341 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:35.317406 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
W1009 19:40:35.317493 212114 out.go:270] X Problems detected in kubelet:
W1009 19:40:35.317536 212114 out.go:270] Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:35.317592 212114 out.go:270] Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:35.317630 212114 out.go:270] Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:35.317662 212114 out.go:270] Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:35.317702 212114 out.go:270] Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:35.317738 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:35.317758 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:36.538245 222294 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.crt ...
I1009 19:40:36.538275 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.crt: {Name:mk646876f00c2860d6e04f53a1e8b63771cabb25 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:36.538468 222294 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.key ...
I1009 19:40:36.538482 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/client.key: {Name:mk311c4b244c4773b53d0d130c88f760abe3622a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:36.538978 222294 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key.baa294b0
I1009 19:40:36.539003 222294 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt.baa294b0 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.85.2]
I1009 19:40:36.848916 222294 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt.baa294b0 ...
I1009 19:40:36.848947 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt.baa294b0: {Name:mk50ca7d566f6027841c62287fe60fe2fc1f4030 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:36.849135 222294 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key.baa294b0 ...
I1009 19:40:36.849151 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key.baa294b0: {Name:mk3c466636e31a368add5e1ff9c60fd9c02cba7f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:36.849695 222294 certs.go:381] copying /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt.baa294b0 -> /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt
I1009 19:40:36.849786 222294 certs.go:385] copying /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key.baa294b0 -> /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key
I1009 19:40:36.849849 222294 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.key
I1009 19:40:36.849867 222294 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.crt with IP's: []
I1009 19:40:37.298921 222294 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.crt ...
I1009 19:40:37.298999 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.crt: {Name:mkca984c86b08b305248adb297a20aab8a1d28fb Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:37.299633 222294 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.key ...
I1009 19:40:37.299656 222294 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.key: {Name:mk4f0ee9c9eafa23baef4df79316a7dd1a4b9d6a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I1009 19:40:37.300209 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596.pem (1338 bytes)
W1009 19:40:37.300257 222294 certs.go:480] ignoring /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596_empty.pem, impossibly tiny 0 bytes
I1009 19:40:37.300267 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca-key.pem (1675 bytes)
I1009 19:40:37.300295 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/ca.pem (1078 bytes)
I1009 19:40:37.300341 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/cert.pem (1123 bytes)
I1009 19:40:37.300380 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/certs/key.pem (1679 bytes)
I1009 19:40:37.300429 222294 certs.go:484] found cert: /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem (1708 bytes)
I1009 19:40:37.301084 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I1009 19:40:37.326044 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I1009 19:40:37.351451 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I1009 19:40:37.377308 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I1009 19:40:37.402144 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I1009 19:40:37.427319 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I1009 19:40:37.451420 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I1009 19:40:37.475834 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/profiles/embed-certs-269650/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I1009 19:40:37.500227 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/files/etc/ssl/certs/75962.pem --> /usr/share/ca-certificates/75962.pem (1708 bytes)
I1009 19:40:37.525092 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I1009 19:40:37.550333 222294 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19780-2290/.minikube/certs/7596.pem --> /usr/share/ca-certificates/7596.pem (1338 bytes)
I1009 19:40:37.574676 222294 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I1009 19:40:37.593328 222294 ssh_runner.go:195] Run: openssl version
I1009 19:40:37.598929 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/75962.pem && ln -fs /usr/share/ca-certificates/75962.pem /etc/ssl/certs/75962.pem"
I1009 19:40:37.608742 222294 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/75962.pem
I1009 19:40:37.612511 222294 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Oct 9 18:57 /usr/share/ca-certificates/75962.pem
I1009 19:40:37.612627 222294 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/75962.pem
I1009 19:40:37.620145 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/75962.pem /etc/ssl/certs/3ec20f2e.0"
I1009 19:40:37.629778 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I1009 19:40:37.639675 222294 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I1009 19:40:37.643286 222294 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Oct 9 18:47 /usr/share/ca-certificates/minikubeCA.pem
I1009 19:40:37.643397 222294 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I1009 19:40:37.650534 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I1009 19:40:37.660613 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/7596.pem && ln -fs /usr/share/ca-certificates/7596.pem /etc/ssl/certs/7596.pem"
I1009 19:40:37.671299 222294 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/7596.pem
I1009 19:40:37.675556 222294 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Oct 9 18:57 /usr/share/ca-certificates/7596.pem
I1009 19:40:37.675630 222294 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/7596.pem
I1009 19:40:37.683708 222294 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/7596.pem /etc/ssl/certs/51391683.0"
I1009 19:40:37.694186 222294 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I1009 19:40:37.697726 222294 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I1009 19:40:37.697777 222294 kubeadm.go:392] StartCluster: {Name:embed-certs-269650 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1728382586-19774@sha256:5d8c4f6d838467365e214e2194dd0153a763e3f78723b5f2a8e06ef7b47409ec Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:embed-certs-269650 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fal
se CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I1009 19:40:37.697852 222294 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I1009 19:40:37.697926 222294 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I1009 19:40:37.741300 222294 cri.go:89] found id: ""
I1009 19:40:37.741372 222294 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I1009 19:40:37.750706 222294 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I1009 19:40:37.760418 222294 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I1009 19:40:37.760486 222294 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I1009 19:40:37.770548 222294 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I1009 19:40:37.770567 222294 kubeadm.go:157] found existing configuration files:
I1009 19:40:37.770616 222294 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I1009 19:40:37.780213 222294 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I1009 19:40:37.780332 222294 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I1009 19:40:37.789176 222294 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I1009 19:40:37.801635 222294 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I1009 19:40:37.801698 222294 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I1009 19:40:37.811415 222294 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I1009 19:40:37.822687 222294 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I1009 19:40:37.822849 222294 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I1009 19:40:37.833002 222294 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I1009 19:40:37.847264 222294 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I1009 19:40:37.847387 222294 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I1009 19:40:37.858907 222294 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.31.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I1009 19:40:37.908259 222294 kubeadm.go:310] W1009 19:40:37.907520 1045 common.go:101] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "ClusterConfiguration"). Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
I1009 19:40:37.909014 222294 kubeadm.go:310] W1009 19:40:37.908414 1045 common.go:101] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta3" (kind: "InitConfiguration"). Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version.
I1009 19:40:37.933898 222294 kubeadm.go:310] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1070-aws\n", err: exit status 1
I1009 19:40:38.000935 222294 kubeadm.go:310] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I1009 19:40:45.319566 212114 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I1009 19:40:45.334167 212114 api_server.go:72] duration metric: took 5m58.486701724s to wait for apiserver process to appear ...
I1009 19:40:45.334193 212114 api_server.go:88] waiting for apiserver healthz status ...
I1009 19:40:45.334230 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I1009 19:40:45.334291 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I1009 19:40:45.404349 212114 cri.go:89] found id: "a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:45.404374 212114 cri.go:89] found id: "4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:45.404379 212114 cri.go:89] found id: ""
I1009 19:40:45.404394 212114 logs.go:282] 2 containers: [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797]
I1009 19:40:45.404454 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.408983 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.413035 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I1009 19:40:45.413099 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I1009 19:40:45.466616 212114 cri.go:89] found id: "5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:45.466636 212114 cri.go:89] found id: "1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:45.466641 212114 cri.go:89] found id: ""
I1009 19:40:45.466651 212114 logs.go:282] 2 containers: [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f]
I1009 19:40:45.466707 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.470602 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.474342 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I1009 19:40:45.474415 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I1009 19:40:45.536508 212114 cri.go:89] found id: "63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:45.536533 212114 cri.go:89] found id: "84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:45.536539 212114 cri.go:89] found id: ""
I1009 19:40:45.536547 212114 logs.go:282] 2 containers: [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4]
I1009 19:40:45.536606 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.542255 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.546144 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I1009 19:40:45.546224 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I1009 19:40:45.610102 212114 cri.go:89] found id: "855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:45.610126 212114 cri.go:89] found id: "187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:45.610131 212114 cri.go:89] found id: ""
I1009 19:40:45.610138 212114 logs.go:282] 2 containers: [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e]
I1009 19:40:45.610196 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.614464 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.618470 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I1009 19:40:45.618545 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I1009 19:40:45.669697 212114 cri.go:89] found id: "4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:45.669720 212114 cri.go:89] found id: "e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:45.669726 212114 cri.go:89] found id: ""
I1009 19:40:45.669733 212114 logs.go:282] 2 containers: [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e]
I1009 19:40:45.669794 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.676366 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.680603 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I1009 19:40:45.680692 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I1009 19:40:45.749148 212114 cri.go:89] found id: "a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:45.749173 212114 cri.go:89] found id: "d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:45.749178 212114 cri.go:89] found id: ""
I1009 19:40:45.749185 212114 logs.go:282] 2 containers: [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4]
I1009 19:40:45.749242 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.753756 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.757836 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I1009 19:40:45.757914 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I1009 19:40:45.935969 212114 cri.go:89] found id: "c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:45.935996 212114 cri.go:89] found id: "c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:45.936001 212114 cri.go:89] found id: ""
I1009 19:40:45.936008 212114 logs.go:282] 2 containers: [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c]
I1009 19:40:45.936065 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.940437 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:45.953098 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I1009 19:40:45.953178 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I1009 19:40:46.012160 212114 cri.go:89] found id: "ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:46.012185 212114 cri.go:89] found id: ""
I1009 19:40:46.012193 212114 logs.go:282] 1 containers: [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60]
I1009 19:40:46.012253 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.016624 212114 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I1009 19:40:46.016713 212114 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I1009 19:40:46.083357 212114 cri.go:89] found id: "70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:46.083383 212114 cri.go:89] found id: "932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:46.083388 212114 cri.go:89] found id: ""
I1009 19:40:46.083396 212114 logs.go:282] 2 containers: [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f]
I1009 19:40:46.083456 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.087674 212114 ssh_runner.go:195] Run: which crictl
I1009 19:40:46.091613 212114 logs.go:123] Gathering logs for storage-provisioner [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c] ...
I1009 19:40:46.091642 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c"
I1009 19:40:46.144851 212114 logs.go:123] Gathering logs for etcd [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a] ...
I1009 19:40:46.144890 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a"
I1009 19:40:46.214756 212114 logs.go:123] Gathering logs for coredns [84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4] ...
I1009 19:40:46.214788 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4"
I1009 19:40:46.268020 212114 logs.go:123] Gathering logs for kube-scheduler [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd] ...
I1009 19:40:46.268058 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd"
I1009 19:40:46.318517 212114 logs.go:123] Gathering logs for kube-scheduler [187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e] ...
I1009 19:40:46.318547 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e"
I1009 19:40:46.374095 212114 logs.go:123] Gathering logs for kube-controller-manager [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184] ...
I1009 19:40:46.374136 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184"
I1009 19:40:46.460506 212114 logs.go:123] Gathering logs for kindnet [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9] ...
I1009 19:40:46.460541 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9"
I1009 19:40:46.524819 212114 logs.go:123] Gathering logs for kindnet [c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c] ...
I1009 19:40:46.524849 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c"
I1009 19:40:46.583620 212114 logs.go:123] Gathering logs for kubelet ...
I1009 19:40:46.583702 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W1009 19:40:46.644048 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141088 655 reflector.go:138] object-"kube-system"/"kube-proxy-token-l28zw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l28zw" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644351 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141226 655 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644618 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141301 655 reflector.go:138] object-"kube-system"/"kindnet-token-ch425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ch425" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.644879 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141420 655 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645133 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.141526 655 reflector.go:138] object-"kube-system"/"coredns-token-x8mx9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-x8mx9" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645400 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.145947 655 reflector.go:138] object-"kube-system"/"metrics-server-token-n5d8s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-n5d8s" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645645 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146058 655 reflector.go:138] object-"default"/"default-token-dtkng": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-dtkng" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.645924 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:03 old-k8s-version-135957 kubelet[655]: E1009 19:35:03.146124 655 reflector.go:138] object-"kube-system"/"storage-provisioner-token-8fzd6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-8fzd6" is forbidden: User "system:node:old-k8s-version-135957" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-135957' and this object
W1009 19:40:46.654238 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.568696 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.655773 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:05 old-k8s-version-135957 kubelet[655]: E1009 19:35:05.908103 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.658740 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:18 old-k8s-version-135957 kubelet[655]: E1009 19:35:18.807162 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.660983 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:27 old-k8s-version-135957 kubelet[655]: E1009 19:35:27.017732 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.661371 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:28 old-k8s-version-135957 kubelet[655]: E1009 19:35:28.030088 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.661645 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:29 old-k8s-version-135957 kubelet[655]: E1009 19:35:29.798736 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.662027 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:34 old-k8s-version-135957 kubelet[655]: E1009 19:35:34.224840 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.662893 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:36 old-k8s-version-135957 kubelet[655]: E1009 19:35:36.070564 655 pod_workers.go:191] Error syncing pod dbfd3538-0cb4-4cf0-b208-e18c725f6d5d ("storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(dbfd3538-0cb4-4cf0-b208-e18c725f6d5d)"
W1009 19:40:46.665881 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:40 old-k8s-version-135957 kubelet[655]: E1009 19:35:40.810507 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.666940 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:48 old-k8s-version-135957 kubelet[655]: E1009 19:35:48.113466 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.667468 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.225991 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.667683 212114 logs.go:138] Found kubelet problem: Oct 09 19:35:54 old-k8s-version-135957 kubelet[655]: E1009 19:35:54.805911 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.667944 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:07 old-k8s-version-135957 kubelet[655]: E1009 19:36:07.794321 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.668632 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:10 old-k8s-version-135957 kubelet[655]: E1009 19:36:10.183835 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.669178 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:14 old-k8s-version-135957 kubelet[655]: E1009 19:36:14.226874 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.669376 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:20 old-k8s-version-135957 kubelet[655]: E1009 19:36:20.794329 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.669743 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:29 old-k8s-version-135957 kubelet[655]: E1009 19:36:29.793939 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.672283 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:33 old-k8s-version-135957 kubelet[655]: E1009 19:36:33.803424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.672629 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:41 old-k8s-version-135957 kubelet[655]: E1009 19:36:41.794984 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.672973 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:44 old-k8s-version-135957 kubelet[655]: E1009 19:36:44.794752 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.673626 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.299505 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.673854 212114 logs.go:138] Found kubelet problem: Oct 09 19:36:57 old-k8s-version-135957 kubelet[655]: E1009 19:36:57.794264 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.674216 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:04 old-k8s-version-135957 kubelet[655]: E1009 19:37:04.225263 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.674431 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:10 old-k8s-version-135957 kubelet[655]: E1009 19:37:10.794424 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.674804 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:19 old-k8s-version-135957 kubelet[655]: E1009 19:37:19.794014 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.675040 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:25 old-k8s-version-135957 kubelet[655]: E1009 19:37:25.794260 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.675400 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:33 old-k8s-version-135957 kubelet[655]: E1009 19:37:33.794463 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.675661 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:36 old-k8s-version-135957 kubelet[655]: E1009 19:37:36.794403 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.676031 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:46 old-k8s-version-135957 kubelet[655]: E1009 19:37:46.794633 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.676261 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:48 old-k8s-version-135957 kubelet[655]: E1009 19:37:48.796873 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.676632 212114 logs.go:138] Found kubelet problem: Oct 09 19:37:59 old-k8s-version-135957 kubelet[655]: E1009 19:37:59.794479 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.679482 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:03 old-k8s-version-135957 kubelet[655]: E1009 19:38:03.802412 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W1009 19:40:46.679873 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:12 old-k8s-version-135957 kubelet[655]: E1009 19:38:12.799928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.680105 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:15 old-k8s-version-135957 kubelet[655]: E1009 19:38:15.794598 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.680775 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:27 old-k8s-version-135957 kubelet[655]: E1009 19:38:27.530323 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.680993 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:28 old-k8s-version-135957 kubelet[655]: E1009 19:38:28.795022 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.681361 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:34 old-k8s-version-135957 kubelet[655]: E1009 19:38:34.226016 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.681590 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:39 old-k8s-version-135957 kubelet[655]: E1009 19:38:39.794229 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.681999 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:49 old-k8s-version-135957 kubelet[655]: E1009 19:38:49.795059 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.682230 212114 logs.go:138] Found kubelet problem: Oct 09 19:38:51 old-k8s-version-135957 kubelet[655]: E1009 19:38:51.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.682621 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.793928 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.682835 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:03 old-k8s-version-135957 kubelet[655]: E1009 19:39:03.795042 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.683069 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:16 old-k8s-version-135957 kubelet[655]: E1009 19:39:16.794422 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.683468 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:18 old-k8s-version-135957 kubelet[655]: E1009 19:39:18.794102 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.683703 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:27 old-k8s-version-135957 kubelet[655]: E1009 19:39:27.794317 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.684106 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: E1009 19:39:30.794130 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.684324 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:42 old-k8s-version-135957 kubelet[655]: E1009 19:39:42.794279 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.684724 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: E1009 19:39:43.794072 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.685423 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.806354 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.686352 212114 logs.go:138] Found kubelet problem: Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.812240 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.687361 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.687730 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.687961 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.688346 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.688589 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:46.688999 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:46.692433 212114 logs.go:138] Found kubelet problem: Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:46.692450 212114 logs.go:123] Gathering logs for kube-apiserver [4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797] ...
I1009 19:40:46.692464 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797"
I1009 19:40:46.775228 212114 logs.go:123] Gathering logs for etcd [1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f] ...
I1009 19:40:46.775261 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f"
I1009 19:40:46.843804 212114 logs.go:123] Gathering logs for kube-controller-manager [d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4] ...
I1009 19:40:46.843886 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4"
I1009 19:40:46.910698 212114 logs.go:123] Gathering logs for containerd ...
I1009 19:40:46.910734 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I1009 19:40:46.996012 212114 logs.go:123] Gathering logs for dmesg ...
I1009 19:40:46.996052 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I1009 19:40:47.015084 212114 logs.go:123] Gathering logs for kube-apiserver [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880] ...
I1009 19:40:47.015112 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880"
I1009 19:40:47.107848 212114 logs.go:123] Gathering logs for kubernetes-dashboard [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60] ...
I1009 19:40:47.107928 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60"
I1009 19:40:47.154334 212114 logs.go:123] Gathering logs for storage-provisioner [932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f] ...
I1009 19:40:47.154412 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f"
I1009 19:40:47.203396 212114 logs.go:123] Gathering logs for describe nodes ...
I1009 19:40:47.203474 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I1009 19:40:47.462255 212114 logs.go:123] Gathering logs for coredns [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33] ...
I1009 19:40:47.462327 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33"
I1009 19:40:47.515099 212114 logs.go:123] Gathering logs for kube-proxy [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454] ...
I1009 19:40:47.515182 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454"
I1009 19:40:47.569213 212114 logs.go:123] Gathering logs for kube-proxy [e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e] ...
I1009 19:40:47.569289 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e"
I1009 19:40:47.620464 212114 logs.go:123] Gathering logs for container status ...
I1009 19:40:47.620545 212114 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I1009 19:40:47.675639 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:47.675797 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
W1009 19:40:47.675885 212114 out.go:270] X Problems detected in kubelet:
W1009 19:40:47.675951 212114 out.go:270] Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:47.675995 212114 out.go:270] Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:47.676047 212114 out.go:270] Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W1009 19:40:47.676086 212114 out.go:270] Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
W1009 19:40:47.676137 212114 out.go:270] Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I1009 19:40:47.676196 212114 out.go:358] Setting ErrFile to fd 2...
I1009 19:40:47.676228 212114 out.go:392] TERM=,COLORTERM=, which probably does not support color
I1009 19:40:56.579286 222294 kubeadm.go:310] [init] Using Kubernetes version: v1.31.1
I1009 19:40:56.579351 222294 kubeadm.go:310] [preflight] Running pre-flight checks
I1009 19:40:56.579441 222294 kubeadm.go:310] [preflight] The system verification failed. Printing the output from the verification:
I1009 19:40:56.579500 222294 kubeadm.go:310] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1070-aws[0m
I1009 19:40:56.579538 222294 kubeadm.go:310] [0;37mOS[0m: [0;32mLinux[0m
I1009 19:40:56.579587 222294 kubeadm.go:310] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I1009 19:40:56.579639 222294 kubeadm.go:310] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I1009 19:40:56.579693 222294 kubeadm.go:310] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I1009 19:40:56.579743 222294 kubeadm.go:310] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I1009 19:40:56.579794 222294 kubeadm.go:310] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I1009 19:40:56.579845 222294 kubeadm.go:310] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I1009 19:40:56.579893 222294 kubeadm.go:310] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I1009 19:40:56.579943 222294 kubeadm.go:310] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I1009 19:40:56.579993 222294 kubeadm.go:310] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I1009 19:40:56.580069 222294 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I1009 19:40:56.580166 222294 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I1009 19:40:56.580258 222294 kubeadm.go:310] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I1009 19:40:56.580322 222294 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I1009 19:40:56.581979 222294 out.go:235] - Generating certificates and keys ...
I1009 19:40:56.582063 222294 kubeadm.go:310] [certs] Using existing ca certificate authority
I1009 19:40:56.582133 222294 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I1009 19:40:56.582205 222294 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I1009 19:40:56.582279 222294 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I1009 19:40:56.582342 222294 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I1009 19:40:56.582396 222294 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I1009 19:40:56.582452 222294 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I1009 19:40:56.582575 222294 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [embed-certs-269650 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I1009 19:40:56.582631 222294 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I1009 19:40:56.582752 222294 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-269650 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I1009 19:40:56.582819 222294 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I1009 19:40:56.582883 222294 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I1009 19:40:56.582930 222294 kubeadm.go:310] [certs] Generating "sa" key and public key
I1009 19:40:56.582988 222294 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I1009 19:40:56.583042 222294 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I1009 19:40:56.583104 222294 kubeadm.go:310] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I1009 19:40:56.583162 222294 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I1009 19:40:56.583231 222294 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I1009 19:40:56.583288 222294 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I1009 19:40:56.583375 222294 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I1009 19:40:56.583444 222294 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I1009 19:40:56.584906 222294 out.go:235] - Booting up control plane ...
I1009 19:40:56.585049 222294 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I1009 19:40:56.585141 222294 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I1009 19:40:56.585236 222294 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I1009 19:40:56.585379 222294 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I1009 19:40:56.585479 222294 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I1009 19:40:56.585523 222294 kubeadm.go:310] [kubelet-start] Starting the kubelet
I1009 19:40:56.585671 222294 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I1009 19:40:56.585784 222294 kubeadm.go:310] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I1009 19:40:56.585849 222294 kubeadm.go:310] [kubelet-check] The kubelet is healthy after 1.501952706s
I1009 19:40:56.585926 222294 kubeadm.go:310] [api-check] Waiting for a healthy API server. This can take up to 4m0s
I1009 19:40:56.585992 222294 kubeadm.go:310] [api-check] The API server is healthy after 7.001446567s
I1009 19:40:56.586100 222294 kubeadm.go:310] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I1009 19:40:56.586226 222294 kubeadm.go:310] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I1009 19:40:56.586287 222294 kubeadm.go:310] [upload-certs] Skipping phase. Please see --upload-certs
I1009 19:40:56.586473 222294 kubeadm.go:310] [mark-control-plane] Marking the node embed-certs-269650 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I1009 19:40:56.586532 222294 kubeadm.go:310] [bootstrap-token] Using token: 8l7pj7.fik8jhdqoqzcp1rj
I1009 19:40:56.588083 222294 out.go:235] - Configuring RBAC rules ...
I1009 19:40:56.588196 222294 kubeadm.go:310] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I1009 19:40:56.588292 222294 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I1009 19:40:56.588434 222294 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I1009 19:40:56.588582 222294 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I1009 19:40:56.588734 222294 kubeadm.go:310] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I1009 19:40:56.588825 222294 kubeadm.go:310] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I1009 19:40:56.588947 222294 kubeadm.go:310] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I1009 19:40:56.588993 222294 kubeadm.go:310] [addons] Applied essential addon: CoreDNS
I1009 19:40:56.589043 222294 kubeadm.go:310] [addons] Applied essential addon: kube-proxy
I1009 19:40:56.589053 222294 kubeadm.go:310]
I1009 19:40:56.589119 222294 kubeadm.go:310] Your Kubernetes control-plane has initialized successfully!
I1009 19:40:56.589128 222294 kubeadm.go:310]
I1009 19:40:56.589204 222294 kubeadm.go:310] To start using your cluster, you need to run the following as a regular user:
I1009 19:40:56.589213 222294 kubeadm.go:310]
I1009 19:40:56.589238 222294 kubeadm.go:310] mkdir -p $HOME/.kube
I1009 19:40:56.589299 222294 kubeadm.go:310] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I1009 19:40:56.589351 222294 kubeadm.go:310] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I1009 19:40:56.589358 222294 kubeadm.go:310]
I1009 19:40:56.589412 222294 kubeadm.go:310] Alternatively, if you are the root user, you can run:
I1009 19:40:56.589420 222294 kubeadm.go:310]
I1009 19:40:56.589467 222294 kubeadm.go:310] export KUBECONFIG=/etc/kubernetes/admin.conf
I1009 19:40:56.589474 222294 kubeadm.go:310]
I1009 19:40:56.589526 222294 kubeadm.go:310] You should now deploy a pod network to the cluster.
I1009 19:40:56.589603 222294 kubeadm.go:310] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I1009 19:40:56.589674 222294 kubeadm.go:310] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I1009 19:40:56.589682 222294 kubeadm.go:310]
I1009 19:40:56.589765 222294 kubeadm.go:310] You can now join any number of control-plane nodes by copying certificate authorities
I1009 19:40:56.589844 222294 kubeadm.go:310] and service account keys on each node and then running the following as root:
I1009 19:40:56.589851 222294 kubeadm.go:310]
I1009 19:40:56.589933 222294 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 8l7pj7.fik8jhdqoqzcp1rj \
I1009 19:40:56.590038 222294 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:46ecff2404792e73c0fde7b74431755068cf24bba8856a1cc3cbe480cfe7ea71 \
I1009 19:40:56.590062 222294 kubeadm.go:310] --control-plane
I1009 19:40:56.590071 222294 kubeadm.go:310]
I1009 19:40:56.590155 222294 kubeadm.go:310] Then you can join any number of worker nodes by running the following on each as root:
I1009 19:40:56.590163 222294 kubeadm.go:310]
I1009 19:40:56.590246 222294 kubeadm.go:310] kubeadm join control-plane.minikube.internal:8443 --token 8l7pj7.fik8jhdqoqzcp1rj \
I1009 19:40:56.590363 222294 kubeadm.go:310] --discovery-token-ca-cert-hash sha256:46ecff2404792e73c0fde7b74431755068cf24bba8856a1cc3cbe480cfe7ea71
I1009 19:40:56.590375 222294 cni.go:84] Creating CNI manager for ""
I1009 19:40:56.590382 222294 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I1009 19:40:56.592324 222294 out.go:177] * Configuring CNI (Container Networking Interface) ...
I1009 19:40:57.677635 212114 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I1009 19:40:57.689852 212114 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I1009 19:40:57.692719 212114 out.go:201]
W1009 19:40:57.694057 212114 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W1009 19:40:57.694098 212114 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W1009 19:40:57.694120 212114 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W1009 19:40:57.694126 212114 out.go:270] *
W1009 19:40:57.695133 212114 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I1009 19:40:57.697281 212114 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
2635990d5e8d9 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 6be23148e49af dashboard-metrics-scraper-8d5bb5db8-hjq4h
70879940cc769 ba04bb24b9575 5 minutes ago Running storage-provisioner 2 1de6e43eebf8b storage-provisioner
ea122f5ea8fc6 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 25f7c96003e73 kubernetes-dashboard-cd95d586-5fpql
63cf0a429f88f db91994f4ee8f 5 minutes ago Running coredns 1 15283005bc80d coredns-74ff55c5b-txr44
c26ad772ee958 0bcd66b03df5f 5 minutes ago Running kindnet-cni 1 7e7bef3881ee5 kindnet-5rbc6
4d322977adb34 25a5233254979 5 minutes ago Running kube-proxy 1 554d6af6d02c2 kube-proxy-whqjp
33227ebf9aad7 1611cd07b61d5 5 minutes ago Running busybox 1 903879911acc8 busybox
932274f4fc040 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 1de6e43eebf8b storage-provisioner
a72ce95e2b81f 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 d80a8b050e9b7 kube-controller-manager-old-k8s-version-135957
5b0aed246c27a 05b738aa1bc63 6 minutes ago Running etcd 1 f93781e502a30 etcd-old-k8s-version-135957
a86cc728bc887 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 8d6ebdc951d62 kube-apiserver-old-k8s-version-135957
855986d2034d9 e7605f88f17d6 6 minutes ago Running kube-scheduler 1 5c03bf2072449 kube-scheduler-old-k8s-version-135957
2e87a804ea0c4 1611cd07b61d5 6 minutes ago Exited busybox 0 c4b496d9d9b18 busybox
84036bbee69a2 db91994f4ee8f 8 minutes ago Exited coredns 0 17c4d6a21c1f4 coredns-74ff55c5b-txr44
c3976d1ea15f2 0bcd66b03df5f 8 minutes ago Exited kindnet-cni 0 dec6672f2aa64 kindnet-5rbc6
e8859b886635d 25a5233254979 8 minutes ago Exited kube-proxy 0 54d3ec4654041 kube-proxy-whqjp
d992b690b4b29 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 c708a812c31ac kube-controller-manager-old-k8s-version-135957
187e30802bad2 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 b26fe1a489965 kube-scheduler-old-k8s-version-135957
1c8e2d58066cf 05b738aa1bc63 8 minutes ago Exited etcd 0 babf9c5bfb80f etcd-old-k8s-version-135957
4ad7bf6ad0e52 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 91b8411ee1710 kube-apiserver-old-k8s-version-135957
==> containerd <==
Oct 09 19:36:56 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:56.824928819Z" level=info msg="StartContainer for \"f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4\""
Oct 09 19:36:56 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:56.921023735Z" level=info msg="StartContainer for \"f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4\" returns successfully"
Oct 09 19:36:56 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:56.955643088Z" level=info msg="shim disconnected" id=f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4 namespace=k8s.io
Oct 09 19:36:56 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:56.955708457Z" level=warning msg="cleaning up after shim disconnected" id=f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4 namespace=k8s.io
Oct 09 19:36:56 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:56.955719164Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Oct 09 19:36:57 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:57.298642213Z" level=info msg="RemoveContainer for \"6db0c2b67644e8f4508105525c5edf8a7ad5b746e42940cc51eade34a7afa6de\""
Oct 09 19:36:57 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:36:57.304346501Z" level=info msg="RemoveContainer for \"6db0c2b67644e8f4508105525c5edf8a7ad5b746e42940cc51eade34a7afa6de\" returns successfully"
Oct 09 19:38:03 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:03.794489427Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:38:03 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:03.800164186Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Oct 09 19:38:03 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:03.801889665Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Oct 09 19:38:03 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:03.801980117Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.797241040Z" level=info msg="CreateContainer within sandbox \"6be23148e49af1776409ef15eb37d2244e83f4904f75761ade1e8031f85793a5\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.817117869Z" level=info msg="CreateContainer within sandbox \"6be23148e49af1776409ef15eb37d2244e83f4904f75761ade1e8031f85793a5\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e\""
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.817787009Z" level=info msg="StartContainer for \"2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e\""
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.894629772Z" level=info msg="StartContainer for \"2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e\" returns successfully"
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.921083479Z" level=info msg="shim disconnected" id=2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e namespace=k8s.io
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.921143704Z" level=warning msg="cleaning up after shim disconnected" id=2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e namespace=k8s.io
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.921154633Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Oct 09 19:38:26 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:26.938721207Z" level=warning msg="cleanup warnings time=\"2024-10-09T19:38:26Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=k8s.io
Oct 09 19:38:27 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:27.531417930Z" level=info msg="RemoveContainer for \"f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4\""
Oct 09 19:38:27 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:38:27.535776571Z" level=info msg="RemoveContainer for \"f40d86867942c0c44ac19f02d48e43abc219e3e13771f3c8586222ed29129ad4\" returns successfully"
Oct 09 19:40:54 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:40:54.795185283Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:54 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:40:54.802911037Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Oct 09 19:40:54 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:40:54.804352764Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Oct 09 19:40:54 old-k8s-version-135957 containerd[566]: time="2024-10-09T19:40:54.804469791Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [63cf0a429f88fba3145d366c1fcc23f5cfdebced0ea1be2236fca2f81829cb33] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:57611 - 7565 "HINFO IN 9088653211537284142.8595202348181463480. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.014879514s
==> coredns [84036bbee69a29ef082da3b3087d897adbb11db9bdadc90f36135bd91a8b88e4] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:36419 - 7371 "HINFO IN 3087926151228416142.221234552813058234. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.043555675s
==> describe nodes <==
Name: old-k8s-version-135957
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-135957
kubernetes.io/os=linux
minikube.k8s.io/commit=0f31bfe1a852f6cc79fedfeb2462ff6b6d86b5e4
minikube.k8s.io/name=old-k8s-version-135957
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_10_09T19_32_22_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 09 Oct 2024 19:32:18 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-135957
AcquireTime: <unset>
RenewTime: Wed, 09 Oct 2024 19:40:55 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Wed, 09 Oct 2024 19:36:03 +0000 Wed, 09 Oct 2024 19:32:11 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Wed, 09 Oct 2024 19:36:03 +0000 Wed, 09 Oct 2024 19:32:11 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Wed, 09 Oct 2024 19:36:03 +0000 Wed, 09 Oct 2024 19:32:11 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Wed, 09 Oct 2024 19:36:03 +0000 Wed, 09 Oct 2024 19:32:37 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-135957
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
System Info:
Machine ID: bac715afa91442c9a804b44a38c50d28
System UUID: ca6776eb-7805-4d48-8777-68103ae2f9fd
Boot ID: 82386538-14d4-4a77-b4cb-0988d545cff7
Kernel Version: 5.15.0-1070-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m45s
kube-system coredns-74ff55c5b-txr44 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m22s
kube-system etcd-old-k8s-version-135957 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m29s
kube-system kindnet-5rbc6 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m22s
kube-system kube-apiserver-old-k8s-version-135957 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m29s
kube-system kube-controller-manager-old-k8s-version-135957 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m29s
kube-system kube-proxy-whqjp 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m22s
kube-system kube-scheduler-old-k8s-version-135957 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m29s
kube-system metrics-server-9975d5f86-jcsl5 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m33s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m20s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-hjq4h 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m39s
kubernetes-dashboard kubernetes-dashboard-cd95d586-5fpql 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m39s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m49s (x5 over 8m49s) kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m49s (x4 over 8m49s) kubelet Node old-k8s-version-135957 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m49s (x4 over 8m49s) kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientPID
Normal Starting 8m29s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m29s kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m29s kubelet Node old-k8s-version-135957 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m29s kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m29s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m22s kubelet Node old-k8s-version-135957 status is now: NodeReady
Normal Starting 8m21s kube-proxy Starting kube-proxy.
Normal Starting 6m5s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m5s (x8 over 6m5s) kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m5s (x8 over 6m5s) kubelet Node old-k8s-version-135957 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m5s (x7 over 6m5s) kubelet Node old-k8s-version-135957 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m5s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m54s kube-proxy Starting kube-proxy.
==> dmesg <==
[Oct 9 18:17] ACPI: SRAT not present
[ +0.000000] ACPI: SRAT not present
[ +0.000000] SPI driver altr_a10sr has no spi_device_id for altr,a10sr
[ +0.015212] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[ +0.462139] systemd[1]: Configuration file /run/systemd/system/netplan-ovs-cleanup.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
[ +0.053294] systemd[1]: /lib/systemd/system/cloud-init.service:20: Unknown key name 'ConditionEnvironment' in section 'Unit', ignoring.
[ +0.014996] systemd[1]: /lib/systemd/system/cloud-init-hotplugd.socket:11: Unknown key name 'ConditionEnvironment' in section 'Unit', ignoring.
[ +0.652682] ena 0000:00:05.0: LLQ is not supported Fallback to host mode policy.
[ +6.112018] kauditd_printk_skb: 36 callbacks suppressed
[Oct 9 19:26] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.
==> etcd [1c8e2d58066cf86a2a21f63512a26645c4aca5e9c0e35ab5524cf050d204d08f] <==
raft2024/10/09 19:32:11 INFO: ea7e25599daad906 is starting a new election at term 1
raft2024/10/09 19:32:11 INFO: ea7e25599daad906 became candidate at term 2
raft2024/10/09 19:32:11 INFO: ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2
raft2024/10/09 19:32:11 INFO: ea7e25599daad906 became leader at term 2
raft2024/10/09 19:32:11 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-10-09 19:32:11.423227 I | etcdserver: setting up the initial cluster version to 3.4
2024-10-09 19:32:11.423494 I | etcdserver: published {Name:old-k8s-version-135957 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-10-09 19:32:11.423692 I | embed: ready to serve client requests
2024-10-09 19:32:11.433298 I | embed: serving client requests on 192.168.76.2:2379
2024-10-09 19:32:11.433574 I | embed: ready to serve client requests
2024-10-09 19:32:11.434880 I | embed: serving client requests on 127.0.0.1:2379
2024-10-09 19:32:11.455668 N | etcdserver/membership: set the initial cluster version to 3.4
2024-10-09 19:32:11.455933 I | etcdserver/api: enabled capabilities for version 3.4
2024-10-09 19:32:36.623352 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:32:38.133096 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:32:48.132755 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:32:58.132872 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:08.132958 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:18.132782 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:28.132940 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:38.132731 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:48.132906 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:33:58.132735 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:34:08.132911 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:34:18.133075 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [5b0aed246c27ad7c1ddcacdb468f3f044ce18481900112163a8cf9ed5eea809a] <==
2024-10-09 19:36:58.951121 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:08.950366 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:18.950189 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:28.950193 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:38.950333 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:48.950333 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:37:58.950396 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:08.950202 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:18.950273 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:28.950168 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:38.950169 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:48.950075 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:38:58.950107 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:08.950141 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:18.950186 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:28.950151 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:38.950097 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:48.950271 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:39:58.950101 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:08.950546 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:18.950181 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:28.950077 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:38.950294 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:48.950172 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-10-09 19:40:58.950787 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
19:40:59 up 1:23, 0 users, load average: 2.82, 2.12, 2.42
Linux old-k8s-version-135957 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:48 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [c26ad772ee958513c626214202493556f6ebc49584722d4a63b9e146a275c4d9] <==
I1009 19:38:57.710276 1 main.go:300] handling current node
I1009 19:39:07.701781 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:07.701814 1 main.go:300] handling current node
I1009 19:39:17.708738 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:17.708774 1 main.go:300] handling current node
I1009 19:39:27.701688 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:27.701729 1 main.go:300] handling current node
I1009 19:39:37.702012 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:37.702044 1 main.go:300] handling current node
I1009 19:39:47.709788 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:47.709824 1 main.go:300] handling current node
I1009 19:39:57.709633 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:39:57.709668 1 main.go:300] handling current node
I1009 19:40:07.702069 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:07.702106 1 main.go:300] handling current node
I1009 19:40:17.710294 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:17.710331 1 main.go:300] handling current node
I1009 19:40:27.708796 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:27.709069 1 main.go:300] handling current node
I1009 19:40:37.702188 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:37.702222 1 main.go:300] handling current node
I1009 19:40:47.708720 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:47.708756 1 main.go:300] handling current node
I1009 19:40:57.710548 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:40:57.710621 1 main.go:300] handling current node
==> kindnet [c3976d1ea15f2ff131facc548ade3a7c232dbda5819961fc0b4a1d787965c66c] <==
I1009 19:32:40.901779 1 controller.go:342] Waiting for informer caches to sync
I1009 19:32:40.901786 1 shared_informer.go:313] Waiting for caches to sync for kube-network-policies
I1009 19:32:41.202338 1 shared_informer.go:320] Caches are synced for kube-network-policies
I1009 19:32:41.202377 1 metrics.go:61] Registering metrics
I1009 19:32:41.202616 1 controller.go:378] Syncing nftables rules
I1009 19:32:50.909243 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:32:50.909312 1 main.go:300] handling current node
I1009 19:33:00.901959 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:00.901995 1 main.go:300] handling current node
I1009 19:33:10.907781 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:10.907815 1 main.go:300] handling current node
I1009 19:33:20.907301 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:20.907336 1 main.go:300] handling current node
I1009 19:33:30.902215 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:30.902248 1 main.go:300] handling current node
I1009 19:33:40.902156 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:40.902218 1 main.go:300] handling current node
I1009 19:33:50.907771 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:33:50.907804 1 main.go:300] handling current node
I1009 19:34:00.909325 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:34:00.909358 1 main.go:300] handling current node
I1009 19:34:10.901683 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:34:10.901716 1 main.go:300] handling current node
I1009 19:34:20.905217 1 main.go:296] Handling node with IPs: map[192.168.76.2:{}]
I1009 19:34:20.905312 1 main.go:300] handling current node
==> kube-apiserver [4ad7bf6ad0e5265c4a9fd21e84e1e1af06eeff17564ed8007acfde961022a797] <==
I1009 19:32:19.524589 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I1009 19:32:19.540607 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I1009 19:32:19.548536 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I1009 19:32:19.548566 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I1009 19:32:20.040936 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I1009 19:32:20.095222 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W1009 19:32:20.219864 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I1009 19:32:20.221073 1 controller.go:606] quota admission added evaluator for: endpoints
I1009 19:32:20.226209 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I1009 19:32:20.563538 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I1009 19:32:21.289998 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I1009 19:32:21.757328 1 controller.go:606] quota admission added evaluator for: deployments.apps
I1009 19:32:21.813892 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I1009 19:32:37.220251 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I1009 19:32:37.385010 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I1009 19:32:51.024175 1 client.go:360] parsed scheme: "passthrough"
I1009 19:32:51.024215 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:32:51.024224 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1009 19:33:23.317564 1 client.go:360] parsed scheme: "passthrough"
I1009 19:33:23.317630 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:33:23.317639 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1009 19:34:00.577725 1 client.go:360] parsed scheme: "passthrough"
I1009 19:34:00.577803 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:34:00.577814 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
E1009 19:34:25.966579 1 available_controller.go:508] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
==> kube-apiserver [a86cc728bc8874c68b3d2f49c664827c126a21d2a5bdecc230cc8b936b97b880] <==
I1009 19:37:33.902863 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:37:33.902873 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W1009 19:38:05.652284 1 handler_proxy.go:102] no RequestInfo found in the context
E1009 19:38:05.652362 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I1009 19:38:05.652373 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I1009 19:38:09.226371 1 client.go:360] parsed scheme: "passthrough"
I1009 19:38:09.226412 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:38:09.226421 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1009 19:38:41.230084 1 client.go:360] parsed scheme: "passthrough"
I1009 19:38:41.230126 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:38:41.230135 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1009 19:39:16.475051 1 client.go:360] parsed scheme: "passthrough"
I1009 19:39:16.475093 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:39:16.475102 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I1009 19:39:53.152599 1 client.go:360] parsed scheme: "passthrough"
I1009 19:39:53.152678 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:39:53.152712 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W1009 19:40:04.234979 1 handler_proxy.go:102] no RequestInfo found in the context
E1009 19:40:04.235056 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I1009 19:40:04.235071 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I1009 19:40:35.028430 1 client.go:360] parsed scheme: "passthrough"
I1009 19:40:35.028483 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I1009 19:40:35.028492 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [a72ce95e2b81fd5c264bda698f6f4d1b0c484887be8796402a59c8bd18d9e184] <==
E1009 19:36:52.314054 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:36:57.907397 1 request.go:655] Throttling request took 1.048392656s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:36:58.758796 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:37:22.815852 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:37:30.409320 1 request.go:655] Throttling request took 1.04836041s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:37:31.260810 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:37:53.317732 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:38:02.911389 1 request.go:655] Throttling request took 1.048530451s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:38:03.762803 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:38:23.819833 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:38:35.413380 1 request.go:655] Throttling request took 1.048261533s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:38:36.266504 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:38:54.321694 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:39:07.916953 1 request.go:655] Throttling request took 1.048490434s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1beta1?timeout=32s
W1009 19:39:08.768401 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:39:24.823508 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:39:40.418854 1 request.go:655] Throttling request took 1.048253073s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:39:41.270205 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:39:55.402783 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:40:12.920597 1 request.go:655] Throttling request took 1.048307103s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:40:13.771971 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:40:25.904789 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I1009 19:40:45.422316 1 request.go:655] Throttling request took 1.04763807s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W1009 19:40:46.274236 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E1009 19:40:56.480518 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
==> kube-controller-manager [d992b690b4b295494e1d4053488cd50d6b5bb5c448447642a979dacd9803d0b4] <==
I1009 19:32:37.466449 1 shared_informer.go:247] Caches are synced for resource quota
I1009 19:32:37.466689 1 shared_informer.go:247] Caches are synced for taint
I1009 19:32:37.467445 1 node_lifecycle_controller.go:1429] Initializing eviction metric for zone:
W1009 19:32:37.467694 1 node_lifecycle_controller.go:1044] Missing timestamp for Node old-k8s-version-135957. Assuming now as a timestamp.
I1009 19:32:37.467963 1 node_lifecycle_controller.go:1195] Controller detected that all Nodes are not-Ready. Entering master disruption mode.
I1009 19:32:37.468434 1 event.go:291] "Event occurred" object="old-k8s-version-135957" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-135957 event: Registered Node old-k8s-version-135957 in Controller"
I1009 19:32:37.466702 1 shared_informer.go:247] Caches are synced for PV protection
I1009 19:32:37.468700 1 taint_manager.go:187] Starting NoExecuteTaintManager
I1009 19:32:37.467202 1 shared_informer.go:247] Caches are synced for expand
I1009 19:32:37.499885 1 shared_informer.go:247] Caches are synced for persistent volume
I1009 19:32:37.521259 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-whqjp"
I1009 19:32:37.522484 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-txr44"
I1009 19:32:37.526853 1 event.go:291] "Event occurred" object="kube-system/kube-controller-manager-old-k8s-version-135957" kind="Pod" apiVersion="v1" type="Warning" reason="NodeNotReady" message="Node is not ready"
I1009 19:32:37.537332 1 shared_informer.go:247] Caches are synced for attach detach
I1009 19:32:37.624979 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I1009 19:32:37.912168 1 shared_informer.go:247] Caches are synced for garbage collector
I1009 19:32:37.912188 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I1009 19:32:37.925245 1 shared_informer.go:247] Caches are synced for garbage collector
I1009 19:32:38.112056 1 request.go:655] Throttling request took 1.047256334s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
I1009 19:32:38.931858 1 shared_informer.go:240] Waiting for caches to sync for resource quota
I1009 19:32:38.932034 1 shared_informer.go:247] Caches are synced for resource quota
I1009 19:32:38.983823 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I1009 19:32:39.018562 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-mhkn2"
I1009 19:32:42.468351 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I1009 19:34:25.495175 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
==> kube-proxy [4d322977adb34505800303fa80faeb24ab7f9ebaab257bc66df4746a1d721454] <==
I1009 19:35:05.725219 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I1009 19:35:05.725361 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W1009 19:35:05.761099 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I1009 19:35:05.761287 1 server_others.go:185] Using iptables Proxier.
I1009 19:35:05.761641 1 server.go:650] Version: v1.20.0
I1009 19:35:05.762315 1 config.go:315] Starting service config controller
I1009 19:35:05.762420 1 shared_informer.go:240] Waiting for caches to sync for service config
I1009 19:35:05.762505 1 config.go:224] Starting endpoint slice config controller
I1009 19:35:05.762577 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I1009 19:35:05.862626 1 shared_informer.go:247] Caches are synced for service config
I1009 19:35:05.862681 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [e8859b886635d3afbca91152808b9f332358001778bf0d7aa5f21759a09bb89e] <==
I1009 19:32:38.431017 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I1009 19:32:38.431104 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W1009 19:32:38.459385 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I1009 19:32:38.459471 1 server_others.go:185] Using iptables Proxier.
I1009 19:32:38.459752 1 server.go:650] Version: v1.20.0
I1009 19:32:38.460244 1 config.go:315] Starting service config controller
I1009 19:32:38.460252 1 shared_informer.go:240] Waiting for caches to sync for service config
I1009 19:32:38.460959 1 config.go:224] Starting endpoint slice config controller
I1009 19:32:38.460966 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I1009 19:32:38.560381 1 shared_informer.go:247] Caches are synced for service config
I1009 19:32:38.564839 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [187e30802bad2aaeab65192a456cb8381d5f88344e4afb2f53890a5b849ebf1e] <==
I1009 19:32:16.023033 1 serving.go:331] Generated self-signed cert in-memory
W1009 19:32:18.813947 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W1009 19:32:18.814092 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W1009 19:32:18.814156 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W1009 19:32:18.814197 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I1009 19:32:18.874169 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I1009 19:32:18.879708 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I1009 19:32:18.879846 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1009 19:32:18.883346 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E1009 19:32:18.887212 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E1009 19:32:18.887726 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E1009 19:32:18.887926 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E1009 19:32:18.888174 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E1009 19:32:18.888836 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E1009 19:32:18.895416 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E1009 19:32:18.895526 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E1009 19:32:18.895582 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E1009 19:32:18.895714 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E1009 19:32:18.895760 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E1009 19:32:18.895823 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E1009 19:32:18.897413 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E1009 19:32:19.720273 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E1009 19:32:19.807119 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E1009 19:32:19.901047 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I1009 19:32:21.683686 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [855986d2034d90a9fde6bf1a7e0380afe758f230033029f7ba587777307ccadd] <==
I1009 19:34:57.568329 1 serving.go:331] Generated self-signed cert in-memory
W1009 19:35:02.842378 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W1009 19:35:02.847200 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W1009 19:35:02.847956 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W1009 19:35:02.848201 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I1009 19:35:03.280703 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1009 19:35:03.280744 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1009 19:35:03.281531 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I1009 19:35:03.281571 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I1009 19:35:03.381173 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Oct 09 19:39:27 old-k8s-version-135957 kubelet[655]: E1009 19:39:27.794317 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: I1009 19:39:30.793735 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:39:30 old-k8s-version-135957 kubelet[655]: E1009 19:39:30.794130 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:39:42 old-k8s-version-135957 kubelet[655]: E1009 19:39:42.794279 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: I1009 19:39:43.793569 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:39:43 old-k8s-version-135957 kubelet[655]: E1009 19:39:43.794072 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: I1009 19:39:56.805620 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.806354 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:39:56 old-k8s-version-135957 kubelet[655]: E1009 19:39:56.812240 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:07 old-k8s-version-135957 kubelet[655]: E1009 19:40:07.793973 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: I1009 19:40:11.793594 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:40:11 old-k8s-version-135957 kubelet[655]: E1009 19:40:11.794417 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:18 old-k8s-version-135957 kubelet[655]: E1009 19:40:18.794442 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: I1009 19:40:23.793618 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:40:23 old-k8s-version-135957 kubelet[655]: E1009 19:40:23.793992 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:30 old-k8s-version-135957 kubelet[655]: E1009 19:40:30.794850 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: I1009 19:40:37.793548 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:40:37 old-k8s-version-135957 kubelet[655]: E1009 19:40:37.793924 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:43 old-k8s-version-135957 kubelet[655]: E1009 19:40:43.794198 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Oct 09 19:40:49 old-k8s-version-135957 kubelet[655]: I1009 19:40:49.793621 655 scope.go:95] [topologymanager] RemoveContainer - Container ID: 2635990d5e8d9417e598217568baa18a891d68e348b67e4337d159507afccb3e
Oct 09 19:40:49 old-k8s-version-135957 kubelet[655]: E1009 19:40:49.793982 655 pod_workers.go:191] Error syncing pod f1441c31-5795-4d25-a253-1d407ce48354 ("dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-hjq4h_kubernetes-dashboard(f1441c31-5795-4d25-a253-1d407ce48354)"
Oct 09 19:40:54 old-k8s-version-135957 kubelet[655]: E1009 19:40:54.804705 655 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Oct 09 19:40:54 old-k8s-version-135957 kubelet[655]: E1009 19:40:54.804766 655 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Oct 09 19:40:54 old-k8s-version-135957 kubelet[655]: E1009 19:40:54.804928 655 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-n5d8s,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-jcsl5_kube-system(5a362d7
7-9bb8-435c-9f86-e1d8bbb32b46): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Oct 09 19:40:54 old-k8s-version-135957 kubelet[655]: E1009 19:40:54.805167 655 pod_workers.go:191] Error syncing pod 5a362d77-9bb8-435c-9f86-e1d8bbb32b46 ("metrics-server-9975d5f86-jcsl5_kube-system(5a362d77-9bb8-435c-9f86-e1d8bbb32b46)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
==> kubernetes-dashboard [ea122f5ea8fc6e36a226a62c42afab032c3c1cdec03c58593e3ae44bd5997a60] <==
2024/10/09 19:35:30 Using namespace: kubernetes-dashboard
2024/10/09 19:35:30 Using in-cluster config to connect to apiserver
2024/10/09 19:35:30 Using secret token for csrf signing
2024/10/09 19:35:30 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/10/09 19:35:30 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/10/09 19:35:30 Successful initial request to the apiserver, version: v1.20.0
2024/10/09 19:35:30 Generating JWE encryption key
2024/10/09 19:35:30 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/10/09 19:35:30 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/10/09 19:35:31 Initializing JWE encryption key from synchronized object
2024/10/09 19:35:31 Creating in-cluster Sidecar client
2024/10/09 19:35:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:35:31 Serving insecurely on HTTP port: 9090
2024/10/09 19:36:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:36:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:37:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:37:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:38:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:38:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:39:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:39:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:40:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:40:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/10/09 19:35:30 Starting overwatch
==> storage-provisioner [70879940cc7690fd1350f5136b44b129f7292089da7ad5afeabde81878228f4c] <==
I1009 19:35:49.967221 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I1009 19:35:50.036826 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I1009 19:35:50.039130 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I1009 19:36:07.550381 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I1009 19:36:07.550825 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-135957_69a63abd-7905-40cc-9489-c9d61be1bc57!
I1009 19:36:07.550537 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"867abbea-7973-464a-a387-d64438a13bad", APIVersion:"v1", ResourceVersion:"833", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-135957_69a63abd-7905-40cc-9489-c9d61be1bc57 became leader
I1009 19:36:07.651124 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-135957_69a63abd-7905-40cc-9489-c9d61be1bc57!
==> storage-provisioner [932274f4fc04057b5ff116ef31ac91a9a3bf6b7f70e2c7b9adbb29c3bd1c151f] <==
I1009 19:35:05.101401 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F1009 19:35:35.103751 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-135957 -n old-k8s-version-135957
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-135957 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-jcsl5
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-135957 describe pod metrics-server-9975d5f86-jcsl5
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-135957 describe pod metrics-server-9975d5f86-jcsl5: exit status 1 (148.858645ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-jcsl5" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-135957 describe pod metrics-server-9975d5f86-jcsl5: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (382.83s)