=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-011723 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
E0912 23:21:58.517352 1597760 cert_rotation.go:171] "Unhandled Error" err="key failed with : open /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/functional-209375/client.crt: no such file or directory" logger="UnhandledError"
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-011723 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m16.220230253s)
-- stdout --
* [old-k8s-version-011723] minikube v1.34.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19616
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19616-1592376/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19616-1592376/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-011723" primary control-plane node in "old-k8s-version-011723" cluster
* Pulling base image v0.0.45-1726156396-19616 ...
* Restarting existing docker container for "old-k8s-version-011723" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
* Verifying Kubernetes components...
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-011723 addons enable metrics-server
* Enabled addons: metrics-server, dashboard, default-storageclass, storage-provisioner
-- /stdout --
** stderr **
I0912 23:21:38.890947 1805825 out.go:345] Setting OutFile to fd 1 ...
I0912 23:21:38.891126 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:21:38.891134 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:21:38.891139 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:21:38.891418 1805825 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19616-1592376/.minikube/bin
I0912 23:21:38.891863 1805825 out.go:352] Setting JSON to false
I0912 23:21:38.892855 1805825 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":29026,"bootTime":1726154273,"procs":199,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1068-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0912 23:21:38.892986 1805825 start.go:139] virtualization:
I0912 23:21:38.895649 1805825 out.go:177] * [old-k8s-version-011723] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0912 23:21:38.898176 1805825 out.go:177] - MINIKUBE_LOCATION=19616
I0912 23:21:38.898235 1805825 notify.go:220] Checking for updates...
I0912 23:21:38.901980 1805825 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0912 23:21:38.903988 1805825 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:21:38.905622 1805825 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19616-1592376/.minikube
I0912 23:21:38.907240 1805825 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0912 23:21:38.909325 1805825 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0912 23:21:38.911422 1805825 config.go:182] Loaded profile config "old-k8s-version-011723": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0912 23:21:38.914169 1805825 out.go:177] * Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
I0912 23:21:38.915960 1805825 driver.go:394] Setting default libvirt URI to qemu:///system
I0912 23:21:38.949352 1805825 docker.go:123] docker version: linux-27.2.1:Docker Engine - Community
I0912 23:21:38.949484 1805825 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0912 23:21:39.037383 1805825 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:49 OomKillDisable:true NGoroutines:68 SystemTime:2024-09-12 23:21:39.026191066 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214827008 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.2.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0912 23:21:39.037501 1805825 docker.go:318] overlay module found
I0912 23:21:39.039832 1805825 out.go:177] * Using the docker driver based on existing profile
I0912 23:21:39.041676 1805825 start.go:297] selected driver: docker
I0912 23:21:39.041691 1805825 start.go:901] validating driver "docker" against &{Name:old-k8s-version-011723 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-011723 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:21:39.041805 1805825 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0912 23:21:39.042474 1805825 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0912 23:21:39.128829 1805825 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:49 OomKillDisable:true NGoroutines:68 SystemTime:2024-09-12 23:21:39.118558996 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214827008 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.2.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0912 23:21:39.129185 1805825 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0912 23:21:39.129212 1805825 cni.go:84] Creating CNI manager for ""
I0912 23:21:39.129220 1805825 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0912 23:21:39.129259 1805825 start.go:340] cluster config:
{Name:old-k8s-version-011723 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-011723 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:21:39.131676 1805825 out.go:177] * Starting "old-k8s-version-011723" primary control-plane node in "old-k8s-version-011723" cluster
I0912 23:21:39.133844 1805825 cache.go:121] Beginning downloading kic base image for docker with containerd
I0912 23:21:39.136055 1805825 out.go:177] * Pulling base image v0.0.45-1726156396-19616 ...
I0912 23:21:39.138050 1805825 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0912 23:21:39.138136 1805825 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0912 23:21:39.138150 1805825 cache.go:56] Caching tarball of preloaded images
I0912 23:21:39.138246 1805825 preload.go:172] Found /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0912 23:21:39.138268 1805825 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0912 23:21:39.138412 1805825 profile.go:143] Saving config to /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/config.json ...
I0912 23:21:39.138646 1805825 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local docker daemon
W0912 23:21:39.166036 1805825 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 is of wrong architecture
I0912 23:21:39.166064 1805825 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 to local cache
I0912 23:21:39.166183 1805825 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local cache directory
I0912 23:21:39.166219 1805825 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local cache directory, skipping pull
I0912 23:21:39.166229 1805825 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 exists in cache, skipping pull
I0912 23:21:39.166238 1805825 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 as a tarball
I0912 23:21:39.166243 1805825 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 from local cache
I0912 23:21:39.306440 1805825 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 from cached tarball
I0912 23:21:39.306477 1805825 cache.go:194] Successfully downloaded all kic artifacts
I0912 23:21:39.306513 1805825 start.go:360] acquireMachinesLock for old-k8s-version-011723: {Name:mke0556fcf2dfec23d3db329fbb0d4d87739d4fb Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:21:39.306581 1805825 start.go:364] duration metric: took 42.215µs to acquireMachinesLock for "old-k8s-version-011723"
I0912 23:21:39.306607 1805825 start.go:96] Skipping create...Using existing machine configuration
I0912 23:21:39.306616 1805825 fix.go:54] fixHost starting:
I0912 23:21:39.306895 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:39.367632 1805825 fix.go:112] recreateIfNeeded on old-k8s-version-011723: state=Stopped err=<nil>
W0912 23:21:39.367659 1805825 fix.go:138] unexpected machine state, will restart: <nil>
I0912 23:21:39.370020 1805825 out.go:177] * Restarting existing docker container for "old-k8s-version-011723" ...
I0912 23:21:39.371586 1805825 cli_runner.go:164] Run: docker start old-k8s-version-011723
I0912 23:21:39.745707 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:39.769887 1805825 kic.go:430] container "old-k8s-version-011723" state is running.
I0912 23:21:39.770309 1805825 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-011723
I0912 23:21:39.806794 1805825 profile.go:143] Saving config to /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/config.json ...
I0912 23:21:39.807023 1805825 machine.go:93] provisionDockerMachine start ...
I0912 23:21:39.807099 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:39.837003 1805825 main.go:141] libmachine: Using SSH client type: native
I0912 23:21:39.837267 1805825 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34934 <nil> <nil>}
I0912 23:21:39.837275 1805825 main.go:141] libmachine: About to run SSH command:
hostname
I0912 23:21:39.838942 1805825 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0912 23:21:42.987716 1805825 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-011723
I0912 23:21:42.987742 1805825 ubuntu.go:169] provisioning hostname "old-k8s-version-011723"
I0912 23:21:42.987817 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:43.009406 1805825 main.go:141] libmachine: Using SSH client type: native
I0912 23:21:43.009686 1805825 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34934 <nil> <nil>}
I0912 23:21:43.009705 1805825 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-011723 && echo "old-k8s-version-011723" | sudo tee /etc/hostname
I0912 23:21:43.164820 1805825 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-011723
I0912 23:21:43.164927 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:43.184487 1805825 main.go:141] libmachine: Using SSH client type: native
I0912 23:21:43.184737 1805825 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34934 <nil> <nil>}
I0912 23:21:43.184761 1805825 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-011723' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-011723/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-011723' | sudo tee -a /etc/hosts;
fi
fi
I0912 23:21:43.327875 1805825 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0912 23:21:43.327903 1805825 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19616-1592376/.minikube CaCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19616-1592376/.minikube}
I0912 23:21:43.327982 1805825 ubuntu.go:177] setting up certificates
I0912 23:21:43.327992 1805825 provision.go:84] configureAuth start
I0912 23:21:43.328082 1805825 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-011723
I0912 23:21:43.351828 1805825 provision.go:143] copyHostCerts
I0912 23:21:43.351897 1805825 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem, removing ...
I0912 23:21:43.351908 1805825 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem
I0912 23:21:43.351993 1805825 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem (1082 bytes)
I0912 23:21:43.352089 1805825 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem, removing ...
I0912 23:21:43.352094 1805825 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem
I0912 23:21:43.352120 1805825 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem (1123 bytes)
I0912 23:21:43.352170 1805825 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem, removing ...
I0912 23:21:43.352175 1805825 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem
I0912 23:21:43.352197 1805825 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem (1675 bytes)
I0912 23:21:43.352242 1805825 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-011723 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-011723]
I0912 23:21:43.550007 1805825 provision.go:177] copyRemoteCerts
I0912 23:21:43.550113 1805825 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0912 23:21:43.550202 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:43.573051 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:43.674416 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0912 23:21:43.699613 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0912 23:21:43.724271 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0912 23:21:43.748504 1805825 provision.go:87] duration metric: took 420.491239ms to configureAuth
I0912 23:21:43.748535 1805825 ubuntu.go:193] setting minikube options for container-runtime
I0912 23:21:43.748770 1805825 config.go:182] Loaded profile config "old-k8s-version-011723": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0912 23:21:43.748785 1805825 machine.go:96] duration metric: took 3.941752167s to provisionDockerMachine
I0912 23:21:43.748794 1805825 start.go:293] postStartSetup for "old-k8s-version-011723" (driver="docker")
I0912 23:21:43.748805 1805825 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0912 23:21:43.748861 1805825 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0912 23:21:43.748907 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:43.765295 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:43.864941 1805825 ssh_runner.go:195] Run: cat /etc/os-release
I0912 23:21:43.868996 1805825 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0912 23:21:43.869030 1805825 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0912 23:21:43.869040 1805825 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0912 23:21:43.869047 1805825 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0912 23:21:43.869058 1805825 filesync.go:126] Scanning /home/jenkins/minikube-integration/19616-1592376/.minikube/addons for local assets ...
I0912 23:21:43.869115 1805825 filesync.go:126] Scanning /home/jenkins/minikube-integration/19616-1592376/.minikube/files for local assets ...
I0912 23:21:43.869198 1805825 filesync.go:149] local asset: /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem -> 15977602.pem in /etc/ssl/certs
I0912 23:21:43.869320 1805825 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0912 23:21:43.882279 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem --> /etc/ssl/certs/15977602.pem (1708 bytes)
I0912 23:21:43.924880 1805825 start.go:296] duration metric: took 176.069334ms for postStartSetup
I0912 23:21:43.924970 1805825 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0912 23:21:43.925015 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:43.947161 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:44.049591 1805825 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0912 23:21:44.054560 1805825 fix.go:56] duration metric: took 4.747937312s for fixHost
I0912 23:21:44.054628 1805825 start.go:83] releasing machines lock for "old-k8s-version-011723", held for 4.748033181s
I0912 23:21:44.054744 1805825 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-011723
I0912 23:21:44.083847 1805825 ssh_runner.go:195] Run: cat /version.json
I0912 23:21:44.083907 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:44.084214 1805825 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0912 23:21:44.084269 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:44.105067 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:44.112583 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:44.211310 1805825 ssh_runner.go:195] Run: systemctl --version
I0912 23:21:44.372114 1805825 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0912 23:21:44.377218 1805825 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0912 23:21:44.402615 1805825 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0912 23:21:44.402729 1805825 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0912 23:21:44.420423 1805825 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0912 23:21:44.420457 1805825 start.go:495] detecting cgroup driver to use...
I0912 23:21:44.420489 1805825 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0912 23:21:44.420573 1805825 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0912 23:21:44.444347 1805825 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0912 23:21:44.468484 1805825 docker.go:217] disabling cri-docker service (if available) ...
I0912 23:21:44.468606 1805825 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0912 23:21:44.490027 1805825 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0912 23:21:44.506251 1805825 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0912 23:21:44.670128 1805825 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0912 23:21:44.811005 1805825 docker.go:233] disabling docker service ...
I0912 23:21:44.811153 1805825 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0912 23:21:44.840274 1805825 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0912 23:21:44.856871 1805825 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0912 23:21:45.012001 1805825 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0912 23:21:45.167076 1805825 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0912 23:21:45.192342 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0912 23:21:45.237253 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0912 23:21:45.254953 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0912 23:21:45.268702 1805825 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0912 23:21:45.268915 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0912 23:21:45.291125 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0912 23:21:45.315046 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0912 23:21:45.330923 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0912 23:21:45.351175 1805825 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0912 23:21:45.368195 1805825 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0912 23:21:45.382082 1805825 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0912 23:21:45.398388 1805825 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0912 23:21:45.409137 1805825 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:21:45.556627 1805825 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0912 23:21:45.822717 1805825 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0912 23:21:45.822862 1805825 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0912 23:21:45.826897 1805825 start.go:563] Will wait 60s for crictl version
I0912 23:21:45.826965 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:21:45.840647 1805825 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0912 23:21:45.918057 1805825 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0912 23:21:45.918215 1805825 ssh_runner.go:195] Run: containerd --version
I0912 23:21:45.959798 1805825 ssh_runner.go:195] Run: containerd --version
I0912 23:21:45.998942 1805825 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
I0912 23:21:46.001042 1805825 cli_runner.go:164] Run: docker network inspect old-k8s-version-011723 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0912 23:21:46.032040 1805825 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0912 23:21:46.038398 1805825 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0912 23:21:46.055429 1805825 kubeadm.go:883] updating cluster {Name:old-k8s-version-011723 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-011723 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0912 23:21:46.055557 1805825 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0912 23:21:46.055633 1805825 ssh_runner.go:195] Run: sudo crictl images --output json
I0912 23:21:46.130609 1805825 containerd.go:627] all images are preloaded for containerd runtime.
I0912 23:21:46.130638 1805825 containerd.go:534] Images already preloaded, skipping extraction
I0912 23:21:46.130701 1805825 ssh_runner.go:195] Run: sudo crictl images --output json
I0912 23:21:46.195878 1805825 containerd.go:627] all images are preloaded for containerd runtime.
I0912 23:21:46.195901 1805825 cache_images.go:84] Images are preloaded, skipping loading
I0912 23:21:46.195910 1805825 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I0912 23:21:46.196036 1805825 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-011723 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-011723 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0912 23:21:46.196106 1805825 ssh_runner.go:195] Run: sudo crictl info
I0912 23:21:46.269895 1805825 cni.go:84] Creating CNI manager for ""
I0912 23:21:46.269961 1805825 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0912 23:21:46.269984 1805825 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0912 23:21:46.270017 1805825 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-011723 NodeName:old-k8s-version-011723 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0912 23:21:46.270226 1805825 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-011723"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0912 23:21:46.270315 1805825 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0912 23:21:46.280973 1805825 binaries.go:44] Found k8s binaries, skipping transfer
I0912 23:21:46.281094 1805825 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0912 23:21:46.304231 1805825 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0912 23:21:46.331249 1805825 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0912 23:21:46.365447 1805825 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0912 23:21:46.392960 1805825 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0912 23:21:46.397948 1805825 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0912 23:21:46.411335 1805825 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:21:46.568064 1805825 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0912 23:21:46.595784 1805825 certs.go:68] Setting up /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723 for IP: 192.168.76.2
I0912 23:21:46.595855 1805825 certs.go:194] generating shared ca certs ...
I0912 23:21:46.595886 1805825 certs.go:226] acquiring lock for ca certs: {Name:mk5b7cca91a053f0ec1ca9c487c600f7eefaa6e5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:21:46.596085 1805825 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.key
I0912 23:21:46.596156 1805825 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.key
I0912 23:21:46.596191 1805825 certs.go:256] generating profile certs ...
I0912 23:21:46.596341 1805825 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/client.key
I0912 23:21:46.596445 1805825 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/apiserver.key.b23fdb97
I0912 23:21:46.596509 1805825 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/proxy-client.key
I0912 23:21:46.596656 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760.pem (1338 bytes)
W0912 23:21:46.596719 1805825 certs.go:480] ignoring /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760_empty.pem, impossibly tiny 0 bytes
I0912 23:21:46.596744 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem (1679 bytes)
I0912 23:21:46.596798 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem (1082 bytes)
I0912 23:21:46.596847 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem (1123 bytes)
I0912 23:21:46.596900 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem (1675 bytes)
I0912 23:21:46.596980 1805825 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem (1708 bytes)
I0912 23:21:46.597717 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0912 23:21:46.697569 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0912 23:21:46.774306 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0912 23:21:46.838650 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0912 23:21:46.887456 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0912 23:21:46.916077 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0912 23:21:46.953731 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0912 23:21:46.996489 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/old-k8s-version-011723/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0912 23:21:47.049121 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem --> /usr/share/ca-certificates/15977602.pem (1708 bytes)
I0912 23:21:47.089641 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0912 23:21:47.132266 1805825 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760.pem --> /usr/share/ca-certificates/1597760.pem (1338 bytes)
I0912 23:21:47.171808 1805825 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0912 23:21:47.197898 1805825 ssh_runner.go:195] Run: openssl version
I0912 23:21:47.205253 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0912 23:21:47.217755 1805825 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0912 23:21:47.222052 1805825 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 12 22:30 /usr/share/ca-certificates/minikubeCA.pem
I0912 23:21:47.222172 1805825 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0912 23:21:47.240252 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0912 23:21:47.266541 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1597760.pem && ln -fs /usr/share/ca-certificates/1597760.pem /etc/ssl/certs/1597760.pem"
I0912 23:21:47.280316 1805825 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1597760.pem
I0912 23:21:47.284193 1805825 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 12 22:41 /usr/share/ca-certificates/1597760.pem
I0912 23:21:47.284296 1805825 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1597760.pem
I0912 23:21:47.293344 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1597760.pem /etc/ssl/certs/51391683.0"
I0912 23:21:47.303956 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15977602.pem && ln -fs /usr/share/ca-certificates/15977602.pem /etc/ssl/certs/15977602.pem"
I0912 23:21:47.313453 1805825 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15977602.pem
I0912 23:21:47.317250 1805825 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 12 22:41 /usr/share/ca-certificates/15977602.pem
I0912 23:21:47.317336 1805825 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15977602.pem
I0912 23:21:47.324530 1805825 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15977602.pem /etc/ssl/certs/3ec20f2e.0"
I0912 23:21:47.337564 1805825 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0912 23:21:47.341689 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0912 23:21:47.348929 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0912 23:21:47.360059 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0912 23:21:47.372499 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0912 23:21:47.380447 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0912 23:21:47.392556 1805825 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0912 23:21:47.403689 1805825 kubeadm.go:392] StartCluster: {Name:old-k8s-version-011723 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-011723 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:21:47.403812 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0912 23:21:47.403901 1805825 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0912 23:21:47.465033 1805825 cri.go:89] found id: "91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:21:47.465057 1805825 cri.go:89] found id: "c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:21:47.465063 1805825 cri.go:89] found id: "2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:21:47.465075 1805825 cri.go:89] found id: "f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:21:47.465078 1805825 cri.go:89] found id: "a39a9cadf13eda3be1ada9a90d5ba41f45987002f8b6bc8a8cb57f06dd3c71c5"
I0912 23:21:47.465082 1805825 cri.go:89] found id: "5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:21:47.465085 1805825 cri.go:89] found id: "ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:21:47.465088 1805825 cri.go:89] found id: "e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:21:47.465091 1805825 cri.go:89] found id: "f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:21:47.465099 1805825 cri.go:89] found id: ""
I0912 23:21:47.465176 1805825 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0912 23:21:47.485486 1805825 cri.go:116] JSON = null
W0912 23:21:47.485549 1805825 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 9
I0912 23:21:47.485620 1805825 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0912 23:21:47.494932 1805825 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0912 23:21:47.494953 1805825 kubeadm.go:593] restartPrimaryControlPlane start ...
I0912 23:21:47.495014 1805825 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0912 23:21:47.503587 1805825 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0912 23:21:47.504101 1805825 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-011723" does not appear in /home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:21:47.504247 1805825 kubeconfig.go:62] /home/jenkins/minikube-integration/19616-1592376/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-011723" cluster setting kubeconfig missing "old-k8s-version-011723" context setting]
I0912 23:21:47.504607 1805825 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19616-1592376/kubeconfig: {Name:mk20814b10c438de6fa8214738e210df331cf1df Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:21:47.506168 1805825 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0912 23:21:47.515196 1805825 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0912 23:21:47.515240 1805825 kubeadm.go:597] duration metric: took 20.27097ms to restartPrimaryControlPlane
I0912 23:21:47.515250 1805825 kubeadm.go:394] duration metric: took 111.57583ms to StartCluster
I0912 23:21:47.515268 1805825 settings.go:142] acquiring lock: {Name:mk1fdbbc4ffc0e3fc6419399beeda4839e1c5a1d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:21:47.515339 1805825 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:21:47.516043 1805825 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19616-1592376/kubeconfig: {Name:mk20814b10c438de6fa8214738e210df331cf1df Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:21:47.516275 1805825 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0912 23:21:47.516618 1805825 config.go:182] Loaded profile config "old-k8s-version-011723": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0912 23:21:47.516668 1805825 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0912 23:21:47.516765 1805825 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-011723"
I0912 23:21:47.516786 1805825 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-011723"
I0912 23:21:47.516794 1805825 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-011723"
W0912 23:21:47.516801 1805825 addons.go:243] addon storage-provisioner should already be in state true
I0912 23:21:47.516814 1805825 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-011723"
I0912 23:21:47.516822 1805825 host.go:66] Checking if "old-k8s-version-011723" exists ...
I0912 23:21:47.517112 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:47.517595 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:47.517967 1805825 addons.go:69] Setting dashboard=true in profile "old-k8s-version-011723"
I0912 23:21:47.518009 1805825 addons.go:234] Setting addon dashboard=true in "old-k8s-version-011723"
W0912 23:21:47.518027 1805825 addons.go:243] addon dashboard should already be in state true
I0912 23:21:47.518053 1805825 host.go:66] Checking if "old-k8s-version-011723" exists ...
I0912 23:21:47.518458 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:47.520181 1805825 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-011723"
I0912 23:21:47.520212 1805825 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-011723"
W0912 23:21:47.520219 1805825 addons.go:243] addon metrics-server should already be in state true
I0912 23:21:47.520245 1805825 host.go:66] Checking if "old-k8s-version-011723" exists ...
I0912 23:21:47.520698 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:47.521225 1805825 out.go:177] * Verifying Kubernetes components...
I0912 23:21:47.522895 1805825 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:21:47.577862 1805825 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0912 23:21:47.579858 1805825 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0912 23:21:47.579893 1805825 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0912 23:21:47.579978 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:47.593553 1805825 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0912 23:21:47.594359 1805825 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-011723"
W0912 23:21:47.594379 1805825 addons.go:243] addon default-storageclass should already be in state true
I0912 23:21:47.594403 1805825 host.go:66] Checking if "old-k8s-version-011723" exists ...
I0912 23:21:47.594826 1805825 cli_runner.go:164] Run: docker container inspect old-k8s-version-011723 --format={{.State.Status}}
I0912 23:21:47.599792 1805825 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0912 23:21:47.602468 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0912 23:21:47.602505 1805825 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0912 23:21:47.602577 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:47.602710 1805825 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0912 23:21:47.607879 1805825 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:47.607902 1805825 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0912 23:21:47.607970 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:47.646557 1805825 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0912 23:21:47.646580 1805825 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0912 23:21:47.646643 1805825 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-011723
I0912 23:21:47.652027 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:47.674870 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:47.694842 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:47.704310 1805825 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34934 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/old-k8s-version-011723/id_rsa Username:docker}
I0912 23:21:47.751852 1805825 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0912 23:21:47.817615 1805825 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-011723" to be "Ready" ...
I0912 23:21:47.889174 1805825 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0912 23:21:47.889208 1805825 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0912 23:21:47.922659 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0912 23:21:47.922690 1805825 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0912 23:21:47.954852 1805825 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0912 23:21:47.954881 1805825 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0912 23:21:47.968520 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0912 23:21:48.001928 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:48.006294 1805825 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:48.006337 1805825 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0912 23:21:48.026291 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0912 23:21:48.026332 1805825 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0912 23:21:48.110899 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:48.118774 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0912 23:21:48.118803 1805825 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0912 23:21:48.241972 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0912 23:21:48.241996 1805825 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0912 23:21:48.380708 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.380760 1805825 retry.go:31] will retry after 367.994911ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:48.381681 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.381704 1805825 retry.go:31] will retry after 324.286224ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.401643 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0912 23:21:48.401689 1805825 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0912 23:21:48.453598 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0912 23:21:48.453642 1805825 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0912 23:21:48.458673 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.458714 1805825 retry.go:31] will retry after 317.821356ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.476660 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0912 23:21:48.476703 1805825 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0912 23:21:48.518324 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0912 23:21:48.518371 1805825 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0912 23:21:48.544612 1805825 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0912 23:21:48.544639 1805825 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0912 23:21:48.564754 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:48.668219 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.668253 1805825 retry.go:31] will retry after 283.447867ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.706429 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:48.749818 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0912 23:21:48.777179 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:48.952384 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:48.973383 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:48.973418 1805825 retry.go:31] will retry after 264.60019ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:49.045737 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:49.045789 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.045805 1805825 retry.go:31] will retry after 555.894279ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.045825 1805825 retry.go:31] will retry after 220.174896ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:49.148450 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.148486 1805825 retry.go:31] will retry after 553.997483ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.238780 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:49.267210 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0912 23:21:49.380232 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.380277 1805825 retry.go:31] will retry after 465.301688ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:49.425270 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.425313 1805825 retry.go:31] will retry after 806.593407ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.602706 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:49.703162 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:49.747906 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.747952 1805825 retry.go:31] will retry after 727.26024ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.818684 1805825 node_ready.go:53] error getting node "old-k8s-version-011723": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-011723": dial tcp 192.168.76.2:8443: connect: connection refused
I0912 23:21:49.845873 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0912 23:21:49.884101 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.884150 1805825 retry.go:31] will retry after 686.280064ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:49.998378 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:49.998422 1805825 retry.go:31] will retry after 473.002247ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.232925 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0912 23:21:50.331938 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.331972 1805825 retry.go:31] will retry after 1.123653662s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.472370 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:50.475653 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:50.570810 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:50.603305 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.603346 1805825 retry.go:31] will retry after 1.465769992s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:50.649493 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.649528 1805825 retry.go:31] will retry after 763.852406ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:50.684655 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:50.684689 1805825 retry.go:31] will retry after 647.993812ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:51.333365 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0912 23:21:51.413750 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:51.456005 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0912 23:21:51.473322 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:51.473402 1805825 retry.go:31] will retry after 736.248769ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:51.554377 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:51.554462 1805825 retry.go:31] will retry after 1.446233953s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0912 23:21:51.584461 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:51.584494 1805825 retry.go:31] will retry after 1.637572967s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:52.069675 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0912 23:21:52.168739 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:52.168772 1805825 retry.go:31] will retry after 1.440286436s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:52.209994 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:52.288169 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:52.288241 1805825 retry.go:31] will retry after 2.44389615s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:52.318744 1805825 node_ready.go:53] error getting node "old-k8s-version-011723": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-011723": dial tcp 192.168.76.2:8443: connect: connection refused
I0912 23:21:53.001904 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0912 23:21:53.090150 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:53.090190 1805825 retry.go:31] will retry after 979.283008ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:53.222271 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0912 23:21:53.313223 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:53.313263 1805825 retry.go:31] will retry after 1.959626442s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:53.610137 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0912 23:21:53.757077 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:53.757159 1805825 retry.go:31] will retry after 2.839734726s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:54.070581 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0912 23:21:54.163478 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:54.163590 1805825 retry.go:31] will retry after 4.15322071s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:54.319188 1805825 node_ready.go:53] error getting node "old-k8s-version-011723": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-011723": dial tcp 192.168.76.2:8443: connect: connection refused
I0912 23:21:54.732388 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0912 23:21:54.805251 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:54.805309 1805825 retry.go:31] will retry after 2.322702818s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:55.273268 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0912 23:21:55.352955 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:55.352988 1805825 retry.go:31] will retry after 3.819769319s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0912 23:21:56.597696 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:21:57.128607 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0912 23:21:58.317001 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:21:59.173261 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0912 23:22:06.319617 1805825 node_ready.go:53] error getting node "old-k8s-version-011723": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-011723": net/http: TLS handshake timeout
I0912 23:22:06.813399 1805825 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (10.215623892s)
W0912 23:22:06.813431 1805825 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Unable to connect to the server: net/http: TLS handshake timeout
I0912 23:22:06.813448 1805825 retry.go:31] will retry after 5.944328604s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
Unable to connect to the server: net/http: TLS handshake timeout
I0912 23:22:07.480812 1805825 node_ready.go:49] node "old-k8s-version-011723" has status "Ready":"True"
I0912 23:22:07.480838 1805825 node_ready.go:38] duration metric: took 19.663179653s for node "old-k8s-version-011723" to be "Ready" ...
I0912 23:22:07.480849 1805825 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0912 23:22:07.571300 1805825 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace to be "Ready" ...
I0912 23:22:08.691647 1805825 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (11.562977276s)
I0912 23:22:08.692016 1805825 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (10.374984775s)
I0912 23:22:08.692076 1805825 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-011723"
I0912 23:22:08.692148 1805825 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (9.518866082s)
I0912 23:22:08.693671 1805825 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-011723 addons enable metrics-server
I0912 23:22:09.578080 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:11.579414 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:12.757956 1805825 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:22:13.159084 1805825 out.go:177] * Enabled addons: metrics-server, dashboard, default-storageclass, storage-provisioner
I0912 23:22:13.161032 1805825 addons.go:510] duration metric: took 25.64435318s for enable addons: enabled=[metrics-server dashboard default-storageclass storage-provisioner]
I0912 23:22:14.082930 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:16.578216 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:18.579158 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:21.078220 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:23.081750 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:25.607191 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:28.080856 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:30.083681 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:32.578903 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:34.579141 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:37.078928 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:39.579194 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:42.079359 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:44.577986 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:47.079427 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:49.577063 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:51.581169 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:52.578988 1805825 pod_ready.go:93] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"True"
I0912 23:22:52.579010 1805825 pod_ready.go:82] duration metric: took 45.007601623s for pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace to be "Ready" ...
I0912 23:22:52.579022 1805825 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:22:54.585950 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:57.086582 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:59.091309 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:01.584645 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:03.584803 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:05.586373 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:07.703232 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:10.092109 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:12.586070 1805825 pod_ready.go:93] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:12.586091 1805825 pod_ready.go:82] duration metric: took 20.007061143s for pod "etcd-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.586105 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.592877 1805825 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:12.592899 1805825 pod_ready.go:82] duration metric: took 6.785661ms for pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.592910 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:14.599464 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:16.600010 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:18.600119 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:21.099631 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:23.598727 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:25.599494 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:28.099402 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:30.108642 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:31.599271 1805825 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.599300 1805825 pod_ready.go:82] duration metric: took 19.006382161s for pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.599314 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-cd4m4" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.604530 1805825 pod_ready.go:93] pod "kube-proxy-cd4m4" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.604558 1805825 pod_ready.go:82] duration metric: took 5.236086ms for pod "kube-proxy-cd4m4" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.604570 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.609948 1805825 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.609978 1805825 pod_ready.go:82] duration metric: took 5.399884ms for pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.609991 1805825 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace to be "Ready" ...
I0912 23:23:33.616901 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:36.117406 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:38.616767 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:41.116618 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:43.117109 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:45.119017 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:47.615873 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:49.616264 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:51.616446 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:54.117042 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:56.617982 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:58.618639 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:01.116490 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:03.615956 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:05.616291 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:07.616383 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:10.118971 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:12.615907 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:14.616327 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:16.616660 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:18.618315 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:21.116721 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:23.120405 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:25.615987 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:27.616285 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:29.616736 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:32.116368 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:34.615597 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:36.615875 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:39.116734 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:41.616349 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:43.616829 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:46.116401 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:48.116475 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:50.116632 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:52.616460 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:55.116563 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:57.116939 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:59.615938 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:01.616962 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:04.117033 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:06.615106 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:08.697132 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:11.117036 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:13.615834 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:15.616644 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:18.118163 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:20.616069 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:23.116443 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:25.118089 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:27.618412 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:30.118003 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:32.615831 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:34.616187 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:36.616671 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:39.116087 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:41.615833 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:43.615886 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:45.616059 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:48.116800 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:50.616480 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:53.115634 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:55.117262 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:57.615811 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:59.617089 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:01.617125 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:04.116539 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:06.615639 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:08.616084 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:10.616278 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:13.116566 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:15.118162 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:17.616104 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:19.616529 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:21.617541 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:24.116954 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:26.618594 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:29.135672 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:31.616184 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:34.117097 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:36.616730 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:39.115929 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:41.116884 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:43.116963 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:45.118983 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:47.617481 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:49.623855 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:52.116732 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:54.125647 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:56.615873 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:58.625134 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:01.117368 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:03.615687 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:05.616841 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:07.617150 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:10.117503 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:12.617018 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:15.117143 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:17.120989 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:19.616374 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:22.117190 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:24.615625 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:26.615774 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:28.619296 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:30.620004 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:31.616410 1805825 pod_ready.go:82] duration metric: took 4m0.006405602s for pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace to be "Ready" ...
E0912 23:27:31.616439 1805825 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0912 23:27:31.616449 1805825 pod_ready.go:39] duration metric: took 5m24.135588484s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0912 23:27:31.616463 1805825 api_server.go:52] waiting for apiserver process to appear ...
I0912 23:27:31.616491 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:31.616558 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:31.656235 1805825 cri.go:89] found id: "e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:31.656258 1805825 cri.go:89] found id: "5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:31.656263 1805825 cri.go:89] found id: ""
I0912 23:27:31.656270 1805825 logs.go:276] 2 containers: [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a]
I0912 23:27:31.656357 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.659874 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.663300 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:31.663382 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:31.701173 1805825 cri.go:89] found id: "acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:31.701208 1805825 cri.go:89] found id: "f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:31.701213 1805825 cri.go:89] found id: ""
I0912 23:27:31.701224 1805825 logs.go:276] 2 containers: [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab]
I0912 23:27:31.701387 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.704920 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.712336 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:31.712435 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:31.749195 1805825 cri.go:89] found id: "761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:31.749217 1805825 cri.go:89] found id: "c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:31.749222 1805825 cri.go:89] found id: ""
I0912 23:27:31.749229 1805825 logs.go:276] 2 containers: [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28]
I0912 23:27:31.749310 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.753118 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.756544 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:31.756618 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:31.795469 1805825 cri.go:89] found id: "ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:31.795545 1805825 cri.go:89] found id: "e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:31.795564 1805825 cri.go:89] found id: ""
I0912 23:27:31.795588 1805825 logs.go:276] 2 containers: [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9]
I0912 23:27:31.795669 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.799094 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.802491 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:31.802601 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:31.839752 1805825 cri.go:89] found id: "94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:31.839776 1805825 cri.go:89] found id: "f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:31.839781 1805825 cri.go:89] found id: ""
I0912 23:27:31.839789 1805825 logs.go:276] 2 containers: [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71]
I0912 23:27:31.839873 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.843319 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.846600 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:31.846683 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:31.887977 1805825 cri.go:89] found id: "e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:31.888039 1805825 cri.go:89] found id: "ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:31.888060 1805825 cri.go:89] found id: ""
I0912 23:27:31.888089 1805825 logs.go:276] 2 containers: [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99]
I0912 23:27:31.888195 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.891769 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.895304 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:31.895389 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:31.933166 1805825 cri.go:89] found id: "f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:31.933244 1805825 cri.go:89] found id: "2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:31.933257 1805825 cri.go:89] found id: ""
I0912 23:27:31.933266 1805825 logs.go:276] 2 containers: [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade]
I0912 23:27:31.933335 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.936945 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.940365 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:31.940485 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:31.981027 1805825 cri.go:89] found id: "3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:31.981107 1805825 cri.go:89] found id: ""
I0912 23:27:31.981131 1805825 logs.go:276] 1 containers: [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8]
I0912 23:27:31.981229 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.985423 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:31.985539 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:32.033051 1805825 cri.go:89] found id: "1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:32.033072 1805825 cri.go:89] found id: "91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:32.033077 1805825 cri.go:89] found id: ""
I0912 23:27:32.033084 1805825 logs.go:276] 2 containers: [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299]
I0912 23:27:32.033166 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:32.036867 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:32.040391 1805825 logs.go:123] Gathering logs for kube-proxy [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168] ...
I0912 23:27:32.040424 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:32.086447 1805825 logs.go:123] Gathering logs for container status ...
I0912 23:27:32.086476 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:32.136905 1805825 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:32.137006 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:32.190946 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411026 664 reflector.go:138] object-"kube-system"/"coredns-token-m2js8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2js8" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191190 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411278 664 reflector.go:138] object-"kube-system"/"kindnet-token-xzbvw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xzbvw" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191408 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411446 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411664 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-k7dwz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-k7dwz" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191851 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411960 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192082 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414071 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-f7cln": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-f7cln" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192298 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414335 664 reflector.go:138] object-"default"/"default-token-bxtgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-bxtgn" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192523 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414403 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fdc76": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fdc76" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.199031 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.557235 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.200921 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.840628 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.205210 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.466661 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.206993 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.472436 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.208452 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.366884 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.210201 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.463324 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.212088 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.478847 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.213664 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.482852 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.214501 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.566317 664 pod_workers.go:191] Error syncing pod 7a260c8b-3e99-476d-bb2a-f42a54017c50 ("busybox_default(7a260c8b-3e99-476d-bb2a-f42a54017c50)"), skipping: failed to "StartContainer" for "busybox" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.217039 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.591024 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.217368 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:12 old-k8s-version-011723 kubelet[664]: E0912 23:22:12.488949 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.222297 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:34 old-k8s-version-011723 kubelet[664]: E0912 23:22:34.530645 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.223276 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:36 old-k8s-version-011723 kubelet[664]: E0912 23:22:36.632305 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.223614 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:37 old-k8s-version-011723 kubelet[664]: E0912 23:22:37.632313 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.223957 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:44 old-k8s-version-011723 kubelet[664]: E0912 23:22:44.683335 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.224486 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:47 old-k8s-version-011723 kubelet[664]: E0912 23:22:47.274232 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.225089 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:58 old-k8s-version-011723 kubelet[664]: E0912 23:22:58.684667 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.227589 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:00 old-k8s-version-011723 kubelet[664]: E0912 23:23:00.296836 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.227925 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:04 old-k8s-version-011723 kubelet[664]: E0912 23:23:04.683877 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.228115 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:12 old-k8s-version-011723 kubelet[664]: E0912 23:23:12.274275 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.228446 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:16 old-k8s-version-011723 kubelet[664]: E0912 23:23:16.273507 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.228639 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:23 old-k8s-version-011723 kubelet[664]: E0912 23:23:23.274327 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.229234 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:29 old-k8s-version-011723 kubelet[664]: E0912 23:23:29.784506 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.229569 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:34 old-k8s-version-011723 kubelet[664]: E0912 23:23:34.683510 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.229761 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:38 old-k8s-version-011723 kubelet[664]: E0912 23:23:38.273957 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.230108 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:50 old-k8s-version-011723 kubelet[664]: E0912 23:23:50.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.232628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:52 old-k8s-version-011723 kubelet[664]: E0912 23:23:52.283800 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.232968 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:01 old-k8s-version-011723 kubelet[664]: E0912 23:24:01.274256 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.233157 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:04 old-k8s-version-011723 kubelet[664]: E0912 23:24:04.274131 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.233766 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:14 old-k8s-version-011723 kubelet[664]: E0912 23:24:14.908668 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.233955 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:15 old-k8s-version-011723 kubelet[664]: E0912 23:24:15.282101 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.234302 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:24 old-k8s-version-011723 kubelet[664]: E0912 23:24:24.683372 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.234497 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:27 old-k8s-version-011723 kubelet[664]: E0912 23:24:27.273932 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.234835 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:40 old-k8s-version-011723 kubelet[664]: E0912 23:24:40.274451 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.235026 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:42 old-k8s-version-011723 kubelet[664]: E0912 23:24:42.274046 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.235222 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:53 old-k8s-version-011723 kubelet[664]: E0912 23:24:53.273989 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.235556 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:54 old-k8s-version-011723 kubelet[664]: E0912 23:24:54.273546 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.235753 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:04 old-k8s-version-011723 kubelet[664]: E0912 23:25:04.273901 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.236095 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:05 old-k8s-version-011723 kubelet[664]: E0912 23:25:05.274360 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.238700 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:15 old-k8s-version-011723 kubelet[664]: E0912 23:25:15.305120 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.239058 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:19 old-k8s-version-011723 kubelet[664]: E0912 23:25:19.274155 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.239305 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:28 old-k8s-version-011723 kubelet[664]: E0912 23:25:28.273983 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.239641 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:30 old-k8s-version-011723 kubelet[664]: E0912 23:25:30.273912 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.239839 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:39 old-k8s-version-011723 kubelet[664]: E0912 23:25:39.277598 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.240445 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:43 old-k8s-version-011723 kubelet[664]: E0912 23:25:43.182848 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.240777 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:44 old-k8s-version-011723 kubelet[664]: E0912 23:25:44.683303 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.240968 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:51 old-k8s-version-011723 kubelet[664]: E0912 23:25:51.278776 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.241306 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:58 old-k8s-version-011723 kubelet[664]: E0912 23:25:58.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.241496 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:03 old-k8s-version-011723 kubelet[664]: E0912 23:26:03.278105 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.241833 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:10 old-k8s-version-011723 kubelet[664]: E0912 23:26:10.273581 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.242022 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:14 old-k8s-version-011723 kubelet[664]: E0912 23:26:14.273888 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.242356 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: E0912 23:26:22.273505 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.242547 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:27 old-k8s-version-011723 kubelet[664]: E0912 23:26:27.278136 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.242883 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: E0912 23:26:36.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.243073 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:38 old-k8s-version-011723 kubelet[664]: E0912 23:26:38.273832 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.243267 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:49 old-k8s-version-011723 kubelet[664]: E0912 23:26:49.273962 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.243600 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: E0912 23:26:50.273517 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.243797 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.244128 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.244318 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.244672 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.244861 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:32.244873 1805825 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:32.244888 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:32.263536 1805825 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:32.263566 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:32.418067 1805825 logs.go:123] Gathering logs for kube-apiserver [5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a] ...
I0912 23:27:32.418100 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:32.472888 1805825 logs.go:123] Gathering logs for etcd [f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab] ...
I0912 23:27:32.472923 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:32.521854 1805825 logs.go:123] Gathering logs for kube-scheduler [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb] ...
I0912 23:27:32.521887 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:32.574517 1805825 logs.go:123] Gathering logs for kube-apiserver [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b] ...
I0912 23:27:32.574547 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:32.634129 1805825 logs.go:123] Gathering logs for coredns [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623] ...
I0912 23:27:32.634165 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:32.680668 1805825 logs.go:123] Gathering logs for kindnet [2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade] ...
I0912 23:27:32.680696 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:32.718806 1805825 logs.go:123] Gathering logs for storage-provisioner [91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299] ...
I0912 23:27:32.718833 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:32.764018 1805825 logs.go:123] Gathering logs for etcd [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa] ...
I0912 23:27:32.764051 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:32.815016 1805825 logs.go:123] Gathering logs for coredns [c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28] ...
I0912 23:27:32.815050 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:32.860482 1805825 logs.go:123] Gathering logs for kube-scheduler [e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9] ...
I0912 23:27:32.860511 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:32.903923 1805825 logs.go:123] Gathering logs for kube-proxy [f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71] ...
I0912 23:27:32.903952 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:32.947569 1805825 logs.go:123] Gathering logs for kube-controller-manager [ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99] ...
I0912 23:27:32.947601 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:33.028599 1805825 logs.go:123] Gathering logs for kindnet [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64] ...
I0912 23:27:33.028643 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:33.070980 1805825 logs.go:123] Gathering logs for kube-controller-manager [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340] ...
I0912 23:27:33.071011 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:33.133140 1805825 logs.go:123] Gathering logs for kubernetes-dashboard [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8] ...
I0912 23:27:33.133176 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:33.185246 1805825 logs.go:123] Gathering logs for storage-provisioner [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9] ...
I0912 23:27:33.185279 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:33.225304 1805825 logs.go:123] Gathering logs for containerd ...
I0912 23:27:33.225338 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:33.298930 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:33.298966 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:33.299025 1805825 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0912 23:27:33.299037 1805825 out.go:270] Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:33.299044 1805825 out.go:270] Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:33.299070 1805825 out.go:270] Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:33.299077 1805825 out.go:270] Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:33.299086 1805825 out.go:270] Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:33.299091 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:33.299097 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:43.301335 1805825 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0912 23:27:43.313968 1805825 api_server.go:72] duration metric: took 5m55.797654969s to wait for apiserver process to appear ...
I0912 23:27:43.313998 1805825 api_server.go:88] waiting for apiserver healthz status ...
I0912 23:27:43.314038 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:43.314095 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:43.353269 1805825 cri.go:89] found id: "e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:43.353292 1805825 cri.go:89] found id: "5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:43.353297 1805825 cri.go:89] found id: ""
I0912 23:27:43.353305 1805825 logs.go:276] 2 containers: [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a]
I0912 23:27:43.353363 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.357194 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.360691 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:43.360764 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:43.409115 1805825 cri.go:89] found id: "acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:43.409135 1805825 cri.go:89] found id: "f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:43.409140 1805825 cri.go:89] found id: ""
I0912 23:27:43.409148 1805825 logs.go:276] 2 containers: [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab]
I0912 23:27:43.409205 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.414385 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.418038 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:43.418103 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:43.470267 1805825 cri.go:89] found id: "761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:43.470287 1805825 cri.go:89] found id: "c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:43.470292 1805825 cri.go:89] found id: ""
I0912 23:27:43.470299 1805825 logs.go:276] 2 containers: [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28]
I0912 23:27:43.470361 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.474528 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.478036 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:43.478114 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:43.520737 1805825 cri.go:89] found id: "ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:43.520766 1805825 cri.go:89] found id: "e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:43.520773 1805825 cri.go:89] found id: ""
I0912 23:27:43.520780 1805825 logs.go:276] 2 containers: [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9]
I0912 23:27:43.520840 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.524564 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.528472 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:43.528544 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:43.575684 1805825 cri.go:89] found id: "94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:43.575760 1805825 cri.go:89] found id: "f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:43.575767 1805825 cri.go:89] found id: ""
I0912 23:27:43.575775 1805825 logs.go:276] 2 containers: [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71]
I0912 23:27:43.575853 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.579573 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.582968 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:43.583040 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:43.622197 1805825 cri.go:89] found id: "e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:43.622232 1805825 cri.go:89] found id: "ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:43.622242 1805825 cri.go:89] found id: ""
I0912 23:27:43.622267 1805825 logs.go:276] 2 containers: [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99]
I0912 23:27:43.622346 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.625917 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.629579 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:43.629657 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:43.672137 1805825 cri.go:89] found id: "f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:43.672160 1805825 cri.go:89] found id: "2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:43.672166 1805825 cri.go:89] found id: ""
I0912 23:27:43.672174 1805825 logs.go:276] 2 containers: [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade]
I0912 23:27:43.672232 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.675927 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.680672 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:43.680740 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:43.718009 1805825 cri.go:89] found id: "3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:43.718034 1805825 cri.go:89] found id: ""
I0912 23:27:43.718042 1805825 logs.go:276] 1 containers: [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8]
I0912 23:27:43.718099 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.721672 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:43.721750 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:43.759833 1805825 cri.go:89] found id: "1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:43.759855 1805825 cri.go:89] found id: "91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:43.759860 1805825 cri.go:89] found id: ""
I0912 23:27:43.759867 1805825 logs.go:276] 2 containers: [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299]
I0912 23:27:43.759939 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.763575 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.767207 1805825 logs.go:123] Gathering logs for kubernetes-dashboard [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8] ...
I0912 23:27:43.767286 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:43.812189 1805825 logs.go:123] Gathering logs for etcd [f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab] ...
I0912 23:27:43.812262 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:43.860473 1805825 logs.go:123] Gathering logs for coredns [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623] ...
I0912 23:27:43.860501 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:43.901572 1805825 logs.go:123] Gathering logs for kube-proxy [f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71] ...
I0912 23:27:43.901599 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:43.941592 1805825 logs.go:123] Gathering logs for kube-controller-manager [ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99] ...
I0912 23:27:43.941623 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:44.000715 1805825 logs.go:123] Gathering logs for kindnet [2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade] ...
I0912 23:27:44.000754 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:44.049375 1805825 logs.go:123] Gathering logs for storage-provisioner [91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299] ...
I0912 23:27:44.049402 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:44.095108 1805825 logs.go:123] Gathering logs for containerd ...
I0912 23:27:44.095139 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:44.155926 1805825 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:44.155961 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:44.207364 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411026 664 reflector.go:138] object-"kube-system"/"coredns-token-m2js8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2js8" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.207620 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411278 664 reflector.go:138] object-"kube-system"/"kindnet-token-xzbvw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xzbvw" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.207833 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411446 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208051 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411664 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-k7dwz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-k7dwz" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208258 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411960 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208485 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414071 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-f7cln": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-f7cln" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208697 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414335 664 reflector.go:138] object-"default"/"default-token-bxtgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-bxtgn" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208921 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414403 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fdc76": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fdc76" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.215368 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.557235 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.217198 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.840628 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.221475 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.466661 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.223249 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.472436 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.224648 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.366884 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.226290 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.463324 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.228181 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.478847 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.229717 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.482852 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.230538 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.566317 664 pod_workers.go:191] Error syncing pod 7a260c8b-3e99-476d-bb2a-f42a54017c50 ("busybox_default(7a260c8b-3e99-476d-bb2a-f42a54017c50)"), skipping: failed to "StartContainer" for "busybox" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.233034 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.591024 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.233356 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:12 old-k8s-version-011723 kubelet[664]: E0912 23:22:12.488949 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.238182 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:34 old-k8s-version-011723 kubelet[664]: E0912 23:22:34.530645 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.239114 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:36 old-k8s-version-011723 kubelet[664]: E0912 23:22:36.632305 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.239454 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:37 old-k8s-version-011723 kubelet[664]: E0912 23:22:37.632313 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.239792 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:44 old-k8s-version-011723 kubelet[664]: E0912 23:22:44.683335 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.240321 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:47 old-k8s-version-011723 kubelet[664]: E0912 23:22:47.274232 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.240914 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:58 old-k8s-version-011723 kubelet[664]: E0912 23:22:58.684667 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.243409 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:00 old-k8s-version-011723 kubelet[664]: E0912 23:23:00.296836 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.243745 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:04 old-k8s-version-011723 kubelet[664]: E0912 23:23:04.683877 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.243932 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:12 old-k8s-version-011723 kubelet[664]: E0912 23:23:12.274275 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.244259 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:16 old-k8s-version-011723 kubelet[664]: E0912 23:23:16.273507 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.244448 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:23 old-k8s-version-011723 kubelet[664]: E0912 23:23:23.274327 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.245036 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:29 old-k8s-version-011723 kubelet[664]: E0912 23:23:29.784506 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.245365 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:34 old-k8s-version-011723 kubelet[664]: E0912 23:23:34.683510 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.245553 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:38 old-k8s-version-011723 kubelet[664]: E0912 23:23:38.273957 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.245882 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:50 old-k8s-version-011723 kubelet[664]: E0912 23:23:50.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.248381 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:52 old-k8s-version-011723 kubelet[664]: E0912 23:23:52.283800 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.248855 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:01 old-k8s-version-011723 kubelet[664]: E0912 23:24:01.274256 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.250523 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:04 old-k8s-version-011723 kubelet[664]: E0912 23:24:04.274131 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.251141 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:14 old-k8s-version-011723 kubelet[664]: E0912 23:24:14.908668 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.251337 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:15 old-k8s-version-011723 kubelet[664]: E0912 23:24:15.282101 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.251674 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:24 old-k8s-version-011723 kubelet[664]: E0912 23:24:24.683372 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.251898 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:27 old-k8s-version-011723 kubelet[664]: E0912 23:24:27.273932 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252232 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:40 old-k8s-version-011723 kubelet[664]: E0912 23:24:40.274451 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.252419 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:42 old-k8s-version-011723 kubelet[664]: E0912 23:24:42.274046 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252606 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:53 old-k8s-version-011723 kubelet[664]: E0912 23:24:53.273989 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252934 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:54 old-k8s-version-011723 kubelet[664]: E0912 23:24:54.273546 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.253120 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:04 old-k8s-version-011723 kubelet[664]: E0912 23:25:04.273901 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.253473 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:05 old-k8s-version-011723 kubelet[664]: E0912 23:25:05.274360 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.255953 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:15 old-k8s-version-011723 kubelet[664]: E0912 23:25:15.305120 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.256289 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:19 old-k8s-version-011723 kubelet[664]: E0912 23:25:19.274155 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.256481 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:28 old-k8s-version-011723 kubelet[664]: E0912 23:25:28.273983 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.256812 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:30 old-k8s-version-011723 kubelet[664]: E0912 23:25:30.273912 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.256999 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:39 old-k8s-version-011723 kubelet[664]: E0912 23:25:39.277598 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.257586 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:43 old-k8s-version-011723 kubelet[664]: E0912 23:25:43.182848 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.257913 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:44 old-k8s-version-011723 kubelet[664]: E0912 23:25:44.683303 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.258101 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:51 old-k8s-version-011723 kubelet[664]: E0912 23:25:51.278776 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.258443 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:58 old-k8s-version-011723 kubelet[664]: E0912 23:25:58.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.258628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:03 old-k8s-version-011723 kubelet[664]: E0912 23:26:03.278105 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.258985 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:10 old-k8s-version-011723 kubelet[664]: E0912 23:26:10.273581 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.259180 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:14 old-k8s-version-011723 kubelet[664]: E0912 23:26:14.273888 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.259511 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: E0912 23:26:22.273505 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.259708 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:27 old-k8s-version-011723 kubelet[664]: E0912 23:26:27.278136 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260056 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: E0912 23:26:36.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.260247 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:38 old-k8s-version-011723 kubelet[664]: E0912 23:26:38.273832 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260434 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:49 old-k8s-version-011723 kubelet[664]: E0912 23:26:49.273962 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260762 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: E0912 23:26:50.273517 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.260949 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.261277 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.261466 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.261795 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.261982 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.262312 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.262501 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:44.262513 1805825 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:44.262527 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:44.406836 1805825 logs.go:123] Gathering logs for kube-apiserver [5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a] ...
I0912 23:27:44.406930 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:44.479657 1805825 logs.go:123] Gathering logs for kube-scheduler [e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9] ...
I0912 23:27:44.479689 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:44.528355 1805825 logs.go:123] Gathering logs for storage-provisioner [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9] ...
I0912 23:27:44.528385 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:44.590340 1805825 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:44.590373 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:44.607129 1805825 logs.go:123] Gathering logs for kube-apiserver [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b] ...
I0912 23:27:44.607185 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:44.669500 1805825 logs.go:123] Gathering logs for kube-scheduler [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb] ...
I0912 23:27:44.669536 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:44.708883 1805825 logs.go:123] Gathering logs for container status ...
I0912 23:27:44.708912 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:44.757271 1805825 logs.go:123] Gathering logs for etcd [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa] ...
I0912 23:27:44.757307 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:44.800471 1805825 logs.go:123] Gathering logs for coredns [c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28] ...
I0912 23:27:44.800505 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:44.845088 1805825 logs.go:123] Gathering logs for kube-proxy [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168] ...
I0912 23:27:44.845117 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:44.883749 1805825 logs.go:123] Gathering logs for kube-controller-manager [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340] ...
I0912 23:27:44.883783 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:44.946173 1805825 logs.go:123] Gathering logs for kindnet [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64] ...
I0912 23:27:44.946213 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:44.987811 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:44.987836 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:44.987883 1805825 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0912 23:27:44.987898 1805825 out.go:270] Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.987905 1805825 out.go:270] Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.987918 1805825 out.go:270] Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.987925 1805825 out.go:270] Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.987931 1805825 out.go:270] Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:44.987940 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:44.987946 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:54.989447 1805825 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0912 23:27:55.001137 1805825 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0912 23:27:55.007655 1805825 out.go:201]
W0912 23:27:55.012275 1805825 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0912 23:27:55.012327 1805825 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0912 23:27:55.012348 1805825 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0912 23:27:55.012355 1805825 out.go:270] *
*
W0912 23:27:55.013626 1805825 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0912 23:27:55.016564 1805825 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-011723 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-011723
helpers_test.go:235: (dbg) docker inspect old-k8s-version-011723:
-- stdout --
[
{
"Id": "680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055",
"Created": "2024-09-12T23:18:38.237870961Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1806034,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-09-12T23:21:39.525605412Z",
"FinishedAt": "2024-09-12T23:21:38.230835917Z"
},
"Image": "sha256:5a18b2e89815d9320db97822722b50bf88d564940d3d81fe93adf39e9c88570e",
"ResolvConfPath": "/var/lib/docker/containers/680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055/hostname",
"HostsPath": "/var/lib/docker/containers/680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055/hosts",
"LogPath": "/var/lib/docker/containers/680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055/680e0605cc8ab0428c03c4b37b6eb130969bf61696c45168bbe1c95be7f83055-json.log",
"Name": "/old-k8s-version-011723",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-011723:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-011723",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/f819df6ae33bb3f28c65f8e735bcfe2ea0fced7f0d81898f3c1b046a6ae30d3e-init/diff:/var/lib/docker/overlay2/22619844066f8062a761e6c26d439ab232db1d4015e623ac6dd91ab5ce435ce2/diff",
"MergedDir": "/var/lib/docker/overlay2/f819df6ae33bb3f28c65f8e735bcfe2ea0fced7f0d81898f3c1b046a6ae30d3e/merged",
"UpperDir": "/var/lib/docker/overlay2/f819df6ae33bb3f28c65f8e735bcfe2ea0fced7f0d81898f3c1b046a6ae30d3e/diff",
"WorkDir": "/var/lib/docker/overlay2/f819df6ae33bb3f28c65f8e735bcfe2ea0fced7f0d81898f3c1b046a6ae30d3e/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-011723",
"Source": "/var/lib/docker/volumes/old-k8s-version-011723/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-011723",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-011723",
"name.minikube.sigs.k8s.io": "old-k8s-version-011723",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "f79fdaac47ab11409b6fcd4a815a1b2b4d460ebd57c33ca2543e4e64254bdf47",
"SandboxKey": "/var/run/docker/netns/f79fdaac47ab",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34934"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34935"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34938"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34936"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34937"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-011723": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "644d5791a3ff4f65673bd300f36f9d4a0426907ab23cc1635f66562e4668773a",
"EndpointID": "6d05e5ea27fa37b0d2a626f16e5ea289d23de985f33c3db5ae7c26cb8fdd9b53",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-011723",
"680e0605cc8a"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-011723 -n old-k8s-version-011723
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-011723 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-011723 logs -n 25: (3.201252001s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| start | -p force-systemd-env-098328 | force-systemd-env-098328 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:17 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p pause-693262 | pause-693262 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:17 UTC |
| | --alsologtostderr -v=5 | | | | | |
| delete | -p pause-693262 | pause-693262 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:17 UTC |
| start | -p cert-expiration-905537 | cert-expiration-905537 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:18 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-098328 | force-systemd-env-098328 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:17 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-098328 | force-systemd-env-098328 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:17 UTC |
| start | -p cert-options-713058 | cert-options-713058 | jenkins | v1.34.0 | 12 Sep 24 23:17 UTC | 12 Sep 24 23:18 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-713058 ssh | cert-options-713058 | jenkins | v1.34.0 | 12 Sep 24 23:18 UTC | 12 Sep 24 23:18 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-713058 -- sudo | cert-options-713058 | jenkins | v1.34.0 | 12 Sep 24 23:18 UTC | 12 Sep 24 23:18 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-713058 | cert-options-713058 | jenkins | v1.34.0 | 12 Sep 24 23:18 UTC | 12 Sep 24 23:18 UTC |
| start | -p old-k8s-version-011723 | old-k8s-version-011723 | jenkins | v1.34.0 | 12 Sep 24 23:18 UTC | 12 Sep 24 23:21 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-905537 | cert-expiration-905537 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:21 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-905537 | cert-expiration-905537 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:21 UTC |
| start | -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:22 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-011723 | old-k8s-version-011723 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:21 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-011723 | old-k8s-version-011723 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:21 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-011723 | old-k8s-version-011723 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | 12 Sep 24 23:21 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-011723 | old-k8s-version-011723 | jenkins | v1.34.0 | 12 Sep 24 23:21 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:22 UTC | 12 Sep 24 23:22 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:22 UTC | 12 Sep 24 23:22 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:22 UTC | 12 Sep 24 23:22 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:22 UTC | 12 Sep 24 23:27 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| image | no-preload-693555 image list | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:27 UTC | 12 Sep 24 23:27 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:27 UTC | 12 Sep 24 23:27 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-693555 | no-preload-693555 | jenkins | v1.34.0 | 12 Sep 24 23:27 UTC | 12 Sep 24 23:27 UTC |
| | --alsologtostderr -v=1 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/12 23:22:51
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.22.5 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0912 23:22:51.453809 1810706 out.go:345] Setting OutFile to fd 1 ...
I0912 23:22:51.453987 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:22:51.453998 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:22:51.454003 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:22:51.454248 1810706 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19616-1592376/.minikube/bin
I0912 23:22:51.454620 1810706 out.go:352] Setting JSON to false
I0912 23:22:51.455796 1810706 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":29099,"bootTime":1726154273,"procs":241,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1068-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0912 23:22:51.455873 1810706 start.go:139] virtualization:
I0912 23:22:51.459603 1810706 out.go:177] * [no-preload-693555] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0912 23:22:51.461716 1810706 out.go:177] - MINIKUBE_LOCATION=19616
I0912 23:22:51.461747 1810706 notify.go:220] Checking for updates...
I0912 23:22:51.467608 1810706 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0912 23:22:51.469595 1810706 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:22:51.471425 1810706 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19616-1592376/.minikube
I0912 23:22:51.473107 1810706 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0912 23:22:51.474862 1810706 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0912 23:22:51.477193 1810706 config.go:182] Loaded profile config "no-preload-693555": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0912 23:22:51.477759 1810706 driver.go:394] Setting default libvirt URI to qemu:///system
I0912 23:22:51.505284 1810706 docker.go:123] docker version: linux-27.2.1:Docker Engine - Community
I0912 23:22:51.505407 1810706 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0912 23:22:51.598837 1810706 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:53 SystemTime:2024-09-12 23:22:51.587213167 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214827008 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.2.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0912 23:22:51.598959 1810706 docker.go:318] overlay module found
I0912 23:22:51.601611 1810706 out.go:177] * Using the docker driver based on existing profile
I0912 23:22:51.603628 1810706 start.go:297] selected driver: docker
I0912 23:22:51.603656 1810706 start.go:901] validating driver "docker" against &{Name:no-preload-693555 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-693555 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false Mount
String:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:22:51.603880 1810706 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0912 23:22:51.604517 1810706 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0912 23:22:51.662385 1810706 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:53 SystemTime:2024-09-12 23:22:51.652413391 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214827008 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.2.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0912 23:22:51.662729 1810706 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0912 23:22:51.662756 1810706 cni.go:84] Creating CNI manager for ""
I0912 23:22:51.662764 1810706 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0912 23:22:51.662801 1810706 start.go:340] cluster config:
{Name:no-preload-693555 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-693555 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker Moun
tIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:22:51.665814 1810706 out.go:177] * Starting "no-preload-693555" primary control-plane node in "no-preload-693555" cluster
I0912 23:22:51.667793 1810706 cache.go:121] Beginning downloading kic base image for docker with containerd
I0912 23:22:51.669728 1810706 out.go:177] * Pulling base image v0.0.45-1726156396-19616 ...
I0912 23:22:51.671763 1810706 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0912 23:22:51.671812 1810706 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local docker daemon
I0912 23:22:51.671917 1810706 profile.go:143] Saving config to /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/config.json ...
I0912 23:22:51.672232 1810706 cache.go:107] acquiring lock: {Name:mk014defa35ee2c3d9682c815f21ab018748b9d0 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672312 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0912 23:22:51.672321 1810706 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 94.268µs
I0912 23:22:51.672334 1810706 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0912 23:22:51.672344 1810706 cache.go:107] acquiring lock: {Name:mk89d9931b1fbbcee4d1ed196f40307b98f8f572 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672378 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 exists
I0912 23:22:51.672383 1810706 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.31.1" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1" took 40.263µs
I0912 23:22:51.672389 1810706 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.31.1 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 succeeded
I0912 23:22:51.672398 1810706 cache.go:107] acquiring lock: {Name:mk16427431b2fe3b246ddc0906a6351076417536 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672427 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0912 23:22:51.672405 1810706 cache.go:107] acquiring lock: {Name:mkc9a5392f7ddedc8280eb1905f178a47141263e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672449 1810706 cache.go:107] acquiring lock: {Name:mk8d71d4f345a7f47d69dd1cea17d5781c5306d5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672482 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 exists
I0912 23:22:51.672484 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 exists
I0912 23:22:51.672487 1810706 cache.go:96] cache image "registry.k8s.io/etcd:3.5.15-0" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0" took 43.922µs
I0912 23:22:51.672494 1810706 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.31.1" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1" took 95.77µs
I0912 23:22:51.672498 1810706 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.15-0 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 succeeded
I0912 23:22:51.672501 1810706 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.31.1 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 succeeded
I0912 23:22:51.672508 1810706 cache.go:107] acquiring lock: {Name:mk922d2ef87e8ea43b12aa0d5c4195ded21fe103 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672513 1810706 cache.go:107] acquiring lock: {Name:mk470ced10fde97f54c66b6c0002c37cf9fae4ae Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672535 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 exists
I0912 23:22:51.672541 1810706 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.3" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3" took 34.322µs
I0912 23:22:51.672547 1810706 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.3 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 succeeded
I0912 23:22:51.672551 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 exists
I0912 23:22:51.672557 1810706 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.31.1" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1" took 46.063µs
I0912 23:22:51.672558 1810706 cache.go:107] acquiring lock: {Name:mk3b46bca000c5dd18145f0f882a526f7be17737 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.672585 1810706 cache.go:115] /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 exists
I0912 23:22:51.672590 1810706 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.31.1" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1" took 33.534µs
I0912 23:22:51.672596 1810706 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.31.1 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 succeeded
I0912 23:22:51.672564 1810706 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.31.1 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 succeeded
I0912 23:22:51.672432 1810706 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 34.707µs
I0912 23:22:51.672605 1810706 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/19616-1592376/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0912 23:22:51.672609 1810706 cache.go:87] Successfully saved all images to host disk.
W0912 23:22:51.690851 1810706 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 is of wrong architecture
I0912 23:22:51.690874 1810706 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 to local cache
I0912 23:22:51.690948 1810706 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local cache directory
I0912 23:22:51.690971 1810706 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 in local cache directory, skipping pull
I0912 23:22:51.690979 1810706 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 exists in cache, skipping pull
I0912 23:22:51.690988 1810706 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 as a tarball
I0912 23:22:51.690993 1810706 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 from local cache
I0912 23:22:51.811856 1810706 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 from cached tarball
I0912 23:22:51.811896 1810706 cache.go:194] Successfully downloaded all kic artifacts
I0912 23:22:51.811926 1810706 start.go:360] acquireMachinesLock for no-preload-693555: {Name:mk83a5748e0533acea5d0ef167bbf0418e788f96 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0912 23:22:51.812004 1810706 start.go:364] duration metric: took 55.344µs to acquireMachinesLock for "no-preload-693555"
I0912 23:22:51.812032 1810706 start.go:96] Skipping create...Using existing machine configuration
I0912 23:22:51.812040 1810706 fix.go:54] fixHost starting:
I0912 23:22:51.812317 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:51.828930 1810706 fix.go:112] recreateIfNeeded on no-preload-693555: state=Stopped err=<nil>
W0912 23:22:51.828963 1810706 fix.go:138] unexpected machine state, will restart: <nil>
I0912 23:22:51.831151 1810706 out.go:177] * Restarting existing docker container for "no-preload-693555" ...
I0912 23:22:49.577063 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:51.581169 1805825 pod_ready.go:103] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:52.578988 1805825 pod_ready.go:93] pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace has status "Ready":"True"
I0912 23:22:52.579010 1805825 pod_ready.go:82] duration metric: took 45.007601623s for pod "coredns-74ff55c5b-lzb66" in "kube-system" namespace to be "Ready" ...
I0912 23:22:52.579022 1805825 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:22:51.833068 1810706 cli_runner.go:164] Run: docker start no-preload-693555
I0912 23:22:52.195858 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:52.220809 1810706 kic.go:430] container "no-preload-693555" state is running.
I0912 23:22:52.221229 1810706 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-693555
I0912 23:22:52.244893 1810706 profile.go:143] Saving config to /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/config.json ...
I0912 23:22:52.245112 1810706 machine.go:93] provisionDockerMachine start ...
I0912 23:22:52.245726 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:52.278113 1810706 main.go:141] libmachine: Using SSH client type: native
I0912 23:22:52.278397 1810706 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34939 <nil> <nil>}
I0912 23:22:52.278408 1810706 main.go:141] libmachine: About to run SSH command:
hostname
I0912 23:22:52.279174 1810706 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0912 23:22:55.427579 1810706 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-693555
I0912 23:22:55.427606 1810706 ubuntu.go:169] provisioning hostname "no-preload-693555"
I0912 23:22:55.427692 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:55.453650 1810706 main.go:141] libmachine: Using SSH client type: native
I0912 23:22:55.453903 1810706 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34939 <nil> <nil>}
I0912 23:22:55.453914 1810706 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-693555 && echo "no-preload-693555" | sudo tee /etc/hostname
I0912 23:22:55.608206 1810706 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-693555
I0912 23:22:55.608362 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:55.625717 1810706 main.go:141] libmachine: Using SSH client type: native
I0912 23:22:55.625960 1810706 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3ebfd0] 0x3ee830 <nil> [] 0s} 127.0.0.1 34939 <nil> <nil>}
I0912 23:22:55.625977 1810706 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-693555' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-693555/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-693555' | sudo tee -a /etc/hosts;
fi
fi
I0912 23:22:55.767807 1810706 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0912 23:22:55.767877 1810706 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19616-1592376/.minikube CaCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19616-1592376/.minikube}
I0912 23:22:55.767939 1810706 ubuntu.go:177] setting up certificates
I0912 23:22:55.767969 1810706 provision.go:84] configureAuth start
I0912 23:22:55.768040 1810706 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-693555
I0912 23:22:55.785381 1810706 provision.go:143] copyHostCerts
I0912 23:22:55.785454 1810706 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem, removing ...
I0912 23:22:55.785469 1810706 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem
I0912 23:22:55.785552 1810706 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/cert.pem (1123 bytes)
I0912 23:22:55.785651 1810706 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem, removing ...
I0912 23:22:55.785660 1810706 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem
I0912 23:22:55.785689 1810706 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/key.pem (1675 bytes)
I0912 23:22:55.785748 1810706 exec_runner.go:144] found /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem, removing ...
I0912 23:22:55.785758 1810706 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem
I0912 23:22:55.785782 1810706 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.pem (1082 bytes)
I0912 23:22:55.785834 1810706 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem org=jenkins.no-preload-693555 san=[127.0.0.1 192.168.85.2 localhost minikube no-preload-693555]
I0912 23:22:56.181548 1810706 provision.go:177] copyRemoteCerts
I0912 23:22:56.181650 1810706 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0912 23:22:56.181705 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:56.199409 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:56.300704 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0912 23:22:56.327776 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0912 23:22:56.353776 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0912 23:22:56.384513 1810706 provision.go:87] duration metric: took 616.522804ms to configureAuth
I0912 23:22:56.384545 1810706 ubuntu.go:193] setting minikube options for container-runtime
I0912 23:22:56.384746 1810706 config.go:182] Loaded profile config "no-preload-693555": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0912 23:22:56.384758 1810706 machine.go:96] duration metric: took 4.139638853s to provisionDockerMachine
I0912 23:22:56.384775 1810706 start.go:293] postStartSetup for "no-preload-693555" (driver="docker")
I0912 23:22:56.384791 1810706 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0912 23:22:56.384853 1810706 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0912 23:22:56.384896 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:56.400964 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:56.504638 1810706 ssh_runner.go:195] Run: cat /etc/os-release
I0912 23:22:56.507853 1810706 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0912 23:22:56.507890 1810706 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0912 23:22:56.507901 1810706 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0912 23:22:56.507923 1810706 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0912 23:22:56.507939 1810706 filesync.go:126] Scanning /home/jenkins/minikube-integration/19616-1592376/.minikube/addons for local assets ...
I0912 23:22:56.508008 1810706 filesync.go:126] Scanning /home/jenkins/minikube-integration/19616-1592376/.minikube/files for local assets ...
I0912 23:22:56.508105 1810706 filesync.go:149] local asset: /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem -> 15977602.pem in /etc/ssl/certs
I0912 23:22:56.508212 1810706 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0912 23:22:56.516938 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem --> /etc/ssl/certs/15977602.pem (1708 bytes)
I0912 23:22:56.542989 1810706 start.go:296] duration metric: took 158.190727ms for postStartSetup
I0912 23:22:56.543082 1810706 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0912 23:22:56.543125 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:56.559653 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:56.661331 1810706 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0912 23:22:56.667125 1810706 fix.go:56] duration metric: took 4.855077246s for fixHost
I0912 23:22:56.667154 1810706 start.go:83] releasing machines lock for "no-preload-693555", held for 4.855134985s
I0912 23:22:56.667225 1810706 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-693555
I0912 23:22:56.686867 1810706 ssh_runner.go:195] Run: cat /version.json
I0912 23:22:56.686911 1810706 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0912 23:22:56.686921 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:56.686983 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:56.707380 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:56.717307 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:56.803050 1810706 ssh_runner.go:195] Run: systemctl --version
I0912 23:22:56.954158 1810706 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0912 23:22:56.958596 1810706 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0912 23:22:56.977516 1810706 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0912 23:22:56.977599 1810706 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0912 23:22:56.987753 1810706 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0912 23:22:56.987776 1810706 start.go:495] detecting cgroup driver to use...
I0912 23:22:56.987810 1810706 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0912 23:22:56.987864 1810706 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0912 23:22:57.002542 1810706 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0912 23:22:57.017803 1810706 docker.go:217] disabling cri-docker service (if available) ...
I0912 23:22:57.017926 1810706 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0912 23:22:57.032638 1810706 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0912 23:22:57.046220 1810706 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0912 23:22:57.140476 1810706 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0912 23:22:57.233077 1810706 docker.go:233] disabling docker service ...
I0912 23:22:57.233149 1810706 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0912 23:22:57.246352 1810706 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0912 23:22:57.257996 1810706 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0912 23:22:57.338244 1810706 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0912 23:22:57.425250 1810706 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0912 23:22:57.436535 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0912 23:22:57.453477 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0912 23:22:57.464991 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0912 23:22:57.476481 1810706 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0912 23:22:57.476570 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0912 23:22:57.487308 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0912 23:22:57.497948 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0912 23:22:57.509828 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0912 23:22:57.521662 1810706 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0912 23:22:57.534918 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0912 23:22:57.547237 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0912 23:22:57.559317 1810706 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0912 23:22:57.571782 1810706 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0912 23:22:57.585116 1810706 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0912 23:22:57.596006 1810706 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:22:57.703561 1810706 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0912 23:22:57.869424 1810706 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0912 23:22:57.869606 1810706 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0912 23:22:57.873492 1810706 start.go:563] Will wait 60s for crictl version
I0912 23:22:57.873595 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:22:57.876893 1810706 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0912 23:22:57.918166 1810706 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0912 23:22:57.918304 1810706 ssh_runner.go:195] Run: containerd --version
I0912 23:22:57.943529 1810706 ssh_runner.go:195] Run: containerd --version
I0912 23:22:57.975237 1810706 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I0912 23:22:54.585950 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:57.086582 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:22:57.976892 1810706 cli_runner.go:164] Run: docker network inspect no-preload-693555 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0912 23:22:57.993326 1810706 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0912 23:22:57.996997 1810706 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0912 23:22:58.012986 1810706 kubeadm.go:883] updating cluster {Name:no-preload-693555 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-693555 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenk
ins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0912 23:22:58.013122 1810706 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0912 23:22:58.013187 1810706 ssh_runner.go:195] Run: sudo crictl images --output json
I0912 23:22:58.060738 1810706 containerd.go:627] all images are preloaded for containerd runtime.
I0912 23:22:58.060765 1810706 cache_images.go:84] Images are preloaded, skipping loading
I0912 23:22:58.060774 1810706 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.31.1 containerd true true} ...
I0912 23:22:58.060885 1810706 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=no-preload-693555 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:no-preload-693555 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0912 23:22:58.060958 1810706 ssh_runner.go:195] Run: sudo crictl info
I0912 23:22:58.109622 1810706 cni.go:84] Creating CNI manager for ""
I0912 23:22:58.109648 1810706 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0912 23:22:58.109658 1810706 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0912 23:22:58.109684 1810706 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-693555 NodeName:no-preload-693555 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP
ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0912 23:22:58.109833 1810706 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "no-preload-693555"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0912 23:22:58.109910 1810706 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0912 23:22:58.121017 1810706 binaries.go:44] Found k8s binaries, skipping transfer
I0912 23:22:58.121094 1810706 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0912 23:22:58.130450 1810706 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (321 bytes)
I0912 23:22:58.149974 1810706 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0912 23:22:58.170029 1810706 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2171 bytes)
I0912 23:22:58.188937 1810706 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0912 23:22:58.192685 1810706 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0912 23:22:58.203296 1810706 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:22:58.302170 1810706 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0912 23:22:58.323682 1810706 certs.go:68] Setting up /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555 for IP: 192.168.85.2
I0912 23:22:58.323840 1810706 certs.go:194] generating shared ca certs ...
I0912 23:22:58.323860 1810706 certs.go:226] acquiring lock for ca certs: {Name:mk5b7cca91a053f0ec1ca9c487c600f7eefaa6e5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:22:58.324032 1810706 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.key
I0912 23:22:58.324074 1810706 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.key
I0912 23:22:58.324081 1810706 certs.go:256] generating profile certs ...
I0912 23:22:58.324171 1810706 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/client.key
I0912 23:22:58.324239 1810706 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/apiserver.key.ce120de9
I0912 23:22:58.324276 1810706 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/proxy-client.key
I0912 23:22:58.324388 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760.pem (1338 bytes)
W0912 23:22:58.324420 1810706 certs.go:480] ignoring /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760_empty.pem, impossibly tiny 0 bytes
I0912 23:22:58.324429 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca-key.pem (1679 bytes)
I0912 23:22:58.324455 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/ca.pem (1082 bytes)
I0912 23:22:58.324477 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/cert.pem (1123 bytes)
I0912 23:22:58.324501 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/key.pem (1675 bytes)
I0912 23:22:58.324543 1810706 certs.go:484] found cert: /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem (1708 bytes)
I0912 23:22:58.325163 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0912 23:22:58.388708 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0912 23:22:58.437687 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0912 23:22:58.492352 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0912 23:22:58.534444 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0912 23:22:58.564689 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0912 23:22:58.605553 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0912 23:22:58.640648 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/profiles/no-preload-693555/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0912 23:22:58.669168 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0912 23:22:58.706342 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/certs/1597760.pem --> /usr/share/ca-certificates/1597760.pem (1338 bytes)
I0912 23:22:58.734618 1810706 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19616-1592376/.minikube/files/etc/ssl/certs/15977602.pem --> /usr/share/ca-certificates/15977602.pem (1708 bytes)
I0912 23:22:58.768729 1810706 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0912 23:22:58.790433 1810706 ssh_runner.go:195] Run: openssl version
I0912 23:22:58.797896 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0912 23:22:58.809289 1810706 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0912 23:22:58.812970 1810706 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 12 22:30 /usr/share/ca-certificates/minikubeCA.pem
I0912 23:22:58.813082 1810706 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0912 23:22:58.820305 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0912 23:22:58.830162 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1597760.pem && ln -fs /usr/share/ca-certificates/1597760.pem /etc/ssl/certs/1597760.pem"
I0912 23:22:58.840024 1810706 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1597760.pem
I0912 23:22:58.844058 1810706 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 12 22:41 /usr/share/ca-certificates/1597760.pem
I0912 23:22:58.844158 1810706 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1597760.pem
I0912 23:22:58.851506 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1597760.pem /etc/ssl/certs/51391683.0"
I0912 23:22:58.861100 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/15977602.pem && ln -fs /usr/share/ca-certificates/15977602.pem /etc/ssl/certs/15977602.pem"
I0912 23:22:58.871304 1810706 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/15977602.pem
I0912 23:22:58.874928 1810706 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 12 22:41 /usr/share/ca-certificates/15977602.pem
I0912 23:22:58.875044 1810706 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/15977602.pem
I0912 23:22:58.881919 1810706 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/15977602.pem /etc/ssl/certs/3ec20f2e.0"
I0912 23:22:58.891627 1810706 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0912 23:22:58.895790 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0912 23:22:58.902805 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0912 23:22:58.909937 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0912 23:22:58.917237 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0912 23:22:58.925840 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0912 23:22:58.932971 1810706 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0912 23:22:58.939952 1810706 kubeadm.go:392] StartCluster: {Name:no-preload-693555 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726156396-19616@sha256:66b06a42534e914a5c8ad765d7508a93a34031939ec9a6b3a818ef0a444ff889 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-693555 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0912 23:22:58.940053 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0912 23:22:58.940151 1810706 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0912 23:22:58.994689 1810706 cri.go:89] found id: "3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:22:58.994713 1810706 cri.go:89] found id: "fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:22:58.994718 1810706 cri.go:89] found id: "f9d27646d9b2033dac6adc77f4beabfa2310b96018de7536884911301bcc8d10"
I0912 23:22:58.994731 1810706 cri.go:89] found id: "28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:22:58.994735 1810706 cri.go:89] found id: "ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:22:58.994778 1810706 cri.go:89] found id: "05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:22:58.994784 1810706 cri.go:89] found id: "9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:22:58.994792 1810706 cri.go:89] found id: "f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:22:58.994796 1810706 cri.go:89] found id: ""
I0912 23:22:58.994869 1810706 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0912 23:22:59.010379 1810706 cri.go:116] JSON = null
W0912 23:22:59.010456 1810706 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0912 23:22:59.010572 1810706 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0912 23:22:59.022750 1810706 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0912 23:22:59.022773 1810706 kubeadm.go:593] restartPrimaryControlPlane start ...
I0912 23:22:59.022860 1810706 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0912 23:22:59.032653 1810706 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0912 23:22:59.033274 1810706 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-693555" does not appear in /home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:22:59.033550 1810706 kubeconfig.go:62] /home/jenkins/minikube-integration/19616-1592376/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-693555" cluster setting kubeconfig missing "no-preload-693555" context setting]
I0912 23:22:59.034051 1810706 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19616-1592376/kubeconfig: {Name:mk20814b10c438de6fa8214738e210df331cf1df Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:22:59.035445 1810706 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0912 23:22:59.045786 1810706 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0912 23:22:59.045818 1810706 kubeadm.go:597] duration metric: took 23.038977ms to restartPrimaryControlPlane
I0912 23:22:59.045828 1810706 kubeadm.go:394] duration metric: took 105.887251ms to StartCluster
I0912 23:22:59.045843 1810706 settings.go:142] acquiring lock: {Name:mk1fdbbc4ffc0e3fc6419399beeda4839e1c5a1d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:22:59.045907 1810706 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19616-1592376/kubeconfig
I0912 23:22:59.046822 1810706 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19616-1592376/kubeconfig: {Name:mk20814b10c438de6fa8214738e210df331cf1df Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0912 23:22:59.047023 1810706 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0912 23:22:59.047323 1810706 config.go:182] Loaded profile config "no-preload-693555": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0912 23:22:59.047366 1810706 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0912 23:22:59.047429 1810706 addons.go:69] Setting storage-provisioner=true in profile "no-preload-693555"
I0912 23:22:59.047451 1810706 addons.go:234] Setting addon storage-provisioner=true in "no-preload-693555"
W0912 23:22:59.047459 1810706 addons.go:243] addon storage-provisioner should already be in state true
I0912 23:22:59.047496 1810706 host.go:66] Checking if "no-preload-693555" exists ...
I0912 23:22:59.047971 1810706 addons.go:69] Setting default-storageclass=true in profile "no-preload-693555"
I0912 23:22:59.048003 1810706 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-693555"
I0912 23:22:59.048242 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:59.048468 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:59.048645 1810706 addons.go:69] Setting dashboard=true in profile "no-preload-693555"
I0912 23:22:59.048684 1810706 addons.go:234] Setting addon dashboard=true in "no-preload-693555"
W0912 23:22:59.048692 1810706 addons.go:243] addon dashboard should already be in state true
I0912 23:22:59.048717 1810706 host.go:66] Checking if "no-preload-693555" exists ...
I0912 23:22:59.049148 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:59.049472 1810706 addons.go:69] Setting metrics-server=true in profile "no-preload-693555"
I0912 23:22:59.049512 1810706 addons.go:234] Setting addon metrics-server=true in "no-preload-693555"
W0912 23:22:59.049523 1810706 addons.go:243] addon metrics-server should already be in state true
I0912 23:22:59.049546 1810706 host.go:66] Checking if "no-preload-693555" exists ...
I0912 23:22:59.049943 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:59.051651 1810706 out.go:177] * Verifying Kubernetes components...
I0912 23:22:59.064899 1810706 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0912 23:22:59.096432 1810706 addons.go:234] Setting addon default-storageclass=true in "no-preload-693555"
W0912 23:22:59.096455 1810706 addons.go:243] addon default-storageclass should already be in state true
I0912 23:22:59.096481 1810706 host.go:66] Checking if "no-preload-693555" exists ...
I0912 23:22:59.096892 1810706 cli_runner.go:164] Run: docker container inspect no-preload-693555 --format={{.State.Status}}
I0912 23:22:59.124148 1810706 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0912 23:22:59.127377 1810706 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0912 23:22:59.127515 1810706 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:22:59.127535 1810706 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0912 23:22:59.127612 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:59.131059 1810706 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0912 23:22:59.133340 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0912 23:22:59.133362 1810706 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0912 23:22:59.133437 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:59.138774 1810706 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0912 23:22:59.140436 1810706 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0912 23:22:59.140458 1810706 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0912 23:22:59.140521 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:59.172016 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:59.189044 1810706 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0912 23:22:59.189066 1810706 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0912 23:22:59.189131 1810706 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-693555
I0912 23:22:59.218563 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:59.218563 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:59.232303 1810706 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34939 SSHKeyPath:/home/jenkins/minikube-integration/19616-1592376/.minikube/machines/no-preload-693555/id_rsa Username:docker}
I0912 23:22:59.289187 1810706 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0912 23:22:59.362366 1810706 node_ready.go:35] waiting up to 6m0s for node "no-preload-693555" to be "Ready" ...
I0912 23:22:59.448900 1810706 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:22:59.515863 1810706 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0912 23:22:59.545756 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0912 23:22:59.545819 1810706 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0912 23:22:59.566590 1810706 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0912 23:22:59.566653 1810706 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0912 23:22:59.661905 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0912 23:22:59.661972 1810706 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0912 23:22:59.713327 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0912 23:22:59.713392 1810706 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0912 23:22:59.719510 1810706 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0912 23:22:59.719586 1810706 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
W0912 23:22:59.816412 1810706 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0912 23:22:59.816509 1810706 retry.go:31] will retry after 192.006509ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0912 23:22:59.865116 1810706 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:22:59.865190 1810706 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0912 23:22:59.877483 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0912 23:22:59.877546 1810706 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0912 23:22:59.988504 1810706 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0912 23:23:00.009000 1810706 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0912 23:23:00.181415 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0912 23:23:00.181526 1810706 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0912 23:23:00.392965 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0912 23:23:00.393059 1810706 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0912 23:23:00.549069 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0912 23:23:00.549142 1810706 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0912 23:23:00.620149 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0912 23:23:00.620215 1810706 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0912 23:23:00.651416 1810706 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0912 23:23:00.651492 1810706 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0912 23:23:00.672406 1810706 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0912 23:22:59.091309 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:01.584645 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:03.584803 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:05.042480 1810706 node_ready.go:49] node "no-preload-693555" has status "Ready":"True"
I0912 23:23:05.042507 1810706 node_ready.go:38] duration metric: took 5.680102929s for node "no-preload-693555" to be "Ready" ...
I0912 23:23:05.042518 1810706 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0912 23:23:05.129210 1810706 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-zhsgq" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.149598 1810706 pod_ready.go:93] pod "coredns-7c65d6cfc9-zhsgq" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.149680 1810706 pod_ready.go:82] duration metric: took 20.371481ms for pod "coredns-7c65d6cfc9-zhsgq" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.149713 1810706 pod_ready.go:79] waiting up to 6m0s for pod "etcd-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.236142 1810706 pod_ready.go:93] pod "etcd-no-preload-693555" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.236216 1810706 pod_ready.go:82] duration metric: took 86.464149ms for pod "etcd-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.236246 1810706 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.254516 1810706 pod_ready.go:93] pod "kube-apiserver-no-preload-693555" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.254580 1810706 pod_ready.go:82] duration metric: took 18.312812ms for pod "kube-apiserver-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.254607 1810706 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.267222 1810706 pod_ready.go:93] pod "kube-controller-manager-no-preload-693555" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.267295 1810706 pod_ready.go:82] duration metric: took 12.664871ms for pod "kube-controller-manager-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.267322 1810706 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-h54sf" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.285301 1810706 pod_ready.go:93] pod "kube-proxy-h54sf" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.285369 1810706 pod_ready.go:82] duration metric: took 18.024666ms for pod "kube-proxy-h54sf" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.285395 1810706 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.355270 1810706 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (5.8393172s)
I0912 23:23:05.673652 1810706 pod_ready.go:93] pod "kube-scheduler-no-preload-693555" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:05.673725 1810706 pod_ready.go:82] duration metric: took 388.309548ms for pod "kube-scheduler-no-preload-693555" in "kube-system" namespace to be "Ready" ...
I0912 23:23:05.673753 1810706 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace to be "Ready" ...
I0912 23:23:07.680189 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:08.473015 1810706 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.4639097s)
I0912 23:23:08.473486 1810706 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (8.484891977s)
I0912 23:23:08.473512 1810706 addons.go:475] Verifying addon metrics-server=true in "no-preload-693555"
I0912 23:23:08.700405 1810706 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (8.027881949s)
I0912 23:23:08.702476 1810706 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-693555 addons enable metrics-server
I0912 23:23:08.704720 1810706 out.go:177] * Enabled addons: default-storageclass, storage-provisioner, metrics-server, dashboard
I0912 23:23:05.586373 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:07.703232 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:08.706616 1810706 addons.go:510] duration metric: took 9.659243972s for enable addons: enabled=[default-storageclass storage-provisioner metrics-server dashboard]
I0912 23:23:10.181098 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:10.092109 1805825 pod_ready.go:103] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:12.586070 1805825 pod_ready.go:93] pod "etcd-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:12.586091 1805825 pod_ready.go:82] duration metric: took 20.007061143s for pod "etcd-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.586105 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.592877 1805825 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:12.592899 1805825 pod_ready.go:82] duration metric: took 6.785661ms for pod "kube-apiserver-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.592910 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:12.181154 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:14.680773 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:14.599464 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:16.600010 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:18.600119 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:16.682604 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:19.182046 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:21.099631 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:23.598727 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:21.679775 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:24.180626 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:25.599494 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:28.099402 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:26.680250 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:28.680704 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:31.179841 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:30.108642 1805825 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:31.599271 1805825 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.599300 1805825 pod_ready.go:82] duration metric: took 19.006382161s for pod "kube-controller-manager-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.599314 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-cd4m4" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.604530 1805825 pod_ready.go:93] pod "kube-proxy-cd4m4" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.604558 1805825 pod_ready.go:82] duration metric: took 5.236086ms for pod "kube-proxy-cd4m4" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.604570 1805825 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.609948 1805825 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace has status "Ready":"True"
I0912 23:23:31.609978 1805825 pod_ready.go:82] duration metric: took 5.399884ms for pod "kube-scheduler-old-k8s-version-011723" in "kube-system" namespace to be "Ready" ...
I0912 23:23:31.609991 1805825 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace to be "Ready" ...
I0912 23:23:33.616901 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:33.179905 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:35.680343 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:36.117406 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:38.616767 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:37.680376 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:39.680631 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:41.116618 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:43.117109 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:42.190548 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:44.680359 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:45.119017 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:47.615873 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:46.680774 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:49.180421 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:49.616264 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:51.616446 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:51.680668 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:54.179787 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:54.117042 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:56.617982 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:58.618639 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:56.680208 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:23:59.179818 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:01.180358 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:01.116490 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:03.615956 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:03.180509 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:05.680239 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:05.616291 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:07.616383 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:08.181629 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:10.679963 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:10.118971 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:12.615907 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:13.179823 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:15.180839 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:14.616327 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:16.616660 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:18.618315 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:17.680094 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:20.180382 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:21.116721 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:23.120405 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:22.180445 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:24.679939 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:25.615987 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:27.616285 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:26.682501 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:29.180183 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:31.180914 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:29.616736 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:32.116368 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:33.681606 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:36.179960 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:34.615597 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:36.615875 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:38.682405 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:41.180329 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:39.116734 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:41.616349 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:43.616829 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:43.680256 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:46.179519 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:46.116401 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:48.116475 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:48.180762 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:50.682900 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:50.116632 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:52.616460 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:53.179834 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:55.180251 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:55.116563 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:57.116939 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:57.181131 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:59.680834 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:24:59.615938 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:01.616962 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:01.680955 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:04.179816 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:06.180030 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:04.117033 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:06.615106 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:08.697132 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:08.682424 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:11.180620 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:11.117036 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:13.615834 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:13.680229 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:16.180488 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:15.616644 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:18.118163 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:18.680183 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:20.680361 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:20.616069 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:23.116443 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:22.680550 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:24.680781 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:25.118089 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:27.618412 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:27.179666 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:29.179790 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:31.179979 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:30.118003 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:32.615831 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:33.180600 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:35.679984 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:34.616187 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:36.616671 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:38.179228 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:40.180625 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:39.116087 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:41.615833 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:43.615886 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:42.186160 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:44.679386 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:45.616059 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:48.116800 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:46.680020 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:48.681153 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:51.179809 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:50.616480 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:53.115634 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:53.679299 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:55.685772 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:55.117262 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:57.615811 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:58.180525 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:00.197848 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:25:59.617089 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:01.617125 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:02.679455 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:04.679633 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:04.116539 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:06.615639 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:08.616084 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:06.680215 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:09.179840 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:11.180247 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:10.616278 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:13.116566 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:13.180819 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:15.181478 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:15.118162 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:17.616104 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:17.184108 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:19.679471 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:19.616529 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:21.617541 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:21.681674 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:24.180390 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:24.116954 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:26.618594 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:26.680710 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:28.681929 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:31.184219 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:29.135672 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:31.616184 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:33.680097 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:36.180273 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:34.117097 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:36.616730 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:38.680425 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:41.179918 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:39.115929 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:41.116884 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:43.116963 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:43.681011 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:46.179639 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:45.118983 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:47.617481 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:48.179957 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:50.181948 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:49.623855 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:52.116732 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:52.680196 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:54.680946 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:54.125647 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:56.615873 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:58.625134 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:57.179843 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:26:59.183687 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:01.117368 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:03.615687 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:01.680977 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:03.682557 1810706 pod_ready.go:103] pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:05.680117 1810706 pod_ready.go:82] duration metric: took 4m0.006337168s for pod "metrics-server-6867b74b74-x8xjw" in "kube-system" namespace to be "Ready" ...
E0912 23:27:05.680146 1810706 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0912 23:27:05.680156 1810706 pod_ready.go:39] duration metric: took 4m0.637627117s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0912 23:27:05.680172 1810706 api_server.go:52] waiting for apiserver process to appear ...
I0912 23:27:05.680202 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:05.680269 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:05.726303 1810706 cri.go:89] found id: "477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:05.726326 1810706 cri.go:89] found id: "ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:05.726331 1810706 cri.go:89] found id: ""
I0912 23:27:05.726339 1810706 logs.go:276] 2 containers: [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54]
I0912 23:27:05.726401 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.730131 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.733979 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:05.734054 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:05.775503 1810706 cri.go:89] found id: "b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:05.775532 1810706 cri.go:89] found id: "f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:05.775538 1810706 cri.go:89] found id: ""
I0912 23:27:05.775546 1810706 logs.go:276] 2 containers: [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a]
I0912 23:27:05.775605 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.779430 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.782776 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:05.782853 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:05.840609 1810706 cri.go:89] found id: "48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:05.840632 1810706 cri.go:89] found id: "3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:05.840637 1810706 cri.go:89] found id: ""
I0912 23:27:05.840644 1810706 logs.go:276] 2 containers: [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570]
I0912 23:27:05.840715 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.844577 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.848252 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:05.848324 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:05.887710 1810706 cri.go:89] found id: "071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:05.887731 1810706 cri.go:89] found id: "9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:05.887738 1810706 cri.go:89] found id: ""
I0912 23:27:05.887745 1810706 logs.go:276] 2 containers: [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee]
I0912 23:27:05.887803 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.891539 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.895373 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:05.895450 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:05.932511 1810706 cri.go:89] found id: "5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:05.932535 1810706 cri.go:89] found id: "28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:05.932540 1810706 cri.go:89] found id: ""
I0912 23:27:05.932548 1810706 logs.go:276] 2 containers: [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40]
I0912 23:27:05.932612 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.936823 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.940579 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:05.940663 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:05.981506 1810706 cri.go:89] found id: "a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:05.981539 1810706 cri.go:89] found id: "05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:05.981545 1810706 cri.go:89] found id: ""
I0912 23:27:05.981571 1810706 logs.go:276] 2 containers: [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205]
I0912 23:27:05.981649 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.985360 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:05.988907 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:05.988986 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:06.032771 1810706 cri.go:89] found id: "ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:06.032841 1810706 cri.go:89] found id: "fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:06.032853 1810706 cri.go:89] found id: ""
I0912 23:27:06.032861 1810706 logs.go:276] 2 containers: [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00]
I0912 23:27:06.032929 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:06.037328 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:06.041115 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:06.041198 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:06.086630 1810706 cri.go:89] found id: "78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:06.086652 1810706 cri.go:89] found id: ""
I0912 23:27:06.086661 1810706 logs.go:276] 1 containers: [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91]
I0912 23:27:06.086729 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:06.090826 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:06.090950 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:06.139503 1810706 cri.go:89] found id: "4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:06.139577 1810706 cri.go:89] found id: "3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:06.139596 1810706 cri.go:89] found id: ""
I0912 23:27:06.139622 1810706 logs.go:276] 2 containers: [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a]
I0912 23:27:06.139783 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:06.143781 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:06.147594 1810706 logs.go:123] Gathering logs for kube-apiserver [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9] ...
I0912 23:27:06.147674 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:06.228468 1810706 logs.go:123] Gathering logs for kube-apiserver [ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54] ...
I0912 23:27:06.228504 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:06.296104 1810706 logs.go:123] Gathering logs for coredns [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11] ...
I0912 23:27:06.296138 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:06.349591 1810706 logs.go:123] Gathering logs for kube-scheduler [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc] ...
I0912 23:27:06.349619 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:06.390184 1810706 logs.go:123] Gathering logs for kube-proxy [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c] ...
I0912 23:27:06.390213 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:06.441081 1810706 logs.go:123] Gathering logs for kube-proxy [28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40] ...
I0912 23:27:06.441148 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:05.616841 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:07.617150 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:06.478957 1810706 logs.go:123] Gathering logs for kindnet [fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00] ...
I0912 23:27:06.478986 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:06.517107 1810706 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:06.517132 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:06.693138 1810706 logs.go:123] Gathering logs for storage-provisioner [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba] ...
I0912 23:27:06.693172 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:06.734434 1810706 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:06.734463 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:06.754206 1810706 logs.go:123] Gathering logs for kube-controller-manager [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a] ...
I0912 23:27:06.754238 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:06.826622 1810706 logs.go:123] Gathering logs for kindnet [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde] ...
I0912 23:27:06.826654 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:06.873063 1810706 logs.go:123] Gathering logs for containerd ...
I0912 23:27:06.873097 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:06.936656 1810706 logs.go:123] Gathering logs for container status ...
I0912 23:27:06.936694 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:06.983454 1810706 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:06.983483 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:07.041658 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:07.041945 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:07.073178 1810706 logs.go:123] Gathering logs for etcd [f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a] ...
I0912 23:27:07.073218 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:07.147535 1810706 logs.go:123] Gathering logs for kubernetes-dashboard [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91] ...
I0912 23:27:07.147567 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:07.193591 1810706 logs.go:123] Gathering logs for storage-provisioner [3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a] ...
I0912 23:27:07.193617 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:07.237149 1810706 logs.go:123] Gathering logs for etcd [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904] ...
I0912 23:27:07.237187 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:07.297207 1810706 logs.go:123] Gathering logs for kube-scheduler [9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee] ...
I0912 23:27:07.297294 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:07.353274 1810706 logs.go:123] Gathering logs for kube-controller-manager [05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205] ...
I0912 23:27:07.353307 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:07.416648 1810706 logs.go:123] Gathering logs for coredns [3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570] ...
I0912 23:27:07.416742 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:07.481538 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:07.481565 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:07.481643 1810706 out.go:270] X Problems detected in kubelet:
W0912 23:27:07.481656 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:07.481692 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:07.481706 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:07.481713 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:10.117503 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:12.617018 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:15.117143 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:17.120989 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:17.482665 1810706 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0912 23:27:17.494575 1810706 api_server.go:72] duration metric: took 4m18.447514875s to wait for apiserver process to appear ...
I0912 23:27:17.494599 1810706 api_server.go:88] waiting for apiserver healthz status ...
I0912 23:27:17.494633 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:17.494689 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:17.534406 1810706 cri.go:89] found id: "477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:17.534426 1810706 cri.go:89] found id: "ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:17.534431 1810706 cri.go:89] found id: ""
I0912 23:27:17.534438 1810706 logs.go:276] 2 containers: [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54]
I0912 23:27:17.534496 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.538209 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.541568 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:17.541642 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:17.609834 1810706 cri.go:89] found id: "b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:17.609857 1810706 cri.go:89] found id: "f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:17.609863 1810706 cri.go:89] found id: ""
I0912 23:27:17.609870 1810706 logs.go:276] 2 containers: [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a]
I0912 23:27:17.609932 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.619360 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.623359 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:17.623474 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:17.668082 1810706 cri.go:89] found id: "48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:17.668105 1810706 cri.go:89] found id: "3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:17.668110 1810706 cri.go:89] found id: ""
I0912 23:27:17.668126 1810706 logs.go:276] 2 containers: [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570]
I0912 23:27:17.668215 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.671997 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.675487 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:17.675579 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:17.713506 1810706 cri.go:89] found id: "071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:17.713530 1810706 cri.go:89] found id: "9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:17.713535 1810706 cri.go:89] found id: ""
I0912 23:27:17.713543 1810706 logs.go:276] 2 containers: [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee]
I0912 23:27:17.713622 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.717566 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.721146 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:17.721222 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:17.761779 1810706 cri.go:89] found id: "5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:17.761845 1810706 cri.go:89] found id: "28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:17.761873 1810706 cri.go:89] found id: ""
I0912 23:27:17.761885 1810706 logs.go:276] 2 containers: [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40]
I0912 23:27:17.761955 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.765654 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.769275 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:17.769392 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:17.817079 1810706 cri.go:89] found id: "a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:17.817151 1810706 cri.go:89] found id: "05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:17.817170 1810706 cri.go:89] found id: ""
I0912 23:27:17.817195 1810706 logs.go:276] 2 containers: [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205]
I0912 23:27:17.817283 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.823255 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.826746 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:17.826819 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:17.868225 1810706 cri.go:89] found id: "ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:17.868248 1810706 cri.go:89] found id: "fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:17.868253 1810706 cri.go:89] found id: ""
I0912 23:27:17.868259 1810706 logs.go:276] 2 containers: [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00]
I0912 23:27:17.868347 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.872534 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.876080 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:17.876176 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:17.912671 1810706 cri.go:89] found id: "4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:17.912696 1810706 cri.go:89] found id: "3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:17.912701 1810706 cri.go:89] found id: ""
I0912 23:27:17.912709 1810706 logs.go:276] 2 containers: [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a]
I0912 23:27:17.912770 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.916682 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.920305 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:17.920380 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:17.959796 1810706 cri.go:89] found id: "78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:17.959819 1810706 cri.go:89] found id: ""
I0912 23:27:17.959827 1810706 logs.go:276] 1 containers: [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91]
I0912 23:27:17.959898 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:17.963592 1810706 logs.go:123] Gathering logs for etcd [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904] ...
I0912 23:27:17.963617 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:18.008283 1810706 logs.go:123] Gathering logs for kube-controller-manager [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a] ...
I0912 23:27:18.008381 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:18.111207 1810706 logs.go:123] Gathering logs for coredns [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11] ...
I0912 23:27:18.111293 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:18.163000 1810706 logs.go:123] Gathering logs for kube-scheduler [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc] ...
I0912 23:27:18.163027 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:18.205560 1810706 logs.go:123] Gathering logs for kube-proxy [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c] ...
I0912 23:27:18.205590 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:18.244956 1810706 logs.go:123] Gathering logs for kube-controller-manager [05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205] ...
I0912 23:27:18.245032 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:18.303877 1810706 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:18.303909 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:18.321794 1810706 logs.go:123] Gathering logs for kube-apiserver [ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54] ...
I0912 23:27:18.321826 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:18.369856 1810706 logs.go:123] Gathering logs for kube-proxy [28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40] ...
I0912 23:27:18.369890 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:18.441349 1810706 logs.go:123] Gathering logs for storage-provisioner [3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a] ...
I0912 23:27:18.441399 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:18.485739 1810706 logs.go:123] Gathering logs for kube-apiserver [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9] ...
I0912 23:27:18.485767 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:18.551483 1810706 logs.go:123] Gathering logs for kube-scheduler [9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee] ...
I0912 23:27:18.551516 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:18.602046 1810706 logs.go:123] Gathering logs for etcd [f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a] ...
I0912 23:27:18.602082 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:18.669708 1810706 logs.go:123] Gathering logs for coredns [3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570] ...
I0912 23:27:18.669742 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:18.713002 1810706 logs.go:123] Gathering logs for kindnet [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde] ...
I0912 23:27:18.713032 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:18.771074 1810706 logs.go:123] Gathering logs for kindnet [fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00] ...
I0912 23:27:18.771103 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:18.814427 1810706 logs.go:123] Gathering logs for storage-provisioner [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba] ...
I0912 23:27:18.814454 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:18.860354 1810706 logs.go:123] Gathering logs for kubernetes-dashboard [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91] ...
I0912 23:27:18.860384 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:18.903812 1810706 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:18.903842 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:18.947070 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:18.947335 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:18.979330 1810706 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:18.979364 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:19.122952 1810706 logs.go:123] Gathering logs for containerd ...
I0912 23:27:19.122984 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:19.190451 1810706 logs.go:123] Gathering logs for container status ...
I0912 23:27:19.190491 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:19.245314 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:19.245341 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:19.245427 1810706 out.go:270] X Problems detected in kubelet:
W0912 23:27:19.245444 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:19.245563 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:19.245578 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:19.245585 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:19.616374 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:22.117190 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:24.615625 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:26.615774 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:28.619296 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:29.246993 1810706 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0912 23:27:29.255600 1810706 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0912 23:27:29.256623 1810706 api_server.go:141] control plane version: v1.31.1
I0912 23:27:29.256661 1810706 api_server.go:131] duration metric: took 11.76205474s to wait for apiserver health ...
I0912 23:27:29.256670 1810706 system_pods.go:43] waiting for kube-system pods to appear ...
I0912 23:27:29.256695 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:29.256763 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:29.306724 1810706 cri.go:89] found id: "477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:29.306745 1810706 cri.go:89] found id: "ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:29.306751 1810706 cri.go:89] found id: ""
I0912 23:27:29.306759 1810706 logs.go:276] 2 containers: [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54]
I0912 23:27:29.306818 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.310799 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.314570 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:29.314642 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:29.352451 1810706 cri.go:89] found id: "b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:29.352478 1810706 cri.go:89] found id: "f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:29.352484 1810706 cri.go:89] found id: ""
I0912 23:27:29.352498 1810706 logs.go:276] 2 containers: [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a]
I0912 23:27:29.352568 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.356513 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.360125 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:29.360217 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:29.401465 1810706 cri.go:89] found id: "48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:29.401486 1810706 cri.go:89] found id: "3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:29.401492 1810706 cri.go:89] found id: ""
I0912 23:27:29.401499 1810706 logs.go:276] 2 containers: [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570]
I0912 23:27:29.401565 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.405380 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.408713 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:29.408781 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:29.456275 1810706 cri.go:89] found id: "071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:29.456296 1810706 cri.go:89] found id: "9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:29.456301 1810706 cri.go:89] found id: ""
I0912 23:27:29.456309 1810706 logs.go:276] 2 containers: [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee]
I0912 23:27:29.456371 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.460070 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.463656 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:29.463771 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:29.515367 1810706 cri.go:89] found id: "5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:29.515388 1810706 cri.go:89] found id: "28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:29.515393 1810706 cri.go:89] found id: ""
I0912 23:27:29.515408 1810706 logs.go:276] 2 containers: [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40]
I0912 23:27:29.515466 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.519312 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.525754 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:29.525827 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:29.578003 1810706 cri.go:89] found id: "a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:29.578028 1810706 cri.go:89] found id: "05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:29.578033 1810706 cri.go:89] found id: ""
I0912 23:27:29.578040 1810706 logs.go:276] 2 containers: [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205]
I0912 23:27:29.578104 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.582129 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.585660 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:29.585762 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:29.631134 1810706 cri.go:89] found id: "ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:29.631162 1810706 cri.go:89] found id: "fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:29.631168 1810706 cri.go:89] found id: ""
I0912 23:27:29.631175 1810706 logs.go:276] 2 containers: [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00]
I0912 23:27:29.631261 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.634759 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.637962 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:29.638029 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:29.681998 1810706 cri.go:89] found id: "78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:29.682021 1810706 cri.go:89] found id: ""
I0912 23:27:29.682030 1810706 logs.go:276] 1 containers: [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91]
I0912 23:27:29.682092 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.687430 1810706 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:29.687516 1810706 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:29.733473 1810706 cri.go:89] found id: "4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:29.733506 1810706 cri.go:89] found id: "3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:29.733511 1810706 cri.go:89] found id: ""
I0912 23:27:29.733518 1810706 logs.go:276] 2 containers: [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a]
I0912 23:27:29.733595 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.737362 1810706 ssh_runner.go:195] Run: which crictl
I0912 23:27:29.741236 1810706 logs.go:123] Gathering logs for kube-controller-manager [05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205] ...
I0912 23:27:29.741261 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05d21ce7223f303ab629c852afd0c23d3bd83ddc7f5e525b19eafcd518d90205"
I0912 23:27:29.804495 1810706 logs.go:123] Gathering logs for kindnet [ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde] ...
I0912 23:27:29.804531 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad43bfb6f08066e2b2b459f86e327a1828a7098f4276f8a976c8c428da2e0cde"
I0912 23:27:29.851503 1810706 logs.go:123] Gathering logs for storage-provisioner [4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba] ...
I0912 23:27:29.851533 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4be7edac801c048e4da6a8f0ebaa4d578467a01e9db5b68dd58cb1db560298ba"
I0912 23:27:29.891526 1810706 logs.go:123] Gathering logs for coredns [3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570] ...
I0912 23:27:29.891598 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3476d1c345d735c195eff6f0e379a19ce6de55dc9031b47cbb8c3846995b3570"
I0912 23:27:29.938568 1810706 logs.go:123] Gathering logs for kube-scheduler [071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc] ...
I0912 23:27:29.938646 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 071450c5d56030c924328604ca60161b7d7d98433dd7345d90f9572494b82dfc"
I0912 23:27:29.991944 1810706 logs.go:123] Gathering logs for kube-apiserver [477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9] ...
I0912 23:27:29.992025 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 477e1b27d6244de5ae6a64237e7e4cf2909e20d681e3a1995c8958dec44ac0d9"
I0912 23:27:30.078375 1810706 logs.go:123] Gathering logs for coredns [48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11] ...
I0912 23:27:30.078465 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 48782a6da8f4b7c0cd71ed667e0936b18b356230e449a6a4fd8e70476aaafc11"
I0912 23:27:30.149583 1810706 logs.go:123] Gathering logs for kube-proxy [28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40] ...
I0912 23:27:30.149623 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 28433c93e2874843ba88312299226db85cf2925d42c4198ced81f444cc7c5e40"
I0912 23:27:30.212709 1810706 logs.go:123] Gathering logs for kindnet [fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00] ...
I0912 23:27:30.212748 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fdd0e8bfd032b19b7d03a25e135b60184cfb1ef8aef841ccf80addc3c98a4a00"
I0912 23:27:30.255121 1810706 logs.go:123] Gathering logs for kubernetes-dashboard [78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91] ...
I0912 23:27:30.255212 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 78034ebb53280de477b2a646a33dfa53044f1449734af88bb23d4704b42a3b91"
I0912 23:27:30.308015 1810706 logs.go:123] Gathering logs for storage-provisioner [3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a] ...
I0912 23:27:30.308089 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3ff147d7cfc903e15eb338718acc2f0b024bbfa87a8593495c3b7ea4c0fac87a"
I0912 23:27:30.352283 1810706 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:30.352307 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:30.401232 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:30.401513 1810706 logs.go:138] Found kubelet problem: Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:30.434451 1810706 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:30.434488 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:30.566368 1810706 logs.go:123] Gathering logs for container status ...
I0912 23:27:30.566398 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:30.623446 1810706 logs.go:123] Gathering logs for containerd ...
I0912 23:27:30.623476 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:30.691082 1810706 logs.go:123] Gathering logs for etcd [b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904] ...
I0912 23:27:30.691115 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b5c67eeae67f566a3e85ad69f208260f5d872989b10777b786b23b67f5d79904"
I0912 23:27:30.739119 1810706 logs.go:123] Gathering logs for kube-scheduler [9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee] ...
I0912 23:27:30.739148 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9e667a77ee1ed22d157b6759c8ddbf053e25b962646044aafe484dfc1f60b8ee"
I0912 23:27:30.794388 1810706 logs.go:123] Gathering logs for etcd [f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a] ...
I0912 23:27:30.794428 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f8905636a6533439b8b3a5fa972466a779c617744979f0d9408405553d59ad5a"
I0912 23:27:30.844807 1810706 logs.go:123] Gathering logs for kube-proxy [5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c] ...
I0912 23:27:30.844838 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c4a6b9252768278b934e06df32e955004d314e150c2f5983264c8897b73c08c"
I0912 23:27:30.885817 1810706 logs.go:123] Gathering logs for kube-controller-manager [a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a] ...
I0912 23:27:30.885846 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a8e3a456f0b4b10613af9107fb417b040b7b6a9f9dd664e224687dec8702388a"
I0912 23:27:30.968069 1810706 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:30.968102 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:30.985663 1810706 logs.go:123] Gathering logs for kube-apiserver [ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54] ...
I0912 23:27:30.985694 1810706 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ffd8e004c279749df62bdc8177862941341c4f58331b2a372e0f822b6ec26c54"
I0912 23:27:31.051395 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:31.051429 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:31.051495 1810706 out.go:270] X Problems detected in kubelet:
W0912 23:27:31.051512 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: W0912 23:23:08.516515 660 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-693555" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-693555' and this object
W0912 23:27:31.051520 1810706 out.go:270] Sep 12 23:23:08 no-preload-693555 kubelet[660]: E0912 23:23:08.516568 660 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-693555\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-693555' and this object" logger="UnhandledError"
I0912 23:27:31.051532 1810706 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:31.051538 1810706 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:30.620004 1805825 pod_ready.go:103] pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace has status "Ready":"False"
I0912 23:27:31.616410 1805825 pod_ready.go:82] duration metric: took 4m0.006405602s for pod "metrics-server-9975d5f86-gklxg" in "kube-system" namespace to be "Ready" ...
E0912 23:27:31.616439 1805825 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0912 23:27:31.616449 1805825 pod_ready.go:39] duration metric: took 5m24.135588484s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0912 23:27:31.616463 1805825 api_server.go:52] waiting for apiserver process to appear ...
I0912 23:27:31.616491 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:31.616558 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:31.656235 1805825 cri.go:89] found id: "e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:31.656258 1805825 cri.go:89] found id: "5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:31.656263 1805825 cri.go:89] found id: ""
I0912 23:27:31.656270 1805825 logs.go:276] 2 containers: [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a]
I0912 23:27:31.656357 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.659874 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.663300 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:31.663382 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:31.701173 1805825 cri.go:89] found id: "acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:31.701208 1805825 cri.go:89] found id: "f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:31.701213 1805825 cri.go:89] found id: ""
I0912 23:27:31.701224 1805825 logs.go:276] 2 containers: [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab]
I0912 23:27:31.701387 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.704920 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.712336 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:31.712435 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:31.749195 1805825 cri.go:89] found id: "761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:31.749217 1805825 cri.go:89] found id: "c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:31.749222 1805825 cri.go:89] found id: ""
I0912 23:27:31.749229 1805825 logs.go:276] 2 containers: [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28]
I0912 23:27:31.749310 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.753118 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.756544 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:31.756618 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:31.795469 1805825 cri.go:89] found id: "ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:31.795545 1805825 cri.go:89] found id: "e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:31.795564 1805825 cri.go:89] found id: ""
I0912 23:27:31.795588 1805825 logs.go:276] 2 containers: [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9]
I0912 23:27:31.795669 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.799094 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.802491 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:31.802601 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:31.839752 1805825 cri.go:89] found id: "94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:31.839776 1805825 cri.go:89] found id: "f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:31.839781 1805825 cri.go:89] found id: ""
I0912 23:27:31.839789 1805825 logs.go:276] 2 containers: [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71]
I0912 23:27:31.839873 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.843319 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.846600 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:31.846683 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:31.887977 1805825 cri.go:89] found id: "e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:31.888039 1805825 cri.go:89] found id: "ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:31.888060 1805825 cri.go:89] found id: ""
I0912 23:27:31.888089 1805825 logs.go:276] 2 containers: [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99]
I0912 23:27:31.888195 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.891769 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.895304 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:31.895389 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:31.933166 1805825 cri.go:89] found id: "f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:31.933244 1805825 cri.go:89] found id: "2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:31.933257 1805825 cri.go:89] found id: ""
I0912 23:27:31.933266 1805825 logs.go:276] 2 containers: [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade]
I0912 23:27:31.933335 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.936945 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.940365 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:31.940485 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:31.981027 1805825 cri.go:89] found id: "3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:31.981107 1805825 cri.go:89] found id: ""
I0912 23:27:31.981131 1805825 logs.go:276] 1 containers: [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8]
I0912 23:27:31.981229 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:31.985423 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:31.985539 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:32.033051 1805825 cri.go:89] found id: "1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:32.033072 1805825 cri.go:89] found id: "91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:32.033077 1805825 cri.go:89] found id: ""
I0912 23:27:32.033084 1805825 logs.go:276] 2 containers: [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299]
I0912 23:27:32.033166 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:32.036867 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:32.040391 1805825 logs.go:123] Gathering logs for kube-proxy [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168] ...
I0912 23:27:32.040424 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:32.086447 1805825 logs.go:123] Gathering logs for container status ...
I0912 23:27:32.086476 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:32.136905 1805825 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:32.137006 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:32.190946 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411026 664 reflector.go:138] object-"kube-system"/"coredns-token-m2js8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2js8" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191190 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411278 664 reflector.go:138] object-"kube-system"/"kindnet-token-xzbvw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xzbvw" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191408 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411446 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411664 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-k7dwz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-k7dwz" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.191851 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411960 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192082 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414071 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-f7cln": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-f7cln" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192298 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414335 664 reflector.go:138] object-"default"/"default-token-bxtgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-bxtgn" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.192523 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414403 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fdc76": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fdc76" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:32.199031 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.557235 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.200921 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.840628 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.205210 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.466661 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.206993 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.472436 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.208452 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.366884 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.210201 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.463324 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.212088 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.478847 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.213664 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.482852 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.214501 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.566317 664 pod_workers.go:191] Error syncing pod 7a260c8b-3e99-476d-bb2a-f42a54017c50 ("busybox_default(7a260c8b-3e99-476d-bb2a-f42a54017c50)"), skipping: failed to "StartContainer" for "busybox" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:32.217039 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.591024 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.217368 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:12 old-k8s-version-011723 kubelet[664]: E0912 23:22:12.488949 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.222297 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:34 old-k8s-version-011723 kubelet[664]: E0912 23:22:34.530645 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.223276 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:36 old-k8s-version-011723 kubelet[664]: E0912 23:22:36.632305 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.223614 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:37 old-k8s-version-011723 kubelet[664]: E0912 23:22:37.632313 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.223957 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:44 old-k8s-version-011723 kubelet[664]: E0912 23:22:44.683335 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.224486 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:47 old-k8s-version-011723 kubelet[664]: E0912 23:22:47.274232 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.225089 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:58 old-k8s-version-011723 kubelet[664]: E0912 23:22:58.684667 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.227589 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:00 old-k8s-version-011723 kubelet[664]: E0912 23:23:00.296836 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.227925 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:04 old-k8s-version-011723 kubelet[664]: E0912 23:23:04.683877 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.228115 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:12 old-k8s-version-011723 kubelet[664]: E0912 23:23:12.274275 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.228446 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:16 old-k8s-version-011723 kubelet[664]: E0912 23:23:16.273507 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.228639 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:23 old-k8s-version-011723 kubelet[664]: E0912 23:23:23.274327 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.229234 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:29 old-k8s-version-011723 kubelet[664]: E0912 23:23:29.784506 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.229569 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:34 old-k8s-version-011723 kubelet[664]: E0912 23:23:34.683510 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.229761 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:38 old-k8s-version-011723 kubelet[664]: E0912 23:23:38.273957 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.230108 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:50 old-k8s-version-011723 kubelet[664]: E0912 23:23:50.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.232628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:52 old-k8s-version-011723 kubelet[664]: E0912 23:23:52.283800 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.232968 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:01 old-k8s-version-011723 kubelet[664]: E0912 23:24:01.274256 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.233157 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:04 old-k8s-version-011723 kubelet[664]: E0912 23:24:04.274131 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.233766 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:14 old-k8s-version-011723 kubelet[664]: E0912 23:24:14.908668 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.233955 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:15 old-k8s-version-011723 kubelet[664]: E0912 23:24:15.282101 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.234302 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:24 old-k8s-version-011723 kubelet[664]: E0912 23:24:24.683372 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.234497 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:27 old-k8s-version-011723 kubelet[664]: E0912 23:24:27.273932 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.234835 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:40 old-k8s-version-011723 kubelet[664]: E0912 23:24:40.274451 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.235026 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:42 old-k8s-version-011723 kubelet[664]: E0912 23:24:42.274046 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.235222 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:53 old-k8s-version-011723 kubelet[664]: E0912 23:24:53.273989 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.235556 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:54 old-k8s-version-011723 kubelet[664]: E0912 23:24:54.273546 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.235753 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:04 old-k8s-version-011723 kubelet[664]: E0912 23:25:04.273901 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.236095 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:05 old-k8s-version-011723 kubelet[664]: E0912 23:25:05.274360 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.238700 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:15 old-k8s-version-011723 kubelet[664]: E0912 23:25:15.305120 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:32.239058 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:19 old-k8s-version-011723 kubelet[664]: E0912 23:25:19.274155 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.239305 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:28 old-k8s-version-011723 kubelet[664]: E0912 23:25:28.273983 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.239641 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:30 old-k8s-version-011723 kubelet[664]: E0912 23:25:30.273912 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.239839 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:39 old-k8s-version-011723 kubelet[664]: E0912 23:25:39.277598 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.240445 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:43 old-k8s-version-011723 kubelet[664]: E0912 23:25:43.182848 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.240777 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:44 old-k8s-version-011723 kubelet[664]: E0912 23:25:44.683303 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.240968 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:51 old-k8s-version-011723 kubelet[664]: E0912 23:25:51.278776 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.241306 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:58 old-k8s-version-011723 kubelet[664]: E0912 23:25:58.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.241496 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:03 old-k8s-version-011723 kubelet[664]: E0912 23:26:03.278105 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.241833 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:10 old-k8s-version-011723 kubelet[664]: E0912 23:26:10.273581 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.242022 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:14 old-k8s-version-011723 kubelet[664]: E0912 23:26:14.273888 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.242356 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: E0912 23:26:22.273505 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.242547 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:27 old-k8s-version-011723 kubelet[664]: E0912 23:26:27.278136 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.242883 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: E0912 23:26:36.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.243073 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:38 old-k8s-version-011723 kubelet[664]: E0912 23:26:38.273832 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.243267 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:49 old-k8s-version-011723 kubelet[664]: E0912 23:26:49.273962 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.243600 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: E0912 23:26:50.273517 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.243797 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.244128 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.244318 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:32.244672 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:32.244861 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:32.244873 1805825 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:32.244888 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:32.263536 1805825 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:32.263566 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:32.418067 1805825 logs.go:123] Gathering logs for kube-apiserver [5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a] ...
I0912 23:27:32.418100 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:32.472888 1805825 logs.go:123] Gathering logs for etcd [f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab] ...
I0912 23:27:32.472923 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:32.521854 1805825 logs.go:123] Gathering logs for kube-scheduler [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb] ...
I0912 23:27:32.521887 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:32.574517 1805825 logs.go:123] Gathering logs for kube-apiserver [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b] ...
I0912 23:27:32.574547 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:32.634129 1805825 logs.go:123] Gathering logs for coredns [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623] ...
I0912 23:27:32.634165 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:32.680668 1805825 logs.go:123] Gathering logs for kindnet [2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade] ...
I0912 23:27:32.680696 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:32.718806 1805825 logs.go:123] Gathering logs for storage-provisioner [91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299] ...
I0912 23:27:32.718833 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:32.764018 1805825 logs.go:123] Gathering logs for etcd [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa] ...
I0912 23:27:32.764051 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:32.815016 1805825 logs.go:123] Gathering logs for coredns [c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28] ...
I0912 23:27:32.815050 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:32.860482 1805825 logs.go:123] Gathering logs for kube-scheduler [e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9] ...
I0912 23:27:32.860511 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:32.903923 1805825 logs.go:123] Gathering logs for kube-proxy [f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71] ...
I0912 23:27:32.903952 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:32.947569 1805825 logs.go:123] Gathering logs for kube-controller-manager [ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99] ...
I0912 23:27:32.947601 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:33.028599 1805825 logs.go:123] Gathering logs for kindnet [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64] ...
I0912 23:27:33.028643 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:33.070980 1805825 logs.go:123] Gathering logs for kube-controller-manager [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340] ...
I0912 23:27:33.071011 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:33.133140 1805825 logs.go:123] Gathering logs for kubernetes-dashboard [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8] ...
I0912 23:27:33.133176 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:33.185246 1805825 logs.go:123] Gathering logs for storage-provisioner [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9] ...
I0912 23:27:33.185279 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:33.225304 1805825 logs.go:123] Gathering logs for containerd ...
I0912 23:27:33.225338 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:33.298930 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:33.298966 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:33.299025 1805825 out.go:270] X Problems detected in kubelet:
W0912 23:27:33.299037 1805825 out.go:270] Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:33.299044 1805825 out.go:270] Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:33.299070 1805825 out.go:270] Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:33.299077 1805825 out.go:270] Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:33.299086 1805825 out.go:270] Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:33.299091 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:33.299097 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:41.059097 1810706 system_pods.go:59] 9 kube-system pods found
I0912 23:27:41.059200 1810706 system_pods.go:61] "coredns-7c65d6cfc9-zhsgq" [16bb9cf4-891d-43c6-b489-4ad94da02766] Running
I0912 23:27:41.059234 1810706 system_pods.go:61] "etcd-no-preload-693555" [20ab9e86-a8a5-4c71-8aec-f7df68bad75c] Running
I0912 23:27:41.059267 1810706 system_pods.go:61] "kindnet-nlh64" [9a37b725-19a3-4280-a4cc-ec8d214d4d0d] Running
I0912 23:27:41.059293 1810706 system_pods.go:61] "kube-apiserver-no-preload-693555" [c656a837-6f11-4d71-82aa-5229b39df334] Running
I0912 23:27:41.059324 1810706 system_pods.go:61] "kube-controller-manager-no-preload-693555" [db016c36-dc2e-4876-975d-8f69679984a4] Running
I0912 23:27:41.059345 1810706 system_pods.go:61] "kube-proxy-h54sf" [9f64daeb-5f5d-4af9-9e19-3dcf5c01eeee] Running
I0912 23:27:41.059377 1810706 system_pods.go:61] "kube-scheduler-no-preload-693555" [6c0e71df-95f0-40a0-b482-5ade3bfec475] Running
I0912 23:27:41.059409 1810706 system_pods.go:61] "metrics-server-6867b74b74-x8xjw" [5a501205-94ed-41dd-bb33-337bf9ecb022] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0912 23:27:41.059436 1810706 system_pods.go:61] "storage-provisioner" [39689ad0-9762-4536-b68d-03470dc67833] Running
I0912 23:27:41.059457 1810706 system_pods.go:74] duration metric: took 11.802780963s to wait for pod list to return data ...
I0912 23:27:41.059499 1810706 default_sa.go:34] waiting for default service account to be created ...
I0912 23:27:41.062561 1810706 default_sa.go:45] found service account: "default"
I0912 23:27:41.062587 1810706 default_sa.go:55] duration metric: took 3.058705ms for default service account to be created ...
I0912 23:27:41.062599 1810706 system_pods.go:116] waiting for k8s-apps to be running ...
I0912 23:27:41.068921 1810706 system_pods.go:86] 9 kube-system pods found
I0912 23:27:41.069003 1810706 system_pods.go:89] "coredns-7c65d6cfc9-zhsgq" [16bb9cf4-891d-43c6-b489-4ad94da02766] Running
I0912 23:27:41.069027 1810706 system_pods.go:89] "etcd-no-preload-693555" [20ab9e86-a8a5-4c71-8aec-f7df68bad75c] Running
I0912 23:27:41.069050 1810706 system_pods.go:89] "kindnet-nlh64" [9a37b725-19a3-4280-a4cc-ec8d214d4d0d] Running
I0912 23:27:41.069085 1810706 system_pods.go:89] "kube-apiserver-no-preload-693555" [c656a837-6f11-4d71-82aa-5229b39df334] Running
I0912 23:27:41.069113 1810706 system_pods.go:89] "kube-controller-manager-no-preload-693555" [db016c36-dc2e-4876-975d-8f69679984a4] Running
I0912 23:27:41.069136 1810706 system_pods.go:89] "kube-proxy-h54sf" [9f64daeb-5f5d-4af9-9e19-3dcf5c01eeee] Running
I0912 23:27:41.069159 1810706 system_pods.go:89] "kube-scheduler-no-preload-693555" [6c0e71df-95f0-40a0-b482-5ade3bfec475] Running
I0912 23:27:41.069197 1810706 system_pods.go:89] "metrics-server-6867b74b74-x8xjw" [5a501205-94ed-41dd-bb33-337bf9ecb022] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0912 23:27:41.069225 1810706 system_pods.go:89] "storage-provisioner" [39689ad0-9762-4536-b68d-03470dc67833] Running
I0912 23:27:41.069252 1810706 system_pods.go:126] duration metric: took 6.646378ms to wait for k8s-apps to be running ...
I0912 23:27:41.069273 1810706 system_svc.go:44] waiting for kubelet service to be running ....
I0912 23:27:41.069361 1810706 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0912 23:27:41.085075 1810706 system_svc.go:56] duration metric: took 15.792129ms WaitForService to wait for kubelet
I0912 23:27:41.085101 1810706 kubeadm.go:582] duration metric: took 4m42.038044796s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0912 23:27:41.085123 1810706 node_conditions.go:102] verifying NodePressure condition ...
I0912 23:27:41.088919 1810706 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0912 23:27:41.088954 1810706 node_conditions.go:123] node cpu capacity is 2
I0912 23:27:41.088967 1810706 node_conditions.go:105] duration metric: took 3.838018ms to run NodePressure ...
I0912 23:27:41.088980 1810706 start.go:241] waiting for startup goroutines ...
I0912 23:27:41.088988 1810706 start.go:246] waiting for cluster config update ...
I0912 23:27:41.088999 1810706 start.go:255] writing updated cluster config ...
I0912 23:27:41.089318 1810706 ssh_runner.go:195] Run: rm -f paused
I0912 23:27:41.153623 1810706 start.go:600] kubectl: 1.31.0, cluster: 1.31.1 (minor skew: 0)
I0912 23:27:41.156052 1810706 out.go:177] * Done! kubectl is now configured to use "no-preload-693555" cluster and "default" namespace by default
I0912 23:27:43.301335 1805825 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0912 23:27:43.313968 1805825 api_server.go:72] duration metric: took 5m55.797654969s to wait for apiserver process to appear ...
I0912 23:27:43.313998 1805825 api_server.go:88] waiting for apiserver healthz status ...
I0912 23:27:43.314038 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0912 23:27:43.314095 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0912 23:27:43.353269 1805825 cri.go:89] found id: "e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:43.353292 1805825 cri.go:89] found id: "5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:43.353297 1805825 cri.go:89] found id: ""
I0912 23:27:43.353305 1805825 logs.go:276] 2 containers: [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a]
I0912 23:27:43.353363 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.357194 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.360691 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0912 23:27:43.360764 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0912 23:27:43.409115 1805825 cri.go:89] found id: "acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:43.409135 1805825 cri.go:89] found id: "f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:43.409140 1805825 cri.go:89] found id: ""
I0912 23:27:43.409148 1805825 logs.go:276] 2 containers: [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab]
I0912 23:27:43.409205 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.414385 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.418038 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0912 23:27:43.418103 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0912 23:27:43.470267 1805825 cri.go:89] found id: "761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:43.470287 1805825 cri.go:89] found id: "c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:43.470292 1805825 cri.go:89] found id: ""
I0912 23:27:43.470299 1805825 logs.go:276] 2 containers: [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28]
I0912 23:27:43.470361 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.474528 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.478036 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0912 23:27:43.478114 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0912 23:27:43.520737 1805825 cri.go:89] found id: "ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:43.520766 1805825 cri.go:89] found id: "e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:43.520773 1805825 cri.go:89] found id: ""
I0912 23:27:43.520780 1805825 logs.go:276] 2 containers: [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9]
I0912 23:27:43.520840 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.524564 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.528472 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0912 23:27:43.528544 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0912 23:27:43.575684 1805825 cri.go:89] found id: "94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:43.575760 1805825 cri.go:89] found id: "f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:43.575767 1805825 cri.go:89] found id: ""
I0912 23:27:43.575775 1805825 logs.go:276] 2 containers: [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71]
I0912 23:27:43.575853 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.579573 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.582968 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0912 23:27:43.583040 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0912 23:27:43.622197 1805825 cri.go:89] found id: "e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:43.622232 1805825 cri.go:89] found id: "ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:43.622242 1805825 cri.go:89] found id: ""
I0912 23:27:43.622267 1805825 logs.go:276] 2 containers: [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99]
I0912 23:27:43.622346 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.625917 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.629579 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0912 23:27:43.629657 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0912 23:27:43.672137 1805825 cri.go:89] found id: "f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:43.672160 1805825 cri.go:89] found id: "2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:43.672166 1805825 cri.go:89] found id: ""
I0912 23:27:43.672174 1805825 logs.go:276] 2 containers: [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade]
I0912 23:27:43.672232 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.675927 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.680672 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0912 23:27:43.680740 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0912 23:27:43.718009 1805825 cri.go:89] found id: "3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:43.718034 1805825 cri.go:89] found id: ""
I0912 23:27:43.718042 1805825 logs.go:276] 1 containers: [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8]
I0912 23:27:43.718099 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.721672 1805825 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0912 23:27:43.721750 1805825 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0912 23:27:43.759833 1805825 cri.go:89] found id: "1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:43.759855 1805825 cri.go:89] found id: "91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:43.759860 1805825 cri.go:89] found id: ""
I0912 23:27:43.759867 1805825 logs.go:276] 2 containers: [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299]
I0912 23:27:43.759939 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.763575 1805825 ssh_runner.go:195] Run: which crictl
I0912 23:27:43.767207 1805825 logs.go:123] Gathering logs for kubernetes-dashboard [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8] ...
I0912 23:27:43.767286 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8"
I0912 23:27:43.812189 1805825 logs.go:123] Gathering logs for etcd [f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab] ...
I0912 23:27:43.812262 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab"
I0912 23:27:43.860473 1805825 logs.go:123] Gathering logs for coredns [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623] ...
I0912 23:27:43.860501 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623"
I0912 23:27:43.901572 1805825 logs.go:123] Gathering logs for kube-proxy [f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71] ...
I0912 23:27:43.901599 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71"
I0912 23:27:43.941592 1805825 logs.go:123] Gathering logs for kube-controller-manager [ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99] ...
I0912 23:27:43.941623 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99"
I0912 23:27:44.000715 1805825 logs.go:123] Gathering logs for kindnet [2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade] ...
I0912 23:27:44.000754 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade"
I0912 23:27:44.049375 1805825 logs.go:123] Gathering logs for storage-provisioner [91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299] ...
I0912 23:27:44.049402 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299"
I0912 23:27:44.095108 1805825 logs.go:123] Gathering logs for containerd ...
I0912 23:27:44.095139 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0912 23:27:44.155926 1805825 logs.go:123] Gathering logs for kubelet ...
I0912 23:27:44.155961 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0912 23:27:44.207364 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411026 664 reflector.go:138] object-"kube-system"/"coredns-token-m2js8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2js8" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.207620 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411278 664 reflector.go:138] object-"kube-system"/"kindnet-token-xzbvw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xzbvw" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.207833 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411446 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208051 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411664 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-k7dwz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-k7dwz" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208258 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.411960 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208485 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414071 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-f7cln": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-f7cln" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208697 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414335 664 reflector.go:138] object-"default"/"default-token-bxtgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-bxtgn" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.208921 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:07 old-k8s-version-011723 kubelet[664]: E0912 23:22:07.414403 664 reflector.go:138] object-"kube-system"/"metrics-server-token-fdc76": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-fdc76" is forbidden: User "system:node:old-k8s-version-011723" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-011723' and this object
W0912 23:27:44.215368 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.557235 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.217198 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:09 old-k8s-version-011723 kubelet[664]: E0912 23:22:09.840628 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.221475 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.466661 664 pod_workers.go:191] Error syncing pod e7e03576-7399-4fcf-8ab1-dfe79e82e9bc ("storage-provisioner_kube-system(e7e03576-7399-4fcf-8ab1-dfe79e82e9bc)"), skipping: failed to "StartContainer" for "storage-provisioner" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.223249 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:10 old-k8s-version-011723 kubelet[664]: E0912 23:22:10.472436 664 pod_workers.go:191] Error syncing pod 5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc ("kindnet-rdqkd_kube-system(5cabfb63-8e1c-4bbb-9965-e00fc5de8bbc)"), skipping: failed to "StartContainer" for "kindnet-cni" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.224648 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.366884 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.226290 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.463324 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.228181 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.478847 664 pod_workers.go:191] Error syncing pod 3b3346e8-fe2e-46fc-8cd9-21264698e11a ("coredns-74ff55c5b-lzb66_kube-system(3b3346e8-fe2e-46fc-8cd9-21264698e11a)"), skipping: failed to "StartContainer" for "coredns" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.229717 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.482852 664 pod_workers.go:191] Error syncing pod c2d2a606-20fe-4e2b-a1c0-ba5741b38145 ("kube-proxy-cd4m4_kube-system(c2d2a606-20fe-4e2b-a1c0-ba5741b38145)"), skipping: failed to "StartContainer" for "kube-proxy" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.230538 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.566317 664 pod_workers.go:191] Error syncing pod 7a260c8b-3e99-476d-bb2a-f42a54017c50 ("busybox_default(7a260c8b-3e99-476d-bb2a-f42a54017c50)"), skipping: failed to "StartContainer" for "busybox" with CreateContainerConfigError: "services have not yet been read at least once, cannot construct envvars"
W0912 23:27:44.233034 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:11 old-k8s-version-011723 kubelet[664]: E0912 23:22:11.591024 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.233356 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:12 old-k8s-version-011723 kubelet[664]: E0912 23:22:12.488949 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.238182 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:34 old-k8s-version-011723 kubelet[664]: E0912 23:22:34.530645 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.239114 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:36 old-k8s-version-011723 kubelet[664]: E0912 23:22:36.632305 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.239454 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:37 old-k8s-version-011723 kubelet[664]: E0912 23:22:37.632313 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.239792 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:44 old-k8s-version-011723 kubelet[664]: E0912 23:22:44.683335 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.240321 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:47 old-k8s-version-011723 kubelet[664]: E0912 23:22:47.274232 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.240914 1805825 logs.go:138] Found kubelet problem: Sep 12 23:22:58 old-k8s-version-011723 kubelet[664]: E0912 23:22:58.684667 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.243409 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:00 old-k8s-version-011723 kubelet[664]: E0912 23:23:00.296836 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.243745 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:04 old-k8s-version-011723 kubelet[664]: E0912 23:23:04.683877 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.243932 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:12 old-k8s-version-011723 kubelet[664]: E0912 23:23:12.274275 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.244259 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:16 old-k8s-version-011723 kubelet[664]: E0912 23:23:16.273507 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.244448 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:23 old-k8s-version-011723 kubelet[664]: E0912 23:23:23.274327 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.245036 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:29 old-k8s-version-011723 kubelet[664]: E0912 23:23:29.784506 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.245365 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:34 old-k8s-version-011723 kubelet[664]: E0912 23:23:34.683510 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.245553 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:38 old-k8s-version-011723 kubelet[664]: E0912 23:23:38.273957 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.245882 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:50 old-k8s-version-011723 kubelet[664]: E0912 23:23:50.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.248381 1805825 logs.go:138] Found kubelet problem: Sep 12 23:23:52 old-k8s-version-011723 kubelet[664]: E0912 23:23:52.283800 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.248855 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:01 old-k8s-version-011723 kubelet[664]: E0912 23:24:01.274256 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.250523 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:04 old-k8s-version-011723 kubelet[664]: E0912 23:24:04.274131 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.251141 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:14 old-k8s-version-011723 kubelet[664]: E0912 23:24:14.908668 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.251337 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:15 old-k8s-version-011723 kubelet[664]: E0912 23:24:15.282101 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.251674 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:24 old-k8s-version-011723 kubelet[664]: E0912 23:24:24.683372 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.251898 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:27 old-k8s-version-011723 kubelet[664]: E0912 23:24:27.273932 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252232 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:40 old-k8s-version-011723 kubelet[664]: E0912 23:24:40.274451 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.252419 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:42 old-k8s-version-011723 kubelet[664]: E0912 23:24:42.274046 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252606 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:53 old-k8s-version-011723 kubelet[664]: E0912 23:24:53.273989 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.252934 1805825 logs.go:138] Found kubelet problem: Sep 12 23:24:54 old-k8s-version-011723 kubelet[664]: E0912 23:24:54.273546 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.253120 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:04 old-k8s-version-011723 kubelet[664]: E0912 23:25:04.273901 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.253473 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:05 old-k8s-version-011723 kubelet[664]: E0912 23:25:05.274360 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.255953 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:15 old-k8s-version-011723 kubelet[664]: E0912 23:25:15.305120 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0912 23:27:44.256289 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:19 old-k8s-version-011723 kubelet[664]: E0912 23:25:19.274155 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.256481 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:28 old-k8s-version-011723 kubelet[664]: E0912 23:25:28.273983 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.256812 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:30 old-k8s-version-011723 kubelet[664]: E0912 23:25:30.273912 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.256999 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:39 old-k8s-version-011723 kubelet[664]: E0912 23:25:39.277598 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.257586 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:43 old-k8s-version-011723 kubelet[664]: E0912 23:25:43.182848 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.257913 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:44 old-k8s-version-011723 kubelet[664]: E0912 23:25:44.683303 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.258101 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:51 old-k8s-version-011723 kubelet[664]: E0912 23:25:51.278776 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.258443 1805825 logs.go:138] Found kubelet problem: Sep 12 23:25:58 old-k8s-version-011723 kubelet[664]: E0912 23:25:58.273502 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.258628 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:03 old-k8s-version-011723 kubelet[664]: E0912 23:26:03.278105 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.258985 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:10 old-k8s-version-011723 kubelet[664]: E0912 23:26:10.273581 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.259180 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:14 old-k8s-version-011723 kubelet[664]: E0912 23:26:14.273888 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.259511 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: E0912 23:26:22.273505 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.259708 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:27 old-k8s-version-011723 kubelet[664]: E0912 23:26:27.278136 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260056 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: E0912 23:26:36.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.260247 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:38 old-k8s-version-011723 kubelet[664]: E0912 23:26:38.273832 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260434 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:49 old-k8s-version-011723 kubelet[664]: E0912 23:26:49.273962 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.260762 1805825 logs.go:138] Found kubelet problem: Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: E0912 23:26:50.273517 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.260949 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.261277 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.261466 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.261795 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.261982 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.262312 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.262501 1805825 logs.go:138] Found kubelet problem: Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:44.262513 1805825 logs.go:123] Gathering logs for describe nodes ...
I0912 23:27:44.262527 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0912 23:27:44.406836 1805825 logs.go:123] Gathering logs for kube-apiserver [5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a] ...
I0912 23:27:44.406930 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a"
I0912 23:27:44.479657 1805825 logs.go:123] Gathering logs for kube-scheduler [e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9] ...
I0912 23:27:44.479689 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9"
I0912 23:27:44.528355 1805825 logs.go:123] Gathering logs for storage-provisioner [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9] ...
I0912 23:27:44.528385 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9"
I0912 23:27:44.590340 1805825 logs.go:123] Gathering logs for dmesg ...
I0912 23:27:44.590373 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0912 23:27:44.607129 1805825 logs.go:123] Gathering logs for kube-apiserver [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b] ...
I0912 23:27:44.607185 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b"
I0912 23:27:44.669500 1805825 logs.go:123] Gathering logs for kube-scheduler [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb] ...
I0912 23:27:44.669536 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb"
I0912 23:27:44.708883 1805825 logs.go:123] Gathering logs for container status ...
I0912 23:27:44.708912 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0912 23:27:44.757271 1805825 logs.go:123] Gathering logs for etcd [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa] ...
I0912 23:27:44.757307 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa"
I0912 23:27:44.800471 1805825 logs.go:123] Gathering logs for coredns [c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28] ...
I0912 23:27:44.800505 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28"
I0912 23:27:44.845088 1805825 logs.go:123] Gathering logs for kube-proxy [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168] ...
I0912 23:27:44.845117 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168"
I0912 23:27:44.883749 1805825 logs.go:123] Gathering logs for kube-controller-manager [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340] ...
I0912 23:27:44.883783 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340"
I0912 23:27:44.946173 1805825 logs.go:123] Gathering logs for kindnet [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64] ...
I0912 23:27:44.946213 1805825 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64"
I0912 23:27:44.987811 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:44.987836 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0912 23:27:44.987883 1805825 out.go:270] X Problems detected in kubelet:
W0912 23:27:44.987898 1805825 out.go:270] Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.987905 1805825 out.go:270] Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.987918 1805825 out.go:270] Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0912 23:27:44.987925 1805825 out.go:270] Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
W0912 23:27:44.987931 1805825 out.go:270] Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0912 23:27:44.987940 1805825 out.go:358] Setting ErrFile to fd 2...
I0912 23:27:44.987946 1805825 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0912 23:27:54.989447 1805825 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0912 23:27:55.001137 1805825 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0912 23:27:55.007655 1805825 out.go:201]
W0912 23:27:55.012275 1805825 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0912 23:27:55.012327 1805825 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0912 23:27:55.012348 1805825 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0912 23:27:55.012355 1805825 out.go:270] *
W0912 23:27:55.013626 1805825 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0912 23:27:55.016564 1805825 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
23ddd9c911f9a 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 8536b8402b347 dashboard-metrics-scraper-8d5bb5db8-gxw94
3bea0dd674f7b 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 e6fd6729758f7 kubernetes-dashboard-cd95d586-nbkcq
f6ce1f61d9cdc 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 e2e9a7894e9a9 kindnet-rdqkd
1b307b6cba68d ba04bb24b9575 5 minutes ago Running storage-provisioner 2 3b4f3f0b9a7ea storage-provisioner
94347fb65e2d7 25a5233254979 5 minutes ago Running kube-proxy 1 a89a35f584206 kube-proxy-cd4m4
761c0b37e3bd3 db91994f4ee8f 5 minutes ago Running coredns 1 d4036057106b1 coredns-74ff55c5b-lzb66
60a332fd5f5ef 1611cd07b61d5 5 minutes ago Running busybox 1 f289cf8c42a9c busybox
e81f940befd9c 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 e26b57d35b17e kube-controller-manager-old-k8s-version-011723
ea0849f05ee3e e7605f88f17d6 6 minutes ago Running kube-scheduler 1 68eb3f2051cfa kube-scheduler-old-k8s-version-011723
e0f86d0bfe33b 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 7b6b5beb05e89 kube-apiserver-old-k8s-version-011723
acc9f1ddc829d 05b738aa1bc63 6 minutes ago Running etcd 1 862d77e5c28c7 etcd-old-k8s-version-011723
a1e8f9e5e6f5a 1611cd07b61d5 6 minutes ago Exited busybox 0 57e50fc774581 busybox
91179dc2e027f ba04bb24b9575 7 minutes ago Exited storage-provisioner 1 e4a3396b752cf storage-provisioner
c93b4416e8ffc db91994f4ee8f 7 minutes ago Exited coredns 0 5169b70f43f3b coredns-74ff55c5b-lzb66
2c2b9c95f2047 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 f9a9023248a6d kindnet-rdqkd
f6290a74934bc 25a5233254979 8 minutes ago Exited kube-proxy 0 7885efd8707da kube-proxy-cd4m4
5a9ffb0bdef03 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 310f2ed043cf4 kube-apiserver-old-k8s-version-011723
ecb0941161c01 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 f34ebf9f08469 kube-controller-manager-old-k8s-version-011723
e76fc4569f919 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 bf5bd507cacea kube-scheduler-old-k8s-version-011723
f32f98ccfd632 05b738aa1bc63 8 minutes ago Exited etcd 0 9830e9e33988c etcd-old-k8s-version-011723
==> containerd <==
Sep 12 23:23:52 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:23:52.282388954Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 12 23:23:52 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:23:52.282509790Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.276467592Z" level=info msg="CreateContainer within sandbox \"8536b8402b347a760910916dc213a9eff37a646ad2d6185562f562ae2d62ab9a\" for container name:\"dashboard-metrics-scraper\" attempt:4"
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.309104420Z" level=info msg="CreateContainer within sandbox \"8536b8402b347a760910916dc213a9eff37a646ad2d6185562f562ae2d62ab9a\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea\""
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.311188483Z" level=info msg="StartContainer for \"e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea\""
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.380224942Z" level=info msg="StartContainer for \"e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea\" returns successfully"
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.409213117Z" level=info msg="shim disconnected" id=e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea namespace=k8s.io
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.409276091Z" level=warning msg="cleaning up after shim disconnected" id=e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea namespace=k8s.io
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.409287061Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.437051855Z" level=warning msg="cleanup warnings time=\"2024-09-12T23:24:14Z\" level=warning msg=\"failed to remove runc container\" error=\"runc did not terminate successfully: exit status 255: \" runtime=io.containerd.runc.v2\n" namespace=k8s.io
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.910257069Z" level=info msg="RemoveContainer for \"9e422168b4f2d9c594c354f070cdcf0beef1fca4a58ac299287c2ec6e8bd52b2\""
Sep 12 23:24:14 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:24:14.916063156Z" level=info msg="RemoveContainer for \"9e422168b4f2d9c594c354f070cdcf0beef1fca4a58ac299287c2ec6e8bd52b2\" returns successfully"
Sep 12 23:25:15 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:15.274623782Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:25:15 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:15.299003060Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Sep 12 23:25:15 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:15.304398128Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Sep 12 23:25:15 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:15.304419560Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.275855691Z" level=info msg="CreateContainer within sandbox \"8536b8402b347a760910916dc213a9eff37a646ad2d6185562f562ae2d62ab9a\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.291787307Z" level=info msg="CreateContainer within sandbox \"8536b8402b347a760910916dc213a9eff37a646ad2d6185562f562ae2d62ab9a\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a\""
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.292591891Z" level=info msg="StartContainer for \"23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a\""
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.364076987Z" level=info msg="StartContainer for \"23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a\" returns successfully"
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.388443055Z" level=info msg="shim disconnected" id=23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a namespace=k8s.io
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.388755562Z" level=warning msg="cleaning up after shim disconnected" id=23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a namespace=k8s.io
Sep 12 23:25:42 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:42.388866036Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 12 23:25:43 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:43.182164610Z" level=info msg="RemoveContainer for \"e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea\""
Sep 12 23:25:43 old-k8s-version-011723 containerd[570]: time="2024-09-12T23:25:43.188134823Z" level=info msg="RemoveContainer for \"e711c2685fc81641a06aa2ffe379c139a40b34354f345ca3f90af3d05f2c21ea\" returns successfully"
==> coredns [761c0b37e3bd3fd65bc1c99066ad373975baea39f4a526a13061e89e5f038623] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] 127.0.0.1:59128 - 50869 "HINFO IN 3671984939070762930.3880467561831516647. udp 57 false 512" NXDOMAIN qr,rd,ra 57 2.034883183s
[INFO] 127.0.0.1:51662 - 32959 "HINFO IN 3671984939070762930.3880467561831516647. udp 57 false 512" NOERROR - 0 6.001300457s
[ERROR] plugin/errors: 2 3671984939070762930.3880467561831516647. HINFO: read udp 10.244.0.2:54622->192.168.76.1:53: i/o timeout
[INFO] 127.0.0.1:59587 - 480 "HINFO IN 3671984939070762930.3880467561831516647. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.003677633s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0912 23:22:43.448178 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-12 23:22:13.44755985 +0000 UTC m=+0.021195275) (total time: 30.000520928s):
Trace[939984059]: [30.000520928s] [30.000520928s] END
E0912 23:22:43.448213 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0912 23:22:43.448453 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-12 23:22:13.447561959 +0000 UTC m=+0.021197375) (total time: 30.000876103s):
Trace[1474941318]: [30.000876103s] [30.000876103s] END
E0912 23:22:43.448465 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0912 23:22:43.448864 1 trace.go:116] Trace[140954425]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-12 23:22:13.447643509 +0000 UTC m=+0.021278926) (total time: 30.001207121s):
Trace[140954425]: [30.001207121s] [30.001207121s] END
E0912 23:22:43.448876 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [c93b4416e8ffc76bcfb3de94b8fa5802dd63dc41e4acc4fc544c34aa1feb1b28] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:58995 - 8857 "HINFO IN 8170924649044574805.1020949474233048069. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.025290821s
==> describe nodes <==
Name: old-k8s-version-011723
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-011723
kubernetes.io/os=linux
minikube.k8s.io/commit=f6bc674a17941874d4e5b792b09c1791d30622b8
minikube.k8s.io/name=old-k8s-version-011723
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_12T23_19_15_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 12 Sep 2024 23:19:11 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-011723
AcquireTime: <unset>
RenewTime: Thu, 12 Sep 2024 23:27:50 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 12 Sep 2024 23:23:00 +0000 Thu, 12 Sep 2024 23:19:05 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 12 Sep 2024 23:23:00 +0000 Thu, 12 Sep 2024 23:19:05 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 12 Sep 2024 23:23:00 +0000 Thu, 12 Sep 2024 23:19:05 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 12 Sep 2024 23:23:00 +0000 Thu, 12 Sep 2024 23:19:30 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-011723
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022292Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022292Ki
pods: 110
System Info:
Machine ID: d2fe619b0021444289fc6c0a284ab2ea
System UUID: 86eab556-700e-47b9-a657-e438f91a4689
Boot ID: df7282e8-9021-4c1b-a6eb-f0483f23e85d
Kernel Version: 5.15.0-1068-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m44s
kube-system coredns-74ff55c5b-lzb66 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m28s
kube-system etcd-old-k8s-version-011723 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m35s
kube-system kindnet-rdqkd 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m28s
kube-system kube-apiserver-old-k8s-version-011723 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m35s
kube-system kube-controller-manager-old-k8s-version-011723 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m35s
kube-system kube-proxy-cd4m4 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m28s
kube-system kube-scheduler-old-k8s-version-011723 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m35s
kube-system metrics-server-9975d5f86-gklxg 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m32s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m26s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-gxw94 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m33s
kubernetes-dashboard kubernetes-dashboard-cd95d586-nbkcq 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m33s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m55s (x5 over 8m55s) kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m55s (x4 over 8m55s) kubelet Node old-k8s-version-011723 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m55s (x4 over 8m55s) kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientPID
Normal Starting 8m35s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m35s kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m35s kubelet Node old-k8s-version-011723 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m35s kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m35s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m28s kubelet Node old-k8s-version-011723 status is now: NodeReady
Normal Starting 8m25s kube-proxy Starting kube-proxy.
Normal Starting 6m3s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-011723 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m3s (x7 over 6m3s) kubelet Node old-k8s-version-011723 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m3s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m35s kube-proxy Starting kube-proxy.
==> dmesg <==
==> etcd [acc9f1ddc829dfa9f554d203a0dc76b0defba802e8450cc94a12fb75c99126aa] <==
2024-09-12 23:23:58.021591 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:08.020606 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:18.020622 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:28.020682 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:38.020609 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:48.020623 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:24:58.020766 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:08.020706 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:18.020709 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:28.020754 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:38.020582 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:48.020629 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:25:58.020593 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:08.020574 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:18.020762 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:28.020474 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:38.020690 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:48.020664 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:26:58.020833 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:08.021113 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:18.020721 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:28.020668 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:38.020699 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:48.021040 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:27:58.021395 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [f32f98ccfd63279d6f3bc06a836ad9216250db5bb7e829a5947b9f9335ab97ab] <==
raft2024/09/12 23:19:05 INFO: ea7e25599daad906 became candidate at term 2
raft2024/09/12 23:19:05 INFO: ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2
raft2024/09/12 23:19:05 INFO: ea7e25599daad906 became leader at term 2
raft2024/09/12 23:19:05 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2024-09-12 23:19:05.371338 I | etcdserver: setting up the initial cluster version to 3.4
2024-09-12 23:19:05.372607 N | etcdserver/membership: set the initial cluster version to 3.4
2024-09-12 23:19:05.372703 I | etcdserver/api: enabled capabilities for version 3.4
2024-09-12 23:19:05.372755 I | etcdserver: published {Name:old-k8s-version-011723 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-09-12 23:19:05.372895 I | embed: ready to serve client requests
2024-09-12 23:19:05.374305 I | embed: serving client requests on 192.168.76.2:2379
2024-09-12 23:19:05.378230 I | embed: ready to serve client requests
2024-09-12 23:19:05.384164 I | embed: serving client requests on 127.0.0.1:2379
2024-09-12 23:19:27.085121 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:19:29.757767 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:19:39.757978 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:19:49.757859 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:19:59.757852 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:09.757924 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:19.757937 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:29.757926 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:39.757921 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:49.757907 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:20:59.757973 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:21:09.758089 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-12 23:21:19.758020 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
23:27:58 up 8:10, 0 users, load average: 0.74, 1.74, 2.48
Linux old-k8s-version-011723 5.15.0-1068-aws #74~20.04.1-Ubuntu SMP Tue Aug 6 19:45:17 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [2c2b9c95f20475fd1ded65056f329c800c5b72090b93755ea84cdae9c1c49ade] <==
I0912 23:19:34.719321 1 shared_informer.go:320] Caches are synced for kube-network-policies
I0912 23:19:34.719345 1 metrics.go:61] Registering metrics
I0912 23:19:34.719410 1 controller.go:374] Syncing nftables rules
I0912 23:19:44.525245 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:19:44.525303 1 main.go:299] handling current node
I0912 23:19:54.516312 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:19:54.516351 1 main.go:299] handling current node
I0912 23:20:04.516217 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:04.516323 1 main.go:299] handling current node
I0912 23:20:14.524940 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:14.524978 1 main.go:299] handling current node
I0912 23:20:24.524947 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:24.525000 1 main.go:299] handling current node
I0912 23:20:34.516856 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:34.516922 1 main.go:299] handling current node
I0912 23:20:44.519579 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:44.519829 1 main.go:299] handling current node
I0912 23:20:54.524918 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:20:54.524954 1 main.go:299] handling current node
I0912 23:21:04.516086 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:21:04.516122 1 main.go:299] handling current node
I0912 23:21:14.519776 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:21:14.519864 1 main.go:299] handling current node
I0912 23:21:24.516069 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:21:24.516103 1 main.go:299] handling current node
==> kindnet [f6ce1f61d9cdc3364f68e851d840e80c8ca80cb36c73fe5e3bebf58fffd67d64] <==
I0912 23:25:56.071852 1 main.go:299] handling current node
I0912 23:26:06.068157 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:06.068192 1 main.go:299] handling current node
I0912 23:26:16.060557 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:16.060592 1 main.go:299] handling current node
I0912 23:26:26.061179 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:26.061214 1 main.go:299] handling current node
I0912 23:26:36.067823 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:36.068173 1 main.go:299] handling current node
I0912 23:26:46.069876 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:46.069915 1 main.go:299] handling current node
I0912 23:26:56.067791 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:26:56.067825 1 main.go:299] handling current node
I0912 23:27:06.067837 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:06.067871 1 main.go:299] handling current node
I0912 23:27:16.063465 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:16.063532 1 main.go:299] handling current node
I0912 23:27:26.061334 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:26.061387 1 main.go:299] handling current node
I0912 23:27:36.067795 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:36.067929 1 main.go:299] handling current node
I0912 23:27:46.072372 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:46.072476 1 main.go:299] handling current node
I0912 23:27:56.070190 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0912 23:27:56.070228 1 main.go:299] handling current node
==> kube-apiserver [5a9ffb0bdef03445db5a2e0deb47eae9b3b5c491cc48e85ec57ddea267eedb4a] <==
I0912 23:19:12.638890 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0912 23:19:12.648376 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0912 23:19:12.652298 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0912 23:19:12.652511 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0912 23:19:13.175258 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0912 23:19:13.215262 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0912 23:19:13.306051 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I0912 23:19:13.307289 1 controller.go:606] quota admission added evaluator for: endpoints
I0912 23:19:13.311439 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0912 23:19:13.683503 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0912 23:19:14.286918 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0912 23:19:14.857071 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0912 23:19:14.985537 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0912 23:19:30.712590 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0912 23:19:30.758930 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0912 23:19:51.196903 1 client.go:360] parsed scheme: "passthrough"
I0912 23:19:51.196948 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:19:51.196958 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0912 23:20:32.410331 1 client.go:360] parsed scheme: "passthrough"
I0912 23:20:32.410375 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:20:32.410384 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0912 23:21:04.987924 1 client.go:360] parsed scheme: "passthrough"
I0912 23:21:04.988112 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:21:04.988130 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
E0912 23:21:24.792645 1 upgradeaware.go:373] Error proxying data from client to backend: write tcp 192.168.76.2:59782->192.168.76.2:10250: write: broken pipe
==> kube-apiserver [e0f86d0bfe33b8095d25853a16abf60a6eb01e25bce62a9626617a714402ae4b] <==
I0912 23:24:07.280314 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:24:07.280324 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0912 23:24:42.925731 1 client.go:360] parsed scheme: "passthrough"
I0912 23:24:42.925774 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:24:42.925783 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0912 23:25:09.871130 1 handler_proxy.go:102] no RequestInfo found in the context
E0912 23:25:09.871389 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0912 23:25:09.871498 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0912 23:25:27.603537 1 client.go:360] parsed scheme: "passthrough"
I0912 23:25:27.603584 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:25:27.603593 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0912 23:26:10.632999 1 client.go:360] parsed scheme: "passthrough"
I0912 23:26:10.633060 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:26:10.633083 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0912 23:26:54.532562 1 client.go:360] parsed scheme: "passthrough"
I0912 23:26:54.532609 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:26:54.532618 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0912 23:27:08.504413 1 handler_proxy.go:102] no RequestInfo found in the context
E0912 23:27:08.504492 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0912 23:27:08.504500 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0912 23:27:25.082001 1 client.go:360] parsed scheme: "passthrough"
I0912 23:27:25.082050 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0912 23:27:25.082070 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [e81f940befd9c32db69b7ccd086e72889df854438d8dbfbae111448f01452340] <==
W0912 23:23:31.018045 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:23:57.059274 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:24:02.668636 1 request.go:655] Throttling request took 1.048341157s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0912 23:24:03.520004 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:24:27.561287 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:24:35.170411 1 request.go:655] Throttling request took 1.04861441s, request: GET:https://192.168.76.2:8443/apis/coordination.k8s.io/v1?timeout=32s
W0912 23:24:36.022086 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:24:58.063200 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:25:07.672635 1 request.go:655] Throttling request took 1.048380054s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0912 23:25:08.524435 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:25:28.565092 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:25:40.174865 1 request.go:655] Throttling request took 1.048236597s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0912 23:25:41.026457 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:25:59.107870 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:26:12.676846 1 request.go:655] Throttling request took 1.048259507s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0912 23:26:13.528392 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:26:29.610319 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:26:45.130267 1 request.go:655] Throttling request took 1.0000563s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0912 23:26:46.030302 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:27:00.188200 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:27:17.680904 1 request.go:655] Throttling request took 1.047839369s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0912 23:27:18.532652 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0912 23:27:30.690245 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0912 23:27:50.183348 1 request.go:655] Throttling request took 1.048584954s, request: GET:https://192.168.76.2:8443/apis/authentication.k8s.io/v1?timeout=32s
W0912 23:27:51.034775 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [ecb0941161c01f2903ae5ed80b627c909a9be9cdbe516b50d630cc3ee9623e99] <==
I0912 23:19:30.717542 1 shared_informer.go:247] Caches are synced for certificate-csrapproving
I0912 23:19:30.725535 1 shared_informer.go:247] Caches are synced for stateful set
I0912 23:19:30.736782 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0912 23:19:30.741518 1 shared_informer.go:247] Caches are synced for crt configmap
I0912 23:19:30.760797 1 shared_informer.go:247] Caches are synced for ReplicationController
I0912 23:19:30.786973 1 range_allocator.go:373] Set node old-k8s-version-011723 PodCIDR to [10.244.0.0/24]
I0912 23:19:30.794869 1 shared_informer.go:247] Caches are synced for endpoint_slice
I0912 23:19:30.847650 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-b6hmj"
I0912 23:19:30.864927 1 shared_informer.go:247] Caches are synced for endpoint
I0912 23:19:30.865400 1 shared_informer.go:247] Caches are synced for endpoint_slice_mirroring
I0912 23:19:30.993538 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-lzb66"
I0912 23:19:30.993578 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-cd4m4"
I0912 23:19:31.019516 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-rdqkd"
I0912 23:19:31.048904 1 shared_informer.go:247] Caches are synced for resource quota
I0912 23:19:31.068646 1 shared_informer.go:247] Caches are synced for resource quota
I0912 23:19:31.165791 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
E0912 23:19:31.204157 1 daemon_controller.go:320] kube-system/kindnet failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kindnet", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"3a79f40c-c3f2-4416-b5d5-ec941e4e0c3a", ResourceVersion:"266", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63861779955, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"},\"name\":\"kindnet\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"k
indnet\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"HOST_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.hostIP\"}}},{\"name\":\"POD_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"POD_SUBNET\",\"value\":\"10.244.0.0/16\"}],\"image\":\"docker.io/kindest/kindnetd:v20240813-c6f155d6\",\"name\":\"kindnet-cni\",\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"}},\"securityContext\":{\"capabilities\":{\"add\":[\"NET_RAW\",\"NET_ADMIN\"]},\"privileged\":false},\"volumeMounts\":[{\"mountPath\":\"/etc/cni/net.d\",\"name\":\"cni-cfg\"},{\"mountPath\":\"/run/xtables.lock\",\"name\":\"xtables-lock\",\"readOnly\":false},{\"mountPath\":\"/lib/modules\",\"name\":\"lib-modules\",\"readOnly\":true}]}],\"hostNetwork\":true,\"serviceAccountName\":\"kindnet\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"operator\
":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/etc/cni/net.d\",\"type\":\"DirectoryOrCreate\"},\"name\":\"cni-cfg\"},{\"hostPath\":{\"path\":\"/run/xtables.lock\",\"type\":\"FileOrCreate\"},\"name\":\"xtables-lock\"},{\"hostPath\":{\"path\":\"/lib/modules\"},\"name\":\"lib-modules\"}]}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001db74a0), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001db74c0)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0x4001db74e0), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string
{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"cni-cfg", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001db7500), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil),
FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001db7520), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Glust
erfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001db7540), EmptyDi
r:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil),
PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kindnet-cni", Image:"docker.io/kindest/kindnetd:v20240813-c6f155d6", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"HOST_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001db7560)}, v1.EnvVar{Name:"POD_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001db75a0)}, v1.EnvVar{Name:"POD_SUBNET", Value:"10.244.0.0/16", ValueFrom:(*v1.EnvVarSource)(nil)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:
0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"cni-cfg", ReadOnly:false, MountPath:"/etc/cni/net.d", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:
(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x4001da5800), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0x4000e8b9c8), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"kindnet", DeprecatedServiceAccount:"kindnet", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x4000ae87e0), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}},
HostAliases:[]v1.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x40002fbe80)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x4000e8ba10)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kindnet": the object has been modified; please apply your changes to the latest version and try again
I0912 23:19:31.363206 1 shared_informer.go:247] Caches are synced for garbage collector
I0912 23:19:31.363230 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0912 23:19:31.366078 1 shared_informer.go:247] Caches are synced for garbage collector
I0912 23:19:32.247887 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0912 23:19:32.309610 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-b6hmj"
I0912 23:19:35.666875 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0912 23:21:25.724288 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0912 23:21:25.835050 1 clusterroleaggregation_controller.go:181] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [94347fb65e2d7117514ca1de8cb6276745b15849b93c7affe7df10dbebbf9168] <==
I0912 23:22:23.473234 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0912 23:22:23.473565 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0912 23:22:23.493076 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0912 23:22:23.493342 1 server_others.go:185] Using iptables Proxier.
I0912 23:22:23.493935 1 server.go:650] Version: v1.20.0
I0912 23:22:23.495074 1 config.go:315] Starting service config controller
I0912 23:22:23.497345 1 shared_informer.go:240] Waiting for caches to sync for service config
I0912 23:22:23.496347 1 config.go:224] Starting endpoint slice config controller
I0912 23:22:23.500851 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0912 23:22:23.597704 1 shared_informer.go:247] Caches are synced for service config
I0912 23:22:23.601051 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [f6290a74934bc9024e7157150e63817e6e4d073704f09667759df0a0cced6b71] <==
I0912 23:19:33.573271 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0912 23:19:33.573357 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0912 23:19:33.619819 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0912 23:19:33.619946 1 server_others.go:185] Using iptables Proxier.
I0912 23:19:33.620187 1 server.go:650] Version: v1.20.0
I0912 23:19:33.620661 1 config.go:315] Starting service config controller
I0912 23:19:33.620669 1 shared_informer.go:240] Waiting for caches to sync for service config
I0912 23:19:33.621266 1 config.go:224] Starting endpoint slice config controller
I0912 23:19:33.621272 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0912 23:19:33.720779 1 shared_informer.go:247] Caches are synced for service config
I0912 23:19:33.721404 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [e76fc4569f9197f1a6d467c766d1036a47241f029e1d985fb693b6a76a3fa1c9] <==
W0912 23:19:11.780104 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0912 23:19:11.780165 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0912 23:19:11.780177 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0912 23:19:11.780183 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0912 23:19:11.893695 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0912 23:19:11.895988 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0912 23:19:11.896206 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0912 23:19:11.896361 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0912 23:19:11.926354 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0912 23:19:11.927454 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0912 23:19:11.927542 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0912 23:19:11.928441 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0912 23:19:11.929052 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0912 23:19:11.929802 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0912 23:19:11.929865 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0912 23:19:11.929629 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0912 23:19:11.929683 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0912 23:19:11.929743 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0912 23:19:11.934886 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0912 23:19:11.958489 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0912 23:19:12.824554 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0912 23:19:12.952242 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0912 23:19:12.971555 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0912 23:19:13.011987 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
I0912 23:19:13.296728 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [ea0849f05ee3ec463fbcc693795336241b92e45201b97c6b584ebd5baa093cbb] <==
I0912 23:22:00.794537 1 serving.go:331] Generated self-signed cert in-memory
W0912 23:22:07.223860 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0912 23:22:07.223945 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0912 23:22:07.223979 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0912 23:22:07.224004 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0912 23:22:07.522791 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0912 23:22:07.524046 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0912 23:22:07.524069 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0912 23:22:07.524128 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0912 23:22:07.725719 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Sep 12 23:26:14 old-k8s-version-011723 kubelet[664]: E0912 23:26:14.273888 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: I0912 23:26:22.273175 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:26:22 old-k8s-version-011723 kubelet[664]: E0912 23:26:22.273505 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:26:27 old-k8s-version-011723 kubelet[664]: E0912 23:26:27.278136 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: I0912 23:26:36.273368 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:26:36 old-k8s-version-011723 kubelet[664]: E0912 23:26:36.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:26:38 old-k8s-version-011723 kubelet[664]: E0912 23:26:38.273832 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:26:49 old-k8s-version-011723 kubelet[664]: E0912 23:26:49.273962 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: I0912 23:26:50.273189 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:26:50 old-k8s-version-011723 kubelet[664]: E0912 23:26:50.273517 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:01 old-k8s-version-011723 kubelet[664]: E0912 23:27:01.273890 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: I0912 23:27:04.273178 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:27:04 old-k8s-version-011723 kubelet[664]: E0912 23:27:04.273536 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:15 old-k8s-version-011723 kubelet[664]: E0912 23:27:15.274398 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: I0912 23:27:18.273245 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:27:18 old-k8s-version-011723 kubelet[664]: E0912 23:27:18.273947 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:30 old-k8s-version-011723 kubelet[664]: E0912 23:27:30.273978 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: I0912 23:27:33.273393 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:27:33 old-k8s-version-011723 kubelet[664]: E0912 23:27:33.273795 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:42 old-k8s-version-011723 kubelet[664]: E0912 23:27:42.274058 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:47 old-k8s-version-011723 kubelet[664]: I0912 23:27:47.273179 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:27:47 old-k8s-version-011723 kubelet[664]: E0912 23:27:47.273941 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
Sep 12 23:27:54 old-k8s-version-011723 kubelet[664]: E0912 23:27:54.273872 664 pod_workers.go:191] Error syncing pod 1150ff82-3fff-429c-aef1-349e4c755bda ("metrics-server-9975d5f86-gklxg_kube-system(1150ff82-3fff-429c-aef1-349e4c755bda)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 12 23:27:58 old-k8s-version-011723 kubelet[664]: I0912 23:27:58.273201 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: 23ddd9c911f9ae9e32ab0ddb7797d7d891c0ddb41354e75d9185df9ba842120a
Sep 12 23:27:58 old-k8s-version-011723 kubelet[664]: E0912 23:27:58.273548 664 pod_workers.go:191] Error syncing pod 1e945ecb-a33d-4b6a-9bd7-caa823971a60 ("dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-gxw94_kubernetes-dashboard(1e945ecb-a33d-4b6a-9bd7-caa823971a60)"
==> kubernetes-dashboard [3bea0dd674f7bd0c10cafef9ebed5cc94e155908f2a0a081b7ea53cc3d5699b8] <==
2024/09/12 23:22:30 Using namespace: kubernetes-dashboard
2024/09/12 23:22:30 Using in-cluster config to connect to apiserver
2024/09/12 23:22:30 Using secret token for csrf signing
2024/09/12 23:22:30 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/09/12 23:22:30 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/09/12 23:22:30 Successful initial request to the apiserver, version: v1.20.0
2024/09/12 23:22:30 Generating JWE encryption key
2024/09/12 23:22:30 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/09/12 23:22:30 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/09/12 23:22:31 Initializing JWE encryption key from synchronized object
2024/09/12 23:22:31 Creating in-cluster Sidecar client
2024/09/12 23:22:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:22:31 Serving insecurely on HTTP port: 9090
2024/09/12 23:23:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:23:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:24:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:24:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:25:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:25:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:26:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:26:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:27:01 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:27:31 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/12 23:22:30 Starting overwatch
==> storage-provisioner [1b307b6cba68d6c8149923a5069fe408b55a8df7afffe8f6b8482bde9ab7acb9] <==
I0912 23:22:24.374525 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0912 23:22:24.392494 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0912 23:22:24.392828 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0912 23:22:41.869533 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0912 23:22:41.869926 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-011723_68fc8e14-f8ca-404c-a536-d9f9fdbb5e8f!
I0912 23:22:41.869732 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"c0500b77-7c87-46c0-88c0-7d5184862bf0", APIVersion:"v1", ResourceVersion:"834", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-011723_68fc8e14-f8ca-404c-a536-d9f9fdbb5e8f became leader
I0912 23:22:41.970322 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-011723_68fc8e14-f8ca-404c-a536-d9f9fdbb5e8f!
==> storage-provisioner [91179dc2e027fa651d685d7cdf05c0aafca67932f357c8b73667610dd490b299] <==
I0912 23:20:03.600865 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0912 23:20:03.619858 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0912 23:20:03.619903 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0912 23:20:03.632545 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0912 23:20:03.632899 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-011723_94c33610-e68f-48ef-9266-61e70d52b772!
I0912 23:20:03.634203 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"c0500b77-7c87-46c0-88c0-7d5184862bf0", APIVersion:"v1", ResourceVersion:"503", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-011723_94c33610-e68f-48ef-9266-61e70d52b772 became leader
I0912 23:20:03.733308 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-011723_94c33610-e68f-48ef-9266-61e70d52b772!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-011723 -n old-k8s-version-011723
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-011723 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-gklxg
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-011723 describe pod metrics-server-9975d5f86-gklxg
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-011723 describe pod metrics-server-9975d5f86-gklxg: exit status 1 (417.301719ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-gklxg" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-011723 describe pod metrics-server-9975d5f86-gklxg: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (381.66s)