=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-633693 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-633693 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0: exit status 102 (6m20.383945433s)
-- stdout --
* [old-k8s-version-633693] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=17877
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/17877-1145955/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/17877-1145955/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
* Using the docker driver based on existing profile
* Starting "old-k8s-version-633693" primary control-plane node in "old-k8s-version-633693" cluster
* Pulling base image v0.0.43-1711559786-18485 ...
* Restarting existing docker container for "old-k8s-version-633693" ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
* Verifying Kubernetes components...
- Using image registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-633693 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, default-storageclass, dashboard
-- /stdout --
** stderr **
I0328 22:06:20.457738 1338826 out.go:291] Setting OutFile to fd 1 ...
I0328 22:06:20.457942 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:06:20.457968 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:06:20.457985 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:06:20.458264 1338826 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17877-1145955/.minikube/bin
I0328 22:06:20.458672 1338826 out.go:298] Setting JSON to false
I0328 22:06:20.459640 1338826 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":20931,"bootTime":1711642650,"procs":190,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1056-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0328 22:06:20.459741 1338826 start.go:139] virtualization:
I0328 22:06:20.463439 1338826 out.go:177] * [old-k8s-version-633693] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0328 22:06:20.465745 1338826 out.go:177] - MINIKUBE_LOCATION=17877
I0328 22:06:20.465819 1338826 notify.go:220] Checking for updates...
I0328 22:06:20.468803 1338826 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0328 22:06:20.471354 1338826 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:06:20.473216 1338826 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/17877-1145955/.minikube
I0328 22:06:20.475099 1338826 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0328 22:06:20.476838 1338826 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0328 22:06:20.479428 1338826 config.go:182] Loaded profile config "old-k8s-version-633693": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0328 22:06:20.483742 1338826 out.go:177] * Kubernetes 1.29.3 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.29.3
I0328 22:06:20.485706 1338826 driver.go:392] Setting default libvirt URI to qemu:///system
I0328 22:06:20.508597 1338826 docker.go:122] docker version: linux-26.0.0:Docker Engine - Community
I0328 22:06:20.508708 1338826 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0328 22:06:20.616696 1338826 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:59 SystemTime:2024-03-28 22:06:20.604272526 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0328 22:06:20.616798 1338826 docker.go:295] overlay module found
I0328 22:06:20.619546 1338826 out.go:177] * Using the docker driver based on existing profile
I0328 22:06:20.621460 1338826 start.go:297] selected driver: docker
I0328 22:06:20.621479 1338826 start.go:901] validating driver "docker" against &{Name:old-k8s-version-633693 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-633693 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountStr
ing:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:06:20.621586 1338826 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0328 22:06:20.622187 1338826 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0328 22:06:20.696601 1338826 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:59 SystemTime:2024-03-28 22:06:20.679942522 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0328 22:06:20.696929 1338826 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0328 22:06:20.696986 1338826 cni.go:84] Creating CNI manager for ""
I0328 22:06:20.696995 1338826 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0328 22:06:20.697035 1338826 start.go:340] cluster config:
{Name:old-k8s-version-633693 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-633693 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP
: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:06:20.700008 1338826 out.go:177] * Starting "old-k8s-version-633693" primary control-plane node in "old-k8s-version-633693" cluster
I0328 22:06:20.702382 1338826 cache.go:121] Beginning downloading kic base image for docker with crio
I0328 22:06:20.704472 1338826 out.go:177] * Pulling base image v0.0.43-1711559786-18485 ...
I0328 22:06:20.706697 1338826 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0328 22:06:20.706747 1338826 preload.go:147] Found local preload: /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4
I0328 22:06:20.706769 1338826 cache.go:56] Caching tarball of preloaded images
I0328 22:06:20.706850 1338826 preload.go:173] Found /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4 in cache, skipping download
I0328 22:06:20.706858 1338826 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0328 22:06:20.706964 1338826 profile.go:143] Saving config to /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/config.json ...
I0328 22:06:20.707195 1338826 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 in local docker daemon
I0328 22:06:20.725508 1338826 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 in local docker daemon, skipping pull
I0328 22:06:20.725531 1338826 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 exists in daemon, skipping load
I0328 22:06:20.725549 1338826 cache.go:194] Successfully downloaded all kic artifacts
I0328 22:06:20.725578 1338826 start.go:360] acquireMachinesLock for old-k8s-version-633693: {Name:mk3fbfb43af77d465332b66bdb01d7d737487536 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:06:20.725638 1338826 start.go:364] duration metric: took 39.623µs to acquireMachinesLock for "old-k8s-version-633693"
I0328 22:06:20.725656 1338826 start.go:96] Skipping create...Using existing machine configuration
I0328 22:06:20.725662 1338826 fix.go:54] fixHost starting:
I0328 22:06:20.725929 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:20.742622 1338826 fix.go:112] recreateIfNeeded on old-k8s-version-633693: state=Stopped err=<nil>
W0328 22:06:20.742658 1338826 fix.go:138] unexpected machine state, will restart: <nil>
I0328 22:06:20.745084 1338826 out.go:177] * Restarting existing docker container for "old-k8s-version-633693" ...
I0328 22:06:20.746945 1338826 cli_runner.go:164] Run: docker start old-k8s-version-633693
I0328 22:06:21.025411 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:21.047957 1338826 kic.go:430] container "old-k8s-version-633693" state is running.
I0328 22:06:21.048372 1338826 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-633693
I0328 22:06:21.075027 1338826 profile.go:143] Saving config to /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/config.json ...
I0328 22:06:21.075277 1338826 machine.go:94] provisionDockerMachine start ...
I0328 22:06:21.075338 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:21.102941 1338826 main.go:141] libmachine: Using SSH client type: native
I0328 22:06:21.103220 1338826 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34554 <nil> <nil>}
I0328 22:06:21.103231 1338826 main.go:141] libmachine: About to run SSH command:
hostname
I0328 22:06:21.103934 1338826 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0328 22:06:24.255358 1338826 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-633693
I0328 22:06:24.255385 1338826 ubuntu.go:169] provisioning hostname "old-k8s-version-633693"
I0328 22:06:24.255475 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:24.272004 1338826 main.go:141] libmachine: Using SSH client type: native
I0328 22:06:24.272399 1338826 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34554 <nil> <nil>}
I0328 22:06:24.272419 1338826 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-633693 && echo "old-k8s-version-633693" | sudo tee /etc/hostname
I0328 22:06:24.432155 1338826 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-633693
I0328 22:06:24.432334 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:24.457681 1338826 main.go:141] libmachine: Using SSH client type: native
I0328 22:06:24.457918 1338826 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34554 <nil> <nil>}
I0328 22:06:24.457936 1338826 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-633693' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-633693/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-633693' | sudo tee -a /etc/hosts;
fi
fi
I0328 22:06:24.600215 1338826 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0328 22:06:24.600295 1338826 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/17877-1145955/.minikube CaCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/17877-1145955/.minikube}
I0328 22:06:24.600341 1338826 ubuntu.go:177] setting up certificates
I0328 22:06:24.600381 1338826 provision.go:84] configureAuth start
I0328 22:06:24.600491 1338826 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-633693
I0328 22:06:24.614561 1338826 provision.go:143] copyHostCerts
I0328 22:06:24.614631 1338826 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem, removing ...
I0328 22:06:24.614652 1338826 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem
I0328 22:06:24.614743 1338826 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem (1123 bytes)
I0328 22:06:24.614841 1338826 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem, removing ...
I0328 22:06:24.614850 1338826 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem
I0328 22:06:24.614878 1338826 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem (1679 bytes)
I0328 22:06:24.614941 1338826 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem, removing ...
I0328 22:06:24.614956 1338826 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem
I0328 22:06:24.614982 1338826 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem (1082 bytes)
I0328 22:06:24.615033 1338826 provision.go:117] generating server cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-633693 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-633693]
I0328 22:06:25.066127 1338826 provision.go:177] copyRemoteCerts
I0328 22:06:25.066207 1338826 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0328 22:06:25.066254 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:25.084182 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:25.195157 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0328 22:06:25.228220 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0328 22:06:25.258013 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0328 22:06:25.309022 1338826 provision.go:87] duration metric: took 708.608266ms to configureAuth
I0328 22:06:25.309052 1338826 ubuntu.go:193] setting minikube options for container-runtime
I0328 22:06:25.309258 1338826 config.go:182] Loaded profile config "old-k8s-version-633693": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0328 22:06:25.309362 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:25.332398 1338826 main.go:141] libmachine: Using SSH client type: native
I0328 22:06:25.332701 1338826 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34554 <nil> <nil>}
I0328 22:06:25.332722 1338826 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0328 22:06:25.792200 1338826 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0328 22:06:25.792234 1338826 machine.go:97] duration metric: took 4.716947343s to provisionDockerMachine
I0328 22:06:25.792246 1338826 start.go:293] postStartSetup for "old-k8s-version-633693" (driver="docker")
I0328 22:06:25.792258 1338826 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0328 22:06:25.792329 1338826 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0328 22:06:25.792375 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:25.813520 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:25.918481 1338826 ssh_runner.go:195] Run: cat /etc/os-release
I0328 22:06:25.922423 1338826 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0328 22:06:25.922457 1338826 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0328 22:06:25.922467 1338826 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0328 22:06:25.922474 1338826 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0328 22:06:25.922485 1338826 filesync.go:126] Scanning /home/jenkins/minikube-integration/17877-1145955/.minikube/addons for local assets ...
I0328 22:06:25.922541 1338826 filesync.go:126] Scanning /home/jenkins/minikube-integration/17877-1145955/.minikube/files for local assets ...
I0328 22:06:25.922625 1338826 filesync.go:149] local asset: /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem -> 11513632.pem in /etc/ssl/certs
I0328 22:06:25.922732 1338826 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0328 22:06:25.932532 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem --> /etc/ssl/certs/11513632.pem (1708 bytes)
I0328 22:06:25.959057 1338826 start.go:296] duration metric: took 166.794749ms for postStartSetup
I0328 22:06:25.959225 1338826 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0328 22:06:25.959289 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:25.974030 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:26.073166 1338826 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0328 22:06:26.078612 1338826 fix.go:56] duration metric: took 5.352940797s for fixHost
I0328 22:06:26.078640 1338826 start.go:83] releasing machines lock for "old-k8s-version-633693", held for 5.352993925s
I0328 22:06:26.078720 1338826 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-633693
I0328 22:06:26.103482 1338826 ssh_runner.go:195] Run: cat /version.json
I0328 22:06:26.103540 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:26.103780 1338826 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0328 22:06:26.103830 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:26.133765 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:26.137934 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:26.236104 1338826 ssh_runner.go:195] Run: systemctl --version
I0328 22:06:26.357280 1338826 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0328 22:06:26.512404 1338826 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0328 22:06:26.517443 1338826 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0328 22:06:26.527716 1338826 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0328 22:06:26.527806 1338826 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0328 22:06:26.537233 1338826 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0328 22:06:26.537261 1338826 start.go:494] detecting cgroup driver to use...
I0328 22:06:26.537293 1338826 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0328 22:06:26.537356 1338826 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0328 22:06:26.552157 1338826 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0328 22:06:26.565388 1338826 docker.go:217] disabling cri-docker service (if available) ...
I0328 22:06:26.565456 1338826 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0328 22:06:26.579489 1338826 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0328 22:06:26.592706 1338826 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0328 22:06:26.707888 1338826 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0328 22:06:26.819405 1338826 docker.go:233] disabling docker service ...
I0328 22:06:26.819531 1338826 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0328 22:06:26.834077 1338826 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0328 22:06:26.846469 1338826 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0328 22:06:26.955411 1338826 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0328 22:06:27.107077 1338826 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0328 22:06:27.124814 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0328 22:06:27.153638 1338826 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0328 22:06:27.153767 1338826 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:06:27.164067 1338826 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0328 22:06:27.164212 1338826 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:06:27.174649 1338826 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:06:27.185299 1338826 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:06:27.195699 1338826 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0328 22:06:27.205515 1338826 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0328 22:06:27.214861 1338826 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0328 22:06:27.223920 1338826 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:06:27.379449 1338826 ssh_runner.go:195] Run: sudo systemctl restart crio
I0328 22:06:28.463286 1338826 ssh_runner.go:235] Completed: sudo systemctl restart crio: (1.08374705s)
I0328 22:06:28.463376 1338826 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0328 22:06:28.463468 1338826 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0328 22:06:28.468081 1338826 start.go:562] Will wait 60s for crictl version
I0328 22:06:28.468237 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:06:28.473254 1338826 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0328 22:06:28.527549 1338826 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0328 22:06:28.527703 1338826 ssh_runner.go:195] Run: crio --version
I0328 22:06:28.570021 1338826 ssh_runner.go:195] Run: crio --version
I0328 22:06:28.661897 1338826 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
I0328 22:06:28.664443 1338826 cli_runner.go:164] Run: docker network inspect old-k8s-version-633693 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0328 22:06:28.695642 1338826 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0328 22:06:28.699406 1338826 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 22:06:28.709557 1338826 kubeadm.go:877] updating cluster {Name:old-k8s-version-633693 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-633693 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0328 22:06:28.709689 1338826 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0328 22:06:28.709751 1338826 ssh_runner.go:195] Run: sudo crictl images --output json
I0328 22:06:28.769089 1338826 crio.go:514] all images are preloaded for cri-o runtime.
I0328 22:06:28.769114 1338826 crio.go:433] Images already preloaded, skipping extraction
I0328 22:06:28.769199 1338826 ssh_runner.go:195] Run: sudo crictl images --output json
I0328 22:06:28.814249 1338826 crio.go:514] all images are preloaded for cri-o runtime.
I0328 22:06:28.814274 1338826 cache_images.go:84] Images are preloaded, skipping loading
I0328 22:06:28.814283 1338826 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.20.0 crio true true} ...
I0328 22:06:28.814405 1338826 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --enforce-node-allocatable= --hostname-override=old-k8s-version-633693 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-633693 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0328 22:06:28.814494 1338826 ssh_runner.go:195] Run: crio config
I0328 22:06:28.934128 1338826 cni.go:84] Creating CNI manager for ""
I0328 22:06:28.934152 1338826 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0328 22:06:28.934164 1338826 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0328 22:06:28.934205 1338826 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-633693 NodeName:old-k8s-version-633693 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0328 22:06:28.934380 1338826 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-633693"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0328 22:06:28.934468 1338826 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0328 22:06:28.943478 1338826 binaries.go:44] Found k8s binaries, skipping transfer
I0328 22:06:28.943568 1338826 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0328 22:06:28.951834 1338826 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (480 bytes)
I0328 22:06:28.971640 1338826 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0328 22:06:28.990717 1338826 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2117 bytes)
I0328 22:06:29.013678 1338826 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0328 22:06:29.017659 1338826 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 22:06:29.029010 1338826 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:06:29.162983 1338826 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0328 22:06:29.181472 1338826 certs.go:68] Setting up /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693 for IP: 192.168.76.2
I0328 22:06:29.181495 1338826 certs.go:194] generating shared ca certs ...
I0328 22:06:29.181511 1338826 certs.go:226] acquiring lock for ca certs: {Name:mk1e4b3d6020f96643d0b806687ddcafb6824b0b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:06:29.181723 1338826 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.key
I0328 22:06:29.181800 1338826 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.key
I0328 22:06:29.181815 1338826 certs.go:256] generating profile certs ...
I0328 22:06:29.181923 1338826 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/client.key
I0328 22:06:29.182007 1338826 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/apiserver.key.e34860cc
I0328 22:06:29.182070 1338826 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/proxy-client.key
I0328 22:06:29.182202 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363.pem (1338 bytes)
W0328 22:06:29.182252 1338826 certs.go:480] ignoring /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363_empty.pem, impossibly tiny 0 bytes
I0328 22:06:29.182267 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem (1679 bytes)
I0328 22:06:29.182297 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem (1082 bytes)
I0328 22:06:29.182355 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem (1123 bytes)
I0328 22:06:29.182407 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem (1679 bytes)
I0328 22:06:29.182474 1338826 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem (1708 bytes)
I0328 22:06:29.183123 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0328 22:06:29.289591 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0328 22:06:29.359403 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0328 22:06:29.385940 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0328 22:06:29.411504 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0328 22:06:29.437705 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0328 22:06:29.463799 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0328 22:06:29.489953 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/old-k8s-version-633693/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0328 22:06:29.516028 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0328 22:06:29.542283 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363.pem --> /usr/share/ca-certificates/1151363.pem (1338 bytes)
I0328 22:06:29.568454 1338826 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem --> /usr/share/ca-certificates/11513632.pem (1708 bytes)
I0328 22:06:29.593883 1338826 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0328 22:06:29.613283 1338826 ssh_runner.go:195] Run: openssl version
I0328 22:06:29.618984 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0328 22:06:29.629274 1338826 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0328 22:06:29.633033 1338826 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 28 21:12 /usr/share/ca-certificates/minikubeCA.pem
I0328 22:06:29.633128 1338826 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0328 22:06:29.640130 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0328 22:06:29.649644 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1151363.pem && ln -fs /usr/share/ca-certificates/1151363.pem /etc/ssl/certs/1151363.pem"
I0328 22:06:29.659725 1338826 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1151363.pem
I0328 22:06:29.663581 1338826 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 28 21:20 /usr/share/ca-certificates/1151363.pem
I0328 22:06:29.663692 1338826 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1151363.pem
I0328 22:06:29.671035 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1151363.pem /etc/ssl/certs/51391683.0"
I0328 22:06:29.680921 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/11513632.pem && ln -fs /usr/share/ca-certificates/11513632.pem /etc/ssl/certs/11513632.pem"
I0328 22:06:29.690830 1338826 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/11513632.pem
I0328 22:06:29.694795 1338826 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 28 21:20 /usr/share/ca-certificates/11513632.pem
I0328 22:06:29.694937 1338826 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/11513632.pem
I0328 22:06:29.702366 1338826 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/11513632.pem /etc/ssl/certs/3ec20f2e.0"
I0328 22:06:29.712274 1338826 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0328 22:06:29.716389 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0328 22:06:29.723623 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0328 22:06:29.731033 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0328 22:06:29.738151 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0328 22:06:29.745420 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0328 22:06:29.752519 1338826 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0328 22:06:29.759632 1338826 kubeadm.go:391] StartCluster: {Name:old-k8s-version-633693 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-633693 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/m
inikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:06:29.759780 1338826 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0328 22:06:29.759887 1338826 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0328 22:06:29.834024 1338826 cri.go:89] found id: ""
I0328 22:06:29.834146 1338826 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0328 22:06:29.844585 1338826 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0328 22:06:29.844656 1338826 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0328 22:06:29.844674 1338826 kubeadm.go:587] restartPrimaryControlPlane start ...
I0328 22:06:29.844768 1338826 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0328 22:06:29.854686 1338826 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0328 22:06:29.855186 1338826 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-633693" does not appear in /home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:06:29.855367 1338826 kubeconfig.go:62] /home/jenkins/minikube-integration/17877-1145955/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-633693" cluster setting kubeconfig missing "old-k8s-version-633693" context setting]
I0328 22:06:29.855776 1338826 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17877-1145955/kubeconfig: {Name:mk01de9100d65131f49674a0d1051891ca674cd9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:06:29.857241 1338826 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0328 22:06:29.866745 1338826 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.76.2
I0328 22:06:29.866817 1338826 kubeadm.go:591] duration metric: took 22.125235ms to restartPrimaryControlPlane
I0328 22:06:29.866841 1338826 kubeadm.go:393] duration metric: took 107.224232ms to StartCluster
I0328 22:06:29.866899 1338826 settings.go:142] acquiring lock: {Name:mka22e5d6cd66b2677ac3cce373c1a6e13c189c9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:06:29.866973 1338826 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:06:29.867680 1338826 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17877-1145955/kubeconfig: {Name:mk01de9100d65131f49674a0d1051891ca674cd9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:06:29.867946 1338826 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0328 22:06:29.870622 1338826 out.go:177] * Verifying Kubernetes components...
I0328 22:06:29.868305 1338826 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0328 22:06:29.868395 1338826 config.go:182] Loaded profile config "old-k8s-version-633693": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0328 22:06:29.872425 1338826 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:06:29.870849 1338826 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-633693"
I0328 22:06:29.872626 1338826 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-633693"
W0328 22:06:29.872637 1338826 addons.go:243] addon storage-provisioner should already be in state true
I0328 22:06:29.872688 1338826 host.go:66] Checking if "old-k8s-version-633693" exists ...
I0328 22:06:29.873103 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:29.870859 1338826 addons.go:69] Setting dashboard=true in profile "old-k8s-version-633693"
I0328 22:06:29.873376 1338826 addons.go:234] Setting addon dashboard=true in "old-k8s-version-633693"
W0328 22:06:29.873399 1338826 addons.go:243] addon dashboard should already be in state true
I0328 22:06:29.873447 1338826 host.go:66] Checking if "old-k8s-version-633693" exists ...
I0328 22:06:29.873880 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:29.870865 1338826 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-633693"
I0328 22:06:29.874339 1338826 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-633693"
I0328 22:06:29.874625 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:29.870888 1338826 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-633693"
I0328 22:06:29.875184 1338826 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-633693"
W0328 22:06:29.875208 1338826 addons.go:243] addon metrics-server should already be in state true
I0328 22:06:29.875258 1338826 host.go:66] Checking if "old-k8s-version-633693" exists ...
I0328 22:06:29.875693 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:29.925218 1338826 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0328 22:06:29.927367 1338826 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0328 22:06:29.932239 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0328 22:06:29.932269 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0328 22:06:29.932334 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:29.928352 1338826 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-633693"
W0328 22:06:29.932551 1338826 addons.go:243] addon default-storageclass should already be in state true
I0328 22:06:29.932581 1338826 host.go:66] Checking if "old-k8s-version-633693" exists ...
I0328 22:06:29.933003 1338826 cli_runner.go:164] Run: docker container inspect old-k8s-version-633693 --format={{.State.Status}}
I0328 22:06:29.957640 1338826 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0328 22:06:29.960027 1338826 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:06:29.960072 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0328 22:06:29.960282 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:29.976133 1338826 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0328 22:06:29.983543 1338826 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0328 22:06:29.983572 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0328 22:06:29.983655 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:30.017743 1338826 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0328 22:06:30.017770 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0328 22:06:30.017866 1338826 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-633693
I0328 22:06:30.024333 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:30.044422 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:30.065458 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:30.084256 1338826 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34554 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/old-k8s-version-633693/id_rsa Username:docker}
I0328 22:06:30.159711 1338826 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0328 22:06:30.198067 1338826 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-633693" to be "Ready" ...
I0328 22:06:30.222912 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0328 22:06:30.222947 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0328 22:06:30.256679 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:06:30.270784 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0328 22:06:30.270830 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0328 22:06:30.311789 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:06:30.340923 1338826 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0328 22:06:30.340955 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0328 22:06:30.347313 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0328 22:06:30.347354 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0328 22:06:30.402844 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0328 22:06:30.402866 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0328 22:06:30.431025 1338826 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0328 22:06:30.431062 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0328 22:06:30.433226 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0328 22:06:30.433249 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0328 22:06:30.457186 1338826 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:30.457212 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0328 22:06:30.494064 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:30.527122 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0328 22:06:30.527199 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0328 22:06:30.532679 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.532773 1338826 retry.go:31] will retry after 132.834426ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.581425 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0328 22:06:30.581463 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
W0328 22:06:30.592873 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.592908 1338826 retry.go:31] will retry after 183.397238ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.633895 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0328 22:06:30.633948 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
W0328 22:06:30.664032 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.664083 1338826 retry.go:31] will retry after 153.380004ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.666213 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:06:30.693271 1338826 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:30.693298 1338826 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0328 22:06:30.714067 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:30.777300 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:06:30.818016 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0328 22:06:30.867707 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.867751 1338826 retry.go:31] will retry after 335.391985ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:30.990044 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:30.990088 1338826 retry.go:31] will retry after 192.126137ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:31.077625 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.077669 1338826 retry.go:31] will retry after 407.221177ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:31.100245 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.100282 1338826 retry.go:31] will retry after 342.986579ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.182438 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:31.204333 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0328 22:06:31.325011 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.325093 1338826 retry.go:31] will retry after 379.697327ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:31.394782 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.394869 1338826 retry.go:31] will retry after 348.480792ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.444229 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:31.485639 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0328 22:06:31.560017 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.560105 1338826 retry.go:31] will retry after 631.077482ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:31.646476 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.646554 1338826 retry.go:31] will retry after 670.51397ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.705236 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:31.743611 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0328 22:06:31.888997 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.889071 1338826 retry.go:31] will retry after 512.688787ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:31.933020 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:31.933093 1338826 retry.go:31] will retry after 554.40876ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.191569 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:32.199286 1338826 node_ready.go:53] error getting node "old-k8s-version-633693": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-633693": dial tcp 192.168.76.2:8443: connect: connection refused
W0328 22:06:32.294118 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.294195 1338826 retry.go:31] will retry after 1.001863178s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.317528 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:06:32.402693 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0328 22:06:32.405024 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.405057 1338826 retry.go:31] will retry after 590.284878ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.488356 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0328 22:06:32.500893 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.500937 1338826 retry.go:31] will retry after 974.32905ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:32.586583 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.586615 1338826 retry.go:31] will retry after 938.141295ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:32.995588 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0328 22:06:33.090153 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.090187 1338826 retry.go:31] will retry after 1.564540504s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.297039 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0328 22:06:33.406185 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.406226 1338826 retry.go:31] will retry after 1.615893783s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.475492 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:33.525063 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0328 22:06:33.575605 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.575654 1338826 retry.go:31] will retry after 1.483884902s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:33.654881 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:33.654927 1338826 retry.go:31] will retry after 2.072986848s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:34.199618 1338826 node_ready.go:53] error getting node "old-k8s-version-633693": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-633693": dial tcp 192.168.76.2:8443: connect: connection refused
I0328 22:06:34.654952 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0328 22:06:34.786056 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:34.786094 1338826 retry.go:31] will retry after 1.950803602s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:35.022341 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:35.059845 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0328 22:06:35.216581 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:35.216621 1338826 retry.go:31] will retry after 2.592482542s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0328 22:06:35.216690 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:35.216716 1338826 retry.go:31] will retry after 2.725049231s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:35.728851 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0328 22:06:35.808082 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:35.808128 1338826 retry.go:31] will retry after 3.101242668s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:36.698678 1338826 node_ready.go:53] error getting node "old-k8s-version-633693": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-633693": dial tcp 192.168.76.2:8443: connect: connection refused
I0328 22:06:36.737961 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0328 22:06:36.817507 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:36.817540 1338826 retry.go:31] will retry after 3.152292884s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:37.809727 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0328 22:06:37.881761 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:37.881794 1338826 retry.go:31] will retry after 2.024843406s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:37.941978 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0328 22:06:38.026274 1338826 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:38.026313 1338826 retry.go:31] will retry after 4.246886334s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0328 22:06:38.910288 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:06:39.907726 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:06:39.970380 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:06:42.274136 1338826 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:06:47.750002 1338826 node_ready.go:49] node "old-k8s-version-633693" has status "Ready":"True"
I0328 22:06:47.750028 1338826 node_ready.go:38] duration metric: took 17.551921336s for node "old-k8s-version-633693" to be "Ready" ...
I0328 22:06:47.750038 1338826 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0328 22:06:48.006334 1338826 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-rq6t8" in "kube-system" namespace to be "Ready" ...
I0328 22:06:48.885781 1338826 pod_ready.go:92] pod "coredns-74ff55c5b-rq6t8" in "kube-system" namespace has status "Ready":"True"
I0328 22:06:48.885855 1338826 pod_ready.go:81] duration metric: took 879.428009ms for pod "coredns-74ff55c5b-rq6t8" in "kube-system" namespace to be "Ready" ...
I0328 22:06:48.885882 1338826 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:06:49.434358 1338826 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (10.524034886s)
I0328 22:06:49.528607 1338826 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (9.55818877s)
I0328 22:06:49.528990 1338826 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (9.621226595s)
I0328 22:06:49.529042 1338826 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-633693"
I0328 22:06:50.113778 1338826 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.839573448s)
I0328 22:06:50.117395 1338826 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-633693 addons enable metrics-server
I0328 22:06:50.119986 1338826 out.go:177] * Enabled addons: storage-provisioner, metrics-server, default-storageclass, dashboard
I0328 22:06:50.122010 1338826 addons.go:505] duration metric: took 20.253710385s for enable addons: enabled=[storage-provisioner metrics-server default-storageclass dashboard]
I0328 22:06:50.911018 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:06:53.406975 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:06:55.892196 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:06:57.895687 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:00.394968 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:02.892992 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:05.392200 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:07.400865 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:09.895160 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:12.393689 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:14.891542 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:16.894523 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:18.911624 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:21.405815 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:23.893456 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:25.897410 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:28.392082 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:30.393127 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:32.891448 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:34.895445 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:37.391764 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:39.392617 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:41.891599 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:43.891771 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:46.391837 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:48.393916 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:50.891892 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:52.893756 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:55.391654 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:57.393698 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:59.891612 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:02.393850 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:04.892201 1338826 pod_ready.go:92] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:04.892227 1338826 pod_ready.go:81] duration metric: took 1m16.006325351s for pod "etcd-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.892259 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.897466 1338826 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:04.897491 1338826 pod_ready.go:81] duration metric: took 5.218747ms for pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.897503 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:06.904157 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:09.403663 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:11.405298 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:13.904187 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:16.403711 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:17.909294 1338826 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.909370 1338826 pod_ready.go:81] duration metric: took 13.011857211s for pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.909397 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-9vs8r" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.918637 1338826 pod_ready.go:92] pod "kube-proxy-9vs8r" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.918672 1338826 pod_ready.go:81] duration metric: took 9.258655ms for pod "kube-proxy-9vs8r" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.918685 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.927105 1338826 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.927130 1338826 pod_ready.go:81] duration metric: took 8.437573ms for pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.927142 1338826 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace to be "Ready" ...
I0328 22:08:19.942294 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:22.434712 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:24.435112 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:26.933676 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:28.935233 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:31.434711 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:33.932938 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:35.933182 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:37.933531 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:40.434985 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:42.932957 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:44.933746 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:47.432638 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:49.433249 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:51.433961 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:53.434580 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:55.932722 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:57.933867 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:00.434560 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:02.932551 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:04.932913 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:07.433550 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:09.433637 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:11.933587 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:14.434340 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:16.434764 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:18.932404 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:20.933066 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:22.938118 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:25.432907 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:27.433591 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:29.933701 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:32.433501 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:34.932827 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:36.934724 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:39.433686 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:41.434345 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:43.434422 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:45.932966 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:47.933007 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:49.933857 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:52.433046 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:54.434259 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:56.933632 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:58.933976 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:01.434400 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:03.434444 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:05.932668 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:07.934860 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:09.935369 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:12.434151 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:14.933434 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:16.934278 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:19.434252 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:21.434530 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:23.933010 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:26.433027 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:28.433948 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:30.933738 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:32.933910 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:35.432932 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:37.434118 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:39.441245 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:41.932963 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:43.933931 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:46.434034 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:48.933457 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:51.432895 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:53.434727 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:55.440622 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:57.933693 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:00.435477 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:02.932933 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:04.933833 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:07.433346 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:09.433437 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:11.933095 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:13.933603 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:16.433215 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:18.933087 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:20.933409 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:23.434532 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:25.434708 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:27.932941 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:29.933456 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:31.933872 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:34.433180 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:36.433308 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:38.433890 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:40.434164 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:42.932888 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:44.932980 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:46.933414 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:49.432985 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:51.433663 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:53.434707 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:55.933135 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:57.933821 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:00.434788 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:02.933354 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:04.934434 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:07.433060 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:09.433317 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:11.433605 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:13.437226 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:15.933636 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:17.933340 1338826 pod_ready.go:81] duration metric: took 4m0.006183924s for pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace to be "Ready" ...
E0328 22:12:17.933366 1338826 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0328 22:12:17.933376 1338826 pod_ready.go:38] duration metric: took 5m30.183327024s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0328 22:12:17.933390 1338826 api_server.go:52] waiting for apiserver process to appear ...
I0328 22:12:17.933418 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:17.933480 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:17.975520 1338826 cri.go:89] found id: "aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:17.975543 1338826 cri.go:89] found id: ""
I0328 22:12:17.975551 1338826 logs.go:276] 1 containers: [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5]
I0328 22:12:17.975605 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:17.978975 1338826 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:17.979046 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:18.021754 1338826 cri.go:89] found id: "54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:18.021779 1338826 cri.go:89] found id: ""
I0328 22:12:18.021787 1338826 logs.go:276] 1 containers: [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263]
I0328 22:12:18.021845 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.025748 1338826 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:18.025846 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:18.069778 1338826 cri.go:89] found id: "9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:18.069809 1338826 cri.go:89] found id: ""
I0328 22:12:18.069819 1338826 logs.go:276] 1 containers: [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962]
I0328 22:12:18.069881 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.073970 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:18.074054 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:18.116221 1338826 cri.go:89] found id: "0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:18.116246 1338826 cri.go:89] found id: ""
I0328 22:12:18.116254 1338826 logs.go:276] 1 containers: [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b]
I0328 22:12:18.116315 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.120360 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:18.120438 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:18.160651 1338826 cri.go:89] found id: "99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:18.160672 1338826 cri.go:89] found id: ""
I0328 22:12:18.160681 1338826 logs.go:276] 1 containers: [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95]
I0328 22:12:18.160741 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.164797 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:18.164870 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:18.208058 1338826 cri.go:89] found id: "accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:18.208165 1338826 cri.go:89] found id: ""
I0328 22:12:18.208192 1338826 logs.go:276] 1 containers: [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591]
I0328 22:12:18.208295 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.212186 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:18.212308 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:18.257286 1338826 cri.go:89] found id: "f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:18.257353 1338826 cri.go:89] found id: ""
I0328 22:12:18.257367 1338826 logs.go:276] 1 containers: [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b]
I0328 22:12:18.257425 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.263454 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:18.263535 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:18.301918 1338826 cri.go:89] found id: "dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:18.301941 1338826 cri.go:89] found id: ""
I0328 22:12:18.301949 1338826 logs.go:276] 1 containers: [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d]
I0328 22:12:18.302004 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.305551 1338826 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:18.305626 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:18.346240 1338826 cri.go:89] found id: "dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:18.346264 1338826 cri.go:89] found id: "de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:18.346269 1338826 cri.go:89] found id: ""
I0328 22:12:18.346277 1338826 logs.go:276] 2 containers: [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0]
I0328 22:12:18.346333 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.349950 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.353205 1338826 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:18.353227 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:18.405413 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826315 742 reflector.go:138] object-"kube-system"/"storage-provisioner-token-htmqq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-htmqq" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.405657 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826559 742 reflector.go:138] object-"default"/"default-token-skqvg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-skqvg" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.405882 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826733 742 reflector.go:138] object-"kube-system"/"metrics-server-token-qkmwr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-qkmwr" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406094 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829336 742 reflector.go:138] object-"kube-system"/"coredns-token-zjvsj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zjvsj" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406306 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829576 742 reflector.go:138] object-"kube-system"/"kindnet-token-g4wkb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g4wkb" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406507 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829784 742 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406719 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829969 742 reflector.go:138] object-"kube-system"/"kube-proxy-token-l4tct": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l4tct" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.830130 742 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.415925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.527663 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.416174 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.814044 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.418221 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:04 old-k8s-version-633693 kubelet[742]: E0328 22:07:04.699349 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.418545 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:07 old-k8s-version-633693 kubelet[742]: E0328 22:07:07.707146 742 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-kddb6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-kddb6" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.421763 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:15 old-k8s-version-633693 kubelet[742]: E0328 22:07:15.934579 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.422221 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:16 old-k8s-version-633693 kubelet[742]: E0328 22:07:16.937591 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.422406 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:19 old-k8s-version-633693 kubelet[742]: E0328 22:07:19.733048 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.422861 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:22 old-k8s-version-633693 kubelet[742]: E0328 22:07:22.808225 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.424949 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:34 old-k8s-version-633693 kubelet[742]: E0328 22:07:34.698485 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.425546 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:36 old-k8s-version-633693 kubelet[742]: E0328 22:07:36.977259 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.425874 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:42 old-k8s-version-633693 kubelet[742]: E0328 22:07:42.808747 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.426059 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:48 old-k8s-version-633693 kubelet[742]: E0328 22:07:48.688107 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.426384 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:54 old-k8s-version-633693 kubelet[742]: E0328 22:07:54.687538 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.426568 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:02 old-k8s-version-633693 kubelet[742]: E0328 22:08:02.688287 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.427151 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:09 old-k8s-version-633693 kubelet[742]: E0328 22:08:09.025390 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.427477 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:12 old-k8s-version-633693 kubelet[742]: E0328 22:08:12.808271 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.427660 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:14 old-k8s-version-633693 kubelet[742]: E0328 22:08:14.687911 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.429727 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:25 old-k8s-version-633693 kubelet[742]: E0328 22:08:25.704436 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.430054 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:27 old-k8s-version-633693 kubelet[742]: E0328 22:08:27.687797 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.430237 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:39 old-k8s-version-633693 kubelet[742]: E0328 22:08:39.688581 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.430562 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:41 old-k8s-version-633693 kubelet[742]: E0328 22:08:41.687550 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.430787 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:51 old-k8s-version-633693 kubelet[742]: E0328 22:08:51.689572 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.431378 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:55 old-k8s-version-633693 kubelet[742]: E0328 22:08:55.094326 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.431561 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.687902 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.431886 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.808415 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432226 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:13 old-k8s-version-633693 kubelet[742]: E0328 22:09:13.689276 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432413 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:14 old-k8s-version-633693 kubelet[742]: E0328 22:09:14.688165 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.432741 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:24 old-k8s-version-633693 kubelet[742]: E0328 22:09:24.687486 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432928 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:29 old-k8s-version-633693 kubelet[742]: E0328 22:09:29.688477 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.433254 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:35 old-k8s-version-633693 kubelet[742]: E0328 22:09:35.687479 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.433952 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:40 old-k8s-version-633693 kubelet[742]: E0328 22:09:40.688238 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.434277 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:48 old-k8s-version-633693 kubelet[742]: E0328 22:09:48.687433 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.436363 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:51 old-k8s-version-633693 kubelet[742]: E0328 22:09:51.697009 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.436691 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:02 old-k8s-version-633693 kubelet[742]: E0328 22:10:02.687437 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.436876 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:03 old-k8s-version-633693 kubelet[742]: E0328 22:10:03.688247 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.437462 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.214323 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.437646 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.688145 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.437971 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:22 old-k8s-version-633693 kubelet[742]: E0328 22:10:22.808176 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.438155 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:29 old-k8s-version-633693 kubelet[742]: E0328 22:10:29.688127 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.438480 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:33 old-k8s-version-633693 kubelet[742]: E0328 22:10:33.687521 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.438666 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:40 old-k8s-version-633693 kubelet[742]: E0328 22:10:40.688295 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.438993 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:46 old-k8s-version-633693 kubelet[742]: E0328 22:10:46.687410 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.439177 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:52 old-k8s-version-633693 kubelet[742]: E0328 22:10:52.688204 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.439502 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:57 old-k8s-version-633693 kubelet[742]: E0328 22:10:57.687778 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.439686 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:07 old-k8s-version-633693 kubelet[742]: E0328 22:11:07.688569 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.440010 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: E0328 22:11:10.687505 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.440203 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:21 old-k8s-version-633693 kubelet[742]: E0328 22:11:21.688528 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.440529 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: E0328 22:11:25.687833 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.440714 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:36 old-k8s-version-633693 kubelet[742]: E0328 22:11:36.687961 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.441289 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: E0328 22:11:39.687565 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.441475 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:51 old-k8s-version-633693 kubelet[742]: E0328 22:11:51.689147 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.441800 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.441983 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.442307 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.442634 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.442816 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:18.442826 1338826 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:18.442840 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:18.610600 1338826 logs.go:123] Gathering logs for coredns [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962] ...
I0328 22:12:18.610629 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:18.651719 1338826 logs.go:123] Gathering logs for storage-provisioner [de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0] ...
I0328 22:12:18.651747 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:18.692899 1338826 logs.go:123] Gathering logs for container status ...
I0328 22:12:18.692993 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:18.756410 1338826 logs.go:123] Gathering logs for kube-apiserver [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5] ...
I0328 22:12:18.756441 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:18.825430 1338826 logs.go:123] Gathering logs for kube-controller-manager [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591] ...
I0328 22:12:18.825468 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:18.915204 1338826 logs.go:123] Gathering logs for kindnet [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b] ...
I0328 22:12:18.915239 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:18.970831 1338826 logs.go:123] Gathering logs for kube-proxy [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95] ...
I0328 22:12:18.970914 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:19.019338 1338826 logs.go:123] Gathering logs for storage-provisioner [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723] ...
I0328 22:12:19.019365 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:19.064283 1338826 logs.go:123] Gathering logs for kubernetes-dashboard [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d] ...
I0328 22:12:19.064313 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:19.104471 1338826 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:19.104499 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:19.184921 1338826 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:19.184961 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:19.203985 1338826 logs.go:123] Gathering logs for etcd [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263] ...
I0328 22:12:19.204137 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:19.250334 1338826 logs.go:123] Gathering logs for kube-scheduler [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b] ...
I0328 22:12:19.250368 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:19.293923 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:19.293952 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:19.294030 1338826 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0328 22:12:19.294045 1338826 out.go:239] Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294087 1338826 out.go:239] Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:19.294098 1338826 out.go:239] Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294105 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294114 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:19.294121 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:19.294128 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:29.295105 1338826 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0328 22:12:29.306579 1338826 api_server.go:72] duration metric: took 5m59.438575826s to wait for apiserver process to appear ...
I0328 22:12:29.306606 1338826 api_server.go:88] waiting for apiserver healthz status ...
I0328 22:12:29.306641 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:29.306705 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:29.345775 1338826 cri.go:89] found id: "aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:29.345797 1338826 cri.go:89] found id: ""
I0328 22:12:29.345805 1338826 logs.go:276] 1 containers: [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5]
I0328 22:12:29.345862 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.349735 1338826 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:29.349812 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:29.389469 1338826 cri.go:89] found id: "54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:29.389500 1338826 cri.go:89] found id: ""
I0328 22:12:29.389510 1338826 logs.go:276] 1 containers: [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263]
I0328 22:12:29.389587 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.393252 1338826 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:29.393335 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:29.435046 1338826 cri.go:89] found id: "9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:29.435068 1338826 cri.go:89] found id: ""
I0328 22:12:29.435076 1338826 logs.go:276] 1 containers: [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962]
I0328 22:12:29.435134 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.439053 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:29.439135 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:29.478063 1338826 cri.go:89] found id: "0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:29.478084 1338826 cri.go:89] found id: ""
I0328 22:12:29.478092 1338826 logs.go:276] 1 containers: [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b]
I0328 22:12:29.478148 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.481825 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:29.481896 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:29.524721 1338826 cri.go:89] found id: "99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:29.524784 1338826 cri.go:89] found id: ""
I0328 22:12:29.524806 1338826 logs.go:276] 1 containers: [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95]
I0328 22:12:29.524879 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.528593 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:29.528672 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:29.570444 1338826 cri.go:89] found id: "accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:29.570465 1338826 cri.go:89] found id: ""
I0328 22:12:29.570472 1338826 logs.go:276] 1 containers: [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591]
I0328 22:12:29.570548 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.574273 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:29.574347 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:29.615918 1338826 cri.go:89] found id: "f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:29.615941 1338826 cri.go:89] found id: ""
I0328 22:12:29.615948 1338826 logs.go:276] 1 containers: [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b]
I0328 22:12:29.616002 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.619941 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:29.620011 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:29.656396 1338826 cri.go:89] found id: "dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:29.656421 1338826 cri.go:89] found id: ""
I0328 22:12:29.656429 1338826 logs.go:276] 1 containers: [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d]
I0328 22:12:29.656484 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.660191 1338826 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:29.660260 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:29.697904 1338826 cri.go:89] found id: "dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:29.697933 1338826 cri.go:89] found id: "de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:29.697938 1338826 cri.go:89] found id: ""
I0328 22:12:29.697945 1338826 logs.go:276] 2 containers: [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0]
I0328 22:12:29.698002 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.701729 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.705384 1338826 logs.go:123] Gathering logs for kube-proxy [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95] ...
I0328 22:12:29.705454 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:29.749991 1338826 logs.go:123] Gathering logs for storage-provisioner [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723] ...
I0328 22:12:29.750020 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:29.795978 1338826 logs.go:123] Gathering logs for storage-provisioner [de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0] ...
I0328 22:12:29.796009 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:29.859914 1338826 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:29.859942 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:29.880830 1338826 logs.go:123] Gathering logs for etcd [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263] ...
I0328 22:12:29.880857 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:29.935958 1338826 logs.go:123] Gathering logs for kubernetes-dashboard [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d] ...
I0328 22:12:29.935992 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:29.981822 1338826 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:29.981850 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:30.152830 1338826 logs.go:123] Gathering logs for kindnet [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b] ...
I0328 22:12:30.152868 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:30.199134 1338826 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:30.199177 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:30.283626 1338826 logs.go:123] Gathering logs for container status ...
I0328 22:12:30.283664 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:30.353160 1338826 logs.go:123] Gathering logs for kube-scheduler [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b] ...
I0328 22:12:30.353193 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:30.405400 1338826 logs.go:123] Gathering logs for kube-controller-manager [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591] ...
I0328 22:12:30.405432 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:30.506110 1338826 logs.go:123] Gathering logs for coredns [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962] ...
I0328 22:12:30.506147 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:30.564490 1338826 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:30.564518 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:30.618150 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826315 742 reflector.go:138] object-"kube-system"/"storage-provisioner-token-htmqq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-htmqq" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618396 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826559 742 reflector.go:138] object-"default"/"default-token-skqvg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-skqvg" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618624 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826733 742 reflector.go:138] object-"kube-system"/"metrics-server-token-qkmwr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-qkmwr" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618835 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829336 742 reflector.go:138] object-"kube-system"/"coredns-token-zjvsj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zjvsj" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619044 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829576 742 reflector.go:138] object-"kube-system"/"kindnet-token-g4wkb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g4wkb" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619249 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829784 742 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619463 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829969 742 reflector.go:138] object-"kube-system"/"kube-proxy-token-l4tct": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l4tct" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619666 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.830130 742 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.628872 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.527663 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.629070 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.814044 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.631140 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:04 old-k8s-version-633693 kubelet[742]: E0328 22:07:04.699349 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.631470 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:07 old-k8s-version-633693 kubelet[742]: E0328 22:07:07.707146 742 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-kddb6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-kddb6" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.634771 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:15 old-k8s-version-633693 kubelet[742]: E0328 22:07:15.934579 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.635235 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:16 old-k8s-version-633693 kubelet[742]: E0328 22:07:16.937591 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.635423 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:19 old-k8s-version-633693 kubelet[742]: E0328 22:07:19.733048 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.635928 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:22 old-k8s-version-633693 kubelet[742]: E0328 22:07:22.808225 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.638464 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:34 old-k8s-version-633693 kubelet[742]: E0328 22:07:34.698485 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.639072 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:36 old-k8s-version-633693 kubelet[742]: E0328 22:07:36.977259 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.639400 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:42 old-k8s-version-633693 kubelet[742]: E0328 22:07:42.808747 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.639586 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:48 old-k8s-version-633693 kubelet[742]: E0328 22:07:48.688107 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.639926 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:54 old-k8s-version-633693 kubelet[742]: E0328 22:07:54.687538 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.640169 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:02 old-k8s-version-633693 kubelet[742]: E0328 22:08:02.688287 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.640768 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:09 old-k8s-version-633693 kubelet[742]: E0328 22:08:09.025390 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.641095 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:12 old-k8s-version-633693 kubelet[742]: E0328 22:08:12.808271 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.641278 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:14 old-k8s-version-633693 kubelet[742]: E0328 22:08:14.687911 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.643355 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:25 old-k8s-version-633693 kubelet[742]: E0328 22:08:25.704436 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.643711 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:27 old-k8s-version-633693 kubelet[742]: E0328 22:08:27.687797 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.643899 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:39 old-k8s-version-633693 kubelet[742]: E0328 22:08:39.688581 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.644248 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:41 old-k8s-version-633693 kubelet[742]: E0328 22:08:41.687550 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.645266 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:51 old-k8s-version-633693 kubelet[742]: E0328 22:08:51.689572 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.645867 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:55 old-k8s-version-633693 kubelet[742]: E0328 22:08:55.094326 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646051 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.687902 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.646397 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.808415 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646732 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:13 old-k8s-version-633693 kubelet[742]: E0328 22:09:13.689276 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646918 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:14 old-k8s-version-633693 kubelet[742]: E0328 22:09:14.688165 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.647245 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:24 old-k8s-version-633693 kubelet[742]: E0328 22:09:24.687486 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.647451 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:29 old-k8s-version-633693 kubelet[742]: E0328 22:09:29.688477 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.647780 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:35 old-k8s-version-633693 kubelet[742]: E0328 22:09:35.687479 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.648521 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:40 old-k8s-version-633693 kubelet[742]: E0328 22:09:40.688238 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.648899 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:48 old-k8s-version-633693 kubelet[742]: E0328 22:09:48.687433 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.651003 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:51 old-k8s-version-633693 kubelet[742]: E0328 22:09:51.697009 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.651336 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:02 old-k8s-version-633693 kubelet[742]: E0328 22:10:02.687437 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.651523 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:03 old-k8s-version-633693 kubelet[742]: E0328 22:10:03.688247 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.652119 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.214323 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.652326 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.688145 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.652674 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:22 old-k8s-version-633693 kubelet[742]: E0328 22:10:22.808176 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.652864 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:29 old-k8s-version-633693 kubelet[742]: E0328 22:10:29.688127 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.653193 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:33 old-k8s-version-633693 kubelet[742]: E0328 22:10:33.687521 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.653377 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:40 old-k8s-version-633693 kubelet[742]: E0328 22:10:40.688295 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.653719 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:46 old-k8s-version-633693 kubelet[742]: E0328 22:10:46.687410 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.653906 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:52 old-k8s-version-633693 kubelet[742]: E0328 22:10:52.688204 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.654232 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:57 old-k8s-version-633693 kubelet[742]: E0328 22:10:57.687778 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.654416 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:07 old-k8s-version-633693 kubelet[742]: E0328 22:11:07.688569 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.654742 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: E0328 22:11:10.687505 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.654925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:21 old-k8s-version-633693 kubelet[742]: E0328 22:11:21.688528 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.655250 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: E0328 22:11:25.687833 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.655433 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:36 old-k8s-version-633693 kubelet[742]: E0328 22:11:36.687961 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.656006 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: E0328 22:11:39.687565 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.656212 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:51 old-k8s-version-633693 kubelet[742]: E0328 22:11:51.689147 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.656539 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.656739 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.657073 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.657397 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.657583 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:30.657593 1338826 logs.go:123] Gathering logs for kube-apiserver [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5] ...
I0328 22:12:30.657607 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:30.726909 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:30.726942 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:30.727001 1338826 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0328 22:12:30.727013 1338826 out.go:239] Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727021 1338826 out.go:239] Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.727028 1338826 out.go:239] Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727039 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727048 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:30.727062 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:30.727068 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:40.727728 1338826 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0328 22:12:40.736874 1338826 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0328 22:12:40.739297 1338826 out.go:177]
W0328 22:12:40.741210 1338826 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0328 22:12:40.741248 1338826 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0328 22:12:40.741269 1338826 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0328 22:12:40.741277 1338826 out.go:239] *
*
W0328 22:12:40.742279 1338826 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0328 22:12:40.744622 1338826 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-633693 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-633693
helpers_test.go:235: (dbg) docker inspect old-k8s-version-633693:
-- stdout --
[
{
"Id": "fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d",
"Created": "2024-03-28T22:03:42.343937493Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1339095,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-03-28T22:06:21.017617251Z",
"FinishedAt": "2024-03-28T22:06:19.787697003Z"
},
"Image": "sha256:d0f05b8b802e4c4af20a90d686bad8329f07849a8fda1b1d1c1dc3f527691df0",
"ResolvConfPath": "/var/lib/docker/containers/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d/hostname",
"HostsPath": "/var/lib/docker/containers/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d/hosts",
"LogPath": "/var/lib/docker/containers/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d-json.log",
"Name": "/old-k8s-version-633693",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-633693:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-633693",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/b1bb84e177934ca9d1e28a6b32442cc506355d3c0f060bb86b85f2547add112f-init/diff:/var/lib/docker/overlay2/0b3d5a8e71016a91702d908cf9c681d5044b73b0921a0445a612c018590a7fd5/diff",
"MergedDir": "/var/lib/docker/overlay2/b1bb84e177934ca9d1e28a6b32442cc506355d3c0f060bb86b85f2547add112f/merged",
"UpperDir": "/var/lib/docker/overlay2/b1bb84e177934ca9d1e28a6b32442cc506355d3c0f060bb86b85f2547add112f/diff",
"WorkDir": "/var/lib/docker/overlay2/b1bb84e177934ca9d1e28a6b32442cc506355d3c0f060bb86b85f2547add112f/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-633693",
"Source": "/var/lib/docker/volumes/old-k8s-version-633693/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-633693",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-633693",
"name.minikube.sigs.k8s.io": "old-k8s-version-633693",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "bfab9cd129f3896983537e7c769fa9e01972d96004aae7d71c0ac352f89c206e",
"SandboxKey": "/var/run/docker/netns/bfab9cd129f3",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34554"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34553"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34550"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34552"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34551"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-633693": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"NetworkID": "d5924c2fd712aa94e2820ede0138d57a37e8180fad21d56577897748b25f755b",
"EndpointID": "deddd649e6507b7d19733664775711215de826fe39b238bc6bb9aa4d6c779464",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-633693",
"fe636ffa3a97"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-633693 -n old-k8s-version-633693
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-633693 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-633693 logs -n 25: (1.64263639s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
| start | -p force-systemd-env-978460 | force-systemd-env-978460 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | 28 Mar 24 22:02 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| ssh | -p cilium-428181 sudo | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | |
| | containerd config dump | | | | | |
| ssh | -p cilium-428181 sudo | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | |
| | systemctl status crio --all | | | | | |
| | --full --no-pager | | | | | |
| ssh | -p cilium-428181 sudo | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | |
| | systemctl cat crio --no-pager | | | | | |
| ssh | -p cilium-428181 sudo find | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p cilium-428181 sudo crio | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | |
| | config | | | | | |
| delete | -p cilium-428181 | cilium-428181 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | 28 Mar 24 22:02 UTC |
| start | -p cert-expiration-478493 | cert-expiration-478493 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | 28 Mar 24 22:02 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| delete | -p force-systemd-env-978460 | force-systemd-env-978460 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:02 UTC | 28 Mar 24 22:03 UTC |
| start | -p cert-options-487813 | cert-options-487813 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:03 UTC | 28 Mar 24 22:03 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| ssh | cert-options-487813 ssh | cert-options-487813 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:03 UTC | 28 Mar 24 22:03 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-487813 -- sudo | cert-options-487813 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:03 UTC | 28 Mar 24 22:03 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-487813 | cert-options-487813 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:03 UTC | 28 Mar 24 22:03 UTC |
| start | -p old-k8s-version-633693 | old-k8s-version-633693 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:03 UTC | 28 Mar 24 22:05 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-478493 | cert-expiration-478493 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:05 UTC | 28 Mar 24 22:06 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| addons | enable metrics-server -p old-k8s-version-633693 | old-k8s-version-633693 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | 28 Mar 24 22:06 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-633693 | old-k8s-version-633693 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | 28 Mar 24 22:06 UTC |
| | --alsologtostderr -v=3 | | | | | |
| delete | -p cert-expiration-478493 | cert-expiration-478493 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | 28 Mar 24 22:06 UTC |
| start | -p no-preload-363849 --memory=2200 | no-preload-363849 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | 28 Mar 24 22:07 UTC |
| | --alsologtostderr --wait=true | | | | | |
| | --preload=false --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.30.0-beta.0 | | | | | |
| addons | enable dashboard -p old-k8s-version-633693 | old-k8s-version-633693 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | 28 Mar 24 22:06 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-633693 | old-k8s-version-633693 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:06 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-363849 | no-preload-363849 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:07 UTC | 28 Mar 24 22:07 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-363849 | no-preload-363849 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:07 UTC | 28 Mar 24 22:07 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-363849 | no-preload-363849 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:07 UTC | 28 Mar 24 22:07 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-363849 --memory=2200 | no-preload-363849 | jenkins | v1.33.0-beta.0 | 28 Mar 24 22:07 UTC | |
| | --alsologtostderr --wait=true | | | | | |
| | --preload=false --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.30.0-beta.0 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|----------------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/03/28 22:07:48
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.22.1 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0328 22:07:48.620080 1343785 out.go:291] Setting OutFile to fd 1 ...
I0328 22:07:48.620269 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:07:48.620280 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:07:48.620285 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:07:48.620542 1343785 root.go:338] Updating PATH: /home/jenkins/minikube-integration/17877-1145955/.minikube/bin
I0328 22:07:48.620929 1343785 out.go:298] Setting JSON to false
I0328 22:07:48.622023 1343785 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":21019,"bootTime":1711642650,"procs":210,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1056-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0328 22:07:48.622105 1343785 start.go:139] virtualization:
I0328 22:07:48.625003 1343785 out.go:177] * [no-preload-363849] minikube v1.33.0-beta.0 on Ubuntu 20.04 (arm64)
I0328 22:07:48.627230 1343785 out.go:177] - MINIKUBE_LOCATION=17877
I0328 22:07:48.629021 1343785 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0328 22:07:48.627436 1343785 notify.go:220] Checking for updates...
I0328 22:07:48.632649 1343785 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:07:48.634987 1343785 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/17877-1145955/.minikube
I0328 22:07:48.636705 1343785 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0328 22:07:48.638234 1343785 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0328 22:07:48.640431 1343785 config.go:182] Loaded profile config "no-preload-363849": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.0-beta.0
I0328 22:07:48.640972 1343785 driver.go:392] Setting default libvirt URI to qemu:///system
I0328 22:07:48.660681 1343785 docker.go:122] docker version: linux-26.0.0:Docker Engine - Community
I0328 22:07:48.660840 1343785 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0328 22:07:48.748915 1343785 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:52 SystemTime:2024-03-28 22:07:48.737712591 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0328 22:07:48.749025 1343785 docker.go:295] overlay module found
I0328 22:07:48.752687 1343785 out.go:177] * Using the docker driver based on existing profile
I0328 22:07:48.754486 1343785 start.go:297] selected driver: docker
I0328 22:07:48.754503 1343785 start.go:901] validating driver "docker" against &{Name:no-preload-363849 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.0-beta.0 ClusterName:no-preload-363849 Namespace:default APIServerHAVIP: APIS
erverName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.0-beta.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false Mou
ntString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:07:48.754638 1343785 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0328 22:07:48.755308 1343785 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0328 22:07:48.820282 1343785 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:52 SystemTime:2024-03-28 22:07:48.809879867 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1056-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215105536 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:26.0.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:ae07eda36dd25f8a1b98dfbf587313b99c0190bb Expected:ae07eda36dd25f8a1b98dfbf587313b99c0190bb} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.13.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.25.0]] Warnings:<nil>}}
I0328 22:07:48.820646 1343785 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0328 22:07:48.820716 1343785 cni.go:84] Creating CNI manager for ""
I0328 22:07:48.820732 1343785 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0328 22:07:48.820777 1343785 start.go:340] cluster config:
{Name:no-preload-363849 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.0-beta.0 ClusterName:no-preload-363849 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local C
ontainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.0-beta.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker Mo
untIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:07:48.824217 1343785 out.go:177] * Starting "no-preload-363849" primary control-plane node in "no-preload-363849" cluster
I0328 22:07:48.826415 1343785 cache.go:121] Beginning downloading kic base image for docker with crio
I0328 22:07:48.828508 1343785 out.go:177] * Pulling base image v0.0.43-1711559786-18485 ...
I0328 22:07:48.830536 1343785 preload.go:132] Checking if preload exists for k8s version v1.30.0-beta.0 and runtime crio
I0328 22:07:48.830622 1343785 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 in local docker daemon
I0328 22:07:48.830743 1343785 profile.go:143] Saving config to /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/config.json ...
I0328 22:07:48.831051 1343785 cache.go:107] acquiring lock: {Name:mke5e55ffb3fe8eafe321d175dc2a8af9b64484c Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831148 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0328 22:07:48.831161 1343785 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 116.873µs
I0328 22:07:48.831177 1343785 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0328 22:07:48.831194 1343785 cache.go:107] acquiring lock: {Name:mk456b65bf19a94d96c5698786230d6851db1915 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831230 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.0-beta.0 exists
I0328 22:07:48.831239 1343785 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.30.0-beta.0" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.0-beta.0" took 47.212µs
I0328 22:07:48.831246 1343785 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.30.0-beta.0 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.0-beta.0 succeeded
I0328 22:07:48.831269 1343785 cache.go:107] acquiring lock: {Name:mkdb2bc151bf6319b00e147f017e9d57994ce976 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831303 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.0-beta.0 exists
I0328 22:07:48.831312 1343785 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.30.0-beta.0" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.0-beta.0" took 45.415µs
I0328 22:07:48.831320 1343785 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.30.0-beta.0 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.0-beta.0 succeeded
I0328 22:07:48.831335 1343785 cache.go:107] acquiring lock: {Name:mka65a8818f844da73cc580450e190a5bbc15dc2 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831427 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.0-beta.0 exists
I0328 22:07:48.831442 1343785 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.30.0-beta.0" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.0-beta.0" took 107.757µs
I0328 22:07:48.831449 1343785 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.30.0-beta.0 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.0-beta.0 succeeded
I0328 22:07:48.831460 1343785 cache.go:107] acquiring lock: {Name:mk544031203ecf3fa88fef78d19d5ed195123373 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831499 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.0-beta.0 exists
I0328 22:07:48.831511 1343785 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.30.0-beta.0" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.0-beta.0" took 51.889µs
I0328 22:07:48.831517 1343785 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.30.0-beta.0 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.0-beta.0 succeeded
I0328 22:07:48.831527 1343785 cache.go:107] acquiring lock: {Name:mk52a2be272d6f2ede165aed0a5a70989754b6b5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831558 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9 exists
I0328 22:07:48.831563 1343785 cache.go:96] cache image "registry.k8s.io/pause:3.9" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9" took 37.957µs
I0328 22:07:48.831569 1343785 cache.go:80] save to tar file registry.k8s.io/pause:3.9 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9 succeeded
I0328 22:07:48.831578 1343785 cache.go:107] acquiring lock: {Name:mk6abde3ba85d27b6e675012e9e17c35a5ee7868 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831607 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0 exists
I0328 22:07:48.831623 1343785 cache.go:96] cache image "registry.k8s.io/etcd:3.5.12-0" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0" took 38.827µs
I0328 22:07:48.831633 1343785 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.12-0 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0 succeeded
I0328 22:07:48.831642 1343785 cache.go:107] acquiring lock: {Name:mk62ee49752e0d711461ab19e817d9681b21c546 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.831673 1343785 cache.go:115] /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 exists
I0328 22:07:48.831682 1343785 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.1" -> "/home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1" took 41.148µs
I0328 22:07:48.831688 1343785 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.1 -> /home/jenkins/minikube-integration/17877-1145955/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 succeeded
I0328 22:07:48.831693 1343785 cache.go:87] Successfully saved all images to host disk.
I0328 22:07:48.853996 1343785 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 in local docker daemon, skipping pull
I0328 22:07:48.854020 1343785 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 exists in daemon, skipping load
I0328 22:07:48.854059 1343785 cache.go:194] Successfully downloaded all kic artifacts
I0328 22:07:48.854088 1343785 start.go:360] acquireMachinesLock for no-preload-363849: {Name:mk59ba1b235dae08e2a8e3c6c446dcb470763db9 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0328 22:07:48.854149 1343785 start.go:364] duration metric: took 42.699µs to acquireMachinesLock for "no-preload-363849"
I0328 22:07:48.854167 1343785 start.go:96] Skipping create...Using existing machine configuration
I0328 22:07:48.854173 1343785 fix.go:54] fixHost starting:
I0328 22:07:48.854440 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:48.869716 1343785 fix.go:112] recreateIfNeeded on no-preload-363849: state=Stopped err=<nil>
W0328 22:07:48.869749 1343785 fix.go:138] unexpected machine state, will restart: <nil>
I0328 22:07:48.872535 1343785 out.go:177] * Restarting existing docker container for "no-preload-363849" ...
I0328 22:07:46.391837 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:48.393916 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:48.874374 1343785 cli_runner.go:164] Run: docker start no-preload-363849
I0328 22:07:49.139516 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:49.167422 1343785 kic.go:430] container "no-preload-363849" state is running.
I0328 22:07:49.167812 1343785 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-363849
I0328 22:07:49.189933 1343785 profile.go:143] Saving config to /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/config.json ...
I0328 22:07:49.190174 1343785 machine.go:94] provisionDockerMachine start ...
I0328 22:07:49.190236 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:49.208226 1343785 main.go:141] libmachine: Using SSH client type: native
I0328 22:07:49.208550 1343785 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34559 <nil> <nil>}
I0328 22:07:49.208566 1343785 main.go:141] libmachine: About to run SSH command:
hostname
I0328 22:07:49.209265 1343785 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0328 22:07:52.347432 1343785 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-363849
I0328 22:07:52.347471 1343785 ubuntu.go:169] provisioning hostname "no-preload-363849"
I0328 22:07:52.347535 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:52.364691 1343785 main.go:141] libmachine: Using SSH client type: native
I0328 22:07:52.364941 1343785 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34559 <nil> <nil>}
I0328 22:07:52.364958 1343785 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-363849 && echo "no-preload-363849" | sudo tee /etc/hostname
I0328 22:07:52.520303 1343785 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-363849
I0328 22:07:52.520380 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:52.536005 1343785 main.go:141] libmachine: Using SSH client type: native
I0328 22:07:52.536283 1343785 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34559 <nil> <nil>}
I0328 22:07:52.536306 1343785 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-363849' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-363849/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-363849' | sudo tee -a /etc/hosts;
fi
fi
I0328 22:07:52.676205 1343785 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0328 22:07:52.676234 1343785 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/17877-1145955/.minikube CaCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/17877-1145955/.minikube}
I0328 22:07:52.676305 1343785 ubuntu.go:177] setting up certificates
I0328 22:07:52.676317 1343785 provision.go:84] configureAuth start
I0328 22:07:52.676391 1343785 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-363849
I0328 22:07:52.691089 1343785 provision.go:143] copyHostCerts
I0328 22:07:52.691154 1343785 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem, removing ...
I0328 22:07:52.691172 1343785 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem
I0328 22:07:52.691246 1343785 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.pem (1082 bytes)
I0328 22:07:52.691342 1343785 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem, removing ...
I0328 22:07:52.691356 1343785 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem
I0328 22:07:52.691384 1343785 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/cert.pem (1123 bytes)
I0328 22:07:52.691439 1343785 exec_runner.go:144] found /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem, removing ...
I0328 22:07:52.691453 1343785 exec_runner.go:203] rm: /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem
I0328 22:07:52.691481 1343785 exec_runner.go:151] cp: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/17877-1145955/.minikube/key.pem (1679 bytes)
I0328 22:07:52.691531 1343785 provision.go:117] generating server cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem org=jenkins.no-preload-363849 san=[127.0.0.1 192.168.85.2 localhost minikube no-preload-363849]
I0328 22:07:52.985314 1343785 provision.go:177] copyRemoteCerts
I0328 22:07:52.985390 1343785 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0328 22:07:52.985434 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.015581 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:53.117192 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0328 22:07:53.143221 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0328 22:07:53.168940 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0328 22:07:53.194153 1343785 provision.go:87] duration metric: took 517.818912ms to configureAuth
I0328 22:07:53.194189 1343785 ubuntu.go:193] setting minikube options for container-runtime
I0328 22:07:53.194393 1343785 config.go:182] Loaded profile config "no-preload-363849": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.0-beta.0
I0328 22:07:53.194506 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.209637 1343785 main.go:141] libmachine: Using SSH client type: native
I0328 22:07:53.209893 1343785 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e1e00] 0x3e4660 <nil> [] 0s} 127.0.0.1 34559 <nil> <nil>}
I0328 22:07:53.209916 1343785 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %!s(MISSING) "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0328 22:07:53.599961 1343785 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0328 22:07:53.599985 1343785 machine.go:97] duration metric: took 4.409799859s to provisionDockerMachine
I0328 22:07:53.599996 1343785 start.go:293] postStartSetup for "no-preload-363849" (driver="docker")
I0328 22:07:53.600009 1343785 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0328 22:07:53.600082 1343785 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0328 22:07:53.600151 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.623997 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:53.725548 1343785 ssh_runner.go:195] Run: cat /etc/os-release
I0328 22:07:53.729249 1343785 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0328 22:07:53.729292 1343785 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0328 22:07:53.729303 1343785 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0328 22:07:53.729311 1343785 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0328 22:07:53.729322 1343785 filesync.go:126] Scanning /home/jenkins/minikube-integration/17877-1145955/.minikube/addons for local assets ...
I0328 22:07:53.729377 1343785 filesync.go:126] Scanning /home/jenkins/minikube-integration/17877-1145955/.minikube/files for local assets ...
I0328 22:07:53.729465 1343785 filesync.go:149] local asset: /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem -> 11513632.pem in /etc/ssl/certs
I0328 22:07:53.729583 1343785 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0328 22:07:53.738707 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem --> /etc/ssl/certs/11513632.pem (1708 bytes)
I0328 22:07:53.766165 1343785 start.go:296] duration metric: took 166.154042ms for postStartSetup
I0328 22:07:53.766269 1343785 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0328 22:07:53.766322 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.783044 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:53.883323 1343785 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0328 22:07:53.898944 1343785 fix.go:56] duration metric: took 5.044753978s for fixHost
I0328 22:07:53.898972 1343785 start.go:83] releasing machines lock for "no-preload-363849", held for 5.044814647s
I0328 22:07:53.899110 1343785 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-363849
I0328 22:07:53.917052 1343785 ssh_runner.go:195] Run: cat /version.json
I0328 22:07:53.917109 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.917358 1343785 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0328 22:07:53.917414 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:53.950893 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:53.958331 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:54.195328 1343785 ssh_runner.go:195] Run: systemctl --version
I0328 22:07:54.199703 1343785 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0328 22:07:54.341099 1343785 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0328 22:07:54.345438 1343785 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0328 22:07:54.354481 1343785 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0328 22:07:54.354558 1343785 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0328 22:07:54.363602 1343785 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0328 22:07:54.363626 1343785 start.go:494] detecting cgroup driver to use...
I0328 22:07:54.363658 1343785 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0328 22:07:54.363705 1343785 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0328 22:07:54.380246 1343785 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0328 22:07:54.396198 1343785 docker.go:217] disabling cri-docker service (if available) ...
I0328 22:07:54.396263 1343785 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0328 22:07:54.409776 1343785 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0328 22:07:54.421184 1343785 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0328 22:07:54.515471 1343785 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0328 22:07:54.599970 1343785 docker.go:233] disabling docker service ...
I0328 22:07:54.600137 1343785 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0328 22:07:54.613036 1343785 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0328 22:07:54.624686 1343785 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0328 22:07:54.721689 1343785 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0328 22:07:54.820705 1343785 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0328 22:07:54.839746 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0328 22:07:54.857001 1343785 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.9" pause image...
I0328 22:07:54.857071 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.9"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.866856 1343785 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0328 22:07:54.866935 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.876456 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.886446 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.899767 1343785 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0328 22:07:54.908742 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *"net.ipv4.ip_unprivileged_port_start=.*"/d' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.918666 1343785 ssh_runner.go:195] Run: sh -c "sudo grep -q "^ *default_sysctls" /etc/crio/crio.conf.d/02-crio.conf || sudo sed -i '/conmon_cgroup = .*/a default_sysctls = \[\n\]' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.928231 1343785 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^default_sysctls *= *\[|&\n "net.ipv4.ip_unprivileged_port_start=0",|' /etc/crio/crio.conf.d/02-crio.conf"
I0328 22:07:54.938351 1343785 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0328 22:07:54.947353 1343785 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0328 22:07:54.955736 1343785 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:07:55.041779 1343785 ssh_runner.go:195] Run: sudo systemctl restart crio
I0328 22:07:55.160958 1343785 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0328 22:07:55.161032 1343785 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0328 22:07:55.165715 1343785 start.go:562] Will wait 60s for crictl version
I0328 22:07:55.165804 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:07:55.170103 1343785 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0328 22:07:55.211461 1343785 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0328 22:07:55.211565 1343785 ssh_runner.go:195] Run: crio --version
I0328 22:07:55.249768 1343785 ssh_runner.go:195] Run: crio --version
I0328 22:07:55.293152 1343785 out.go:177] * Preparing Kubernetes v1.30.0-beta.0 on CRI-O 1.24.6 ...
I0328 22:07:50.891892 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:52.893756 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:55.391654 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:55.294914 1343785 cli_runner.go:164] Run: docker network inspect no-preload-363849 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0328 22:07:55.308457 1343785 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0328 22:07:55.312143 1343785 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 22:07:55.323009 1343785 kubeadm.go:877] updating cluster {Name:no-preload-363849 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.0-beta.0 ClusterName:no-preload-363849 Namespace:default APIServerHAVIP: APIServerName:minikub
eCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.0-beta.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/je
nkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0328 22:07:55.323153 1343785 preload.go:132] Checking if preload exists for k8s version v1.30.0-beta.0 and runtime crio
I0328 22:07:55.323203 1343785 ssh_runner.go:195] Run: sudo crictl images --output json
I0328 22:07:55.376403 1343785 crio.go:514] all images are preloaded for cri-o runtime.
I0328 22:07:55.376428 1343785 cache_images.go:84] Images are preloaded, skipping loading
I0328 22:07:55.376436 1343785 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.30.0-beta.0 crio true true} ...
I0328 22:07:55.376537 1343785 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.30.0-beta.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=no-preload-363849 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.30.0-beta.0 ClusterName:no-preload-363849 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0328 22:07:55.376623 1343785 ssh_runner.go:195] Run: crio config
I0328 22:07:55.432846 1343785 cni.go:84] Creating CNI manager for ""
I0328 22:07:55.432873 1343785 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0328 22:07:55.432889 1343785 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0328 22:07:55.432913 1343785 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.30.0-beta.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-363849 NodeName:no-preload-363849 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPa
th:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0328 22:07:55.433063 1343785 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/crio/crio.sock
name: "no-preload-363849"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.30.0-beta.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/crio/crio.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0328 22:07:55.433146 1343785 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.30.0-beta.0
I0328 22:07:55.443403 1343785 binaries.go:44] Found k8s binaries, skipping transfer
I0328 22:07:55.443496 1343785 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0328 22:07:55.452427 1343785 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (374 bytes)
I0328 22:07:55.471647 1343785 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (359 bytes)
I0328 22:07:55.490185 1343785 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2162 bytes)
I0328 22:07:55.508680 1343785 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0328 22:07:55.512223 1343785 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0328 22:07:55.523310 1343785 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:07:55.613276 1343785 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0328 22:07:55.627173 1343785 certs.go:68] Setting up /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849 for IP: 192.168.85.2
I0328 22:07:55.627239 1343785 certs.go:194] generating shared ca certs ...
I0328 22:07:55.627296 1343785 certs.go:226] acquiring lock for ca certs: {Name:mk1e4b3d6020f96643d0b806687ddcafb6824b0b Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:07:55.627484 1343785 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.key
I0328 22:07:55.627555 1343785 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.key
I0328 22:07:55.627618 1343785 certs.go:256] generating profile certs ...
I0328 22:07:55.627761 1343785 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/client.key
I0328 22:07:55.627850 1343785 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/apiserver.key.8d62a42c
I0328 22:07:55.627935 1343785 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/proxy-client.key
I0328 22:07:55.628078 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363.pem (1338 bytes)
W0328 22:07:55.628190 1343785 certs.go:480] ignoring /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363_empty.pem, impossibly tiny 0 bytes
I0328 22:07:55.628228 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca-key.pem (1679 bytes)
I0328 22:07:55.628274 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/ca.pem (1082 bytes)
I0328 22:07:55.628337 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/cert.pem (1123 bytes)
I0328 22:07:55.628389 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/key.pem (1679 bytes)
I0328 22:07:55.628479 1343785 certs.go:484] found cert: /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem (1708 bytes)
I0328 22:07:55.629418 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0328 22:07:55.656776 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0328 22:07:55.694519 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0328 22:07:55.732617 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0328 22:07:55.759187 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0328 22:07:55.818372 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0328 22:07:55.858446 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0328 22:07:55.890894 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/profiles/no-preload-363849/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0328 22:07:55.917568 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/files/etc/ssl/certs/11513632.pem --> /usr/share/ca-certificates/11513632.pem (1708 bytes)
I0328 22:07:55.942473 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0328 22:07:55.967358 1343785 ssh_runner.go:362] scp /home/jenkins/minikube-integration/17877-1145955/.minikube/certs/1151363.pem --> /usr/share/ca-certificates/1151363.pem (1338 bytes)
I0328 22:07:55.993664 1343785 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (752 bytes)
I0328 22:07:56.015382 1343785 ssh_runner.go:195] Run: openssl version
I0328 22:07:56.022693 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/11513632.pem && ln -fs /usr/share/ca-certificates/11513632.pem /etc/ssl/certs/11513632.pem"
I0328 22:07:56.033305 1343785 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/11513632.pem
I0328 22:07:56.036793 1343785 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Mar 28 21:20 /usr/share/ca-certificates/11513632.pem
I0328 22:07:56.036873 1343785 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/11513632.pem
I0328 22:07:56.043726 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/11513632.pem /etc/ssl/certs/3ec20f2e.0"
I0328 22:07:56.053451 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0328 22:07:56.062983 1343785 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0328 22:07:56.066657 1343785 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Mar 28 21:12 /usr/share/ca-certificates/minikubeCA.pem
I0328 22:07:56.066794 1343785 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0328 22:07:56.074313 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0328 22:07:56.085299 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1151363.pem && ln -fs /usr/share/ca-certificates/1151363.pem /etc/ssl/certs/1151363.pem"
I0328 22:07:56.095052 1343785 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1151363.pem
I0328 22:07:56.098592 1343785 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Mar 28 21:20 /usr/share/ca-certificates/1151363.pem
I0328 22:07:56.098657 1343785 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1151363.pem
I0328 22:07:56.105920 1343785 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1151363.pem /etc/ssl/certs/51391683.0"
I0328 22:07:56.116437 1343785 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0328 22:07:56.119824 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0328 22:07:56.126737 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0328 22:07:56.134027 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0328 22:07:56.141130 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0328 22:07:56.147932 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0328 22:07:56.155528 1343785 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0328 22:07:56.162411 1343785 kubeadm.go:391] StartCluster: {Name:no-preload-363849 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.43-1711559786-18485@sha256:2dcab64da240d825290a528fa79ad3c32db45fe5f8be5150468234a7114eff82 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.0-beta.0 ClusterName:no-preload-363849 Namespace:default APIServerHAVIP: APIServerName:minikubeCA
APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.0-beta.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenki
ns:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0328 22:07:56.162508 1343785 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0328 22:07:56.162578 1343785 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0328 22:07:56.211566 1343785 cri.go:89] found id: ""
I0328 22:07:56.211709 1343785 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0328 22:07:56.222645 1343785 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0328 22:07:56.222720 1343785 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0328 22:07:56.222754 1343785 kubeadm.go:587] restartPrimaryControlPlane start ...
I0328 22:07:56.222845 1343785 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0328 22:07:56.235882 1343785 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0328 22:07:56.236662 1343785 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-363849" does not appear in /home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:07:56.237002 1343785 kubeconfig.go:62] /home/jenkins/minikube-integration/17877-1145955/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-363849" cluster setting kubeconfig missing "no-preload-363849" context setting]
I0328 22:07:56.237568 1343785 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17877-1145955/kubeconfig: {Name:mk01de9100d65131f49674a0d1051891ca674cd9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:07:56.239285 1343785 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0328 22:07:56.251230 1343785 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.85.2
I0328 22:07:56.251308 1343785 kubeadm.go:591] duration metric: took 28.529214ms to restartPrimaryControlPlane
I0328 22:07:56.251330 1343785 kubeadm.go:393] duration metric: took 88.927944ms to StartCluster
I0328 22:07:56.251385 1343785 settings.go:142] acquiring lock: {Name:mka22e5d6cd66b2677ac3cce373c1a6e13c189c9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:07:56.251474 1343785 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/17877-1145955/kubeconfig
I0328 22:07:56.252580 1343785 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/17877-1145955/kubeconfig: {Name:mk01de9100d65131f49674a0d1051891ca674cd9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0328 22:07:56.252853 1343785 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.0-beta.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0328 22:07:56.259968 1343785 out.go:177] * Verifying Kubernetes components...
I0328 22:07:56.253446 1343785 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0328 22:07:56.253642 1343785 config.go:182] Loaded profile config "no-preload-363849": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.0-beta.0
I0328 22:07:56.262777 1343785 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0328 22:07:56.262813 1343785 addons.go:69] Setting storage-provisioner=true in profile "no-preload-363849"
I0328 22:07:56.263026 1343785 addons.go:234] Setting addon storage-provisioner=true in "no-preload-363849"
W0328 22:07:56.263039 1343785 addons.go:243] addon storage-provisioner should already be in state true
I0328 22:07:56.263066 1343785 host.go:66] Checking if "no-preload-363849" exists ...
I0328 22:07:56.263510 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:56.262822 1343785 addons.go:69] Setting dashboard=true in profile "no-preload-363849"
I0328 22:07:56.263668 1343785 addons.go:234] Setting addon dashboard=true in "no-preload-363849"
W0328 22:07:56.263680 1343785 addons.go:243] addon dashboard should already be in state true
I0328 22:07:56.263700 1343785 host.go:66] Checking if "no-preload-363849" exists ...
I0328 22:07:56.264064 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:56.262829 1343785 addons.go:69] Setting default-storageclass=true in profile "no-preload-363849"
I0328 22:07:56.266653 1343785 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-363849"
I0328 22:07:56.267042 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:56.262835 1343785 addons.go:69] Setting metrics-server=true in profile "no-preload-363849"
I0328 22:07:56.268242 1343785 addons.go:234] Setting addon metrics-server=true in "no-preload-363849"
W0328 22:07:56.268269 1343785 addons.go:243] addon metrics-server should already be in state true
I0328 22:07:56.268304 1343785 host.go:66] Checking if "no-preload-363849" exists ...
I0328 22:07:56.268922 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:56.356692 1343785 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0328 22:07:56.361252 1343785 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0328 22:07:56.356629 1343785 addons.go:234] Setting addon default-storageclass=true in "no-preload-363849"
I0328 22:07:56.363996 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0328 22:07:56.364996 1343785 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0328 22:07:56.365007 1343785 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0328 22:07:56.366967 1343785 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0328 22:07:56.364997 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0328 22:07:56.366992 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0328 22:07:56.367055 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:56.367071 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:56.374647 1343785 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:07:56.374671 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0328 22:07:56.374736 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
W0328 22:07:56.364010 1343785 addons.go:243] addon default-storageclass should already be in state true
I0328 22:07:56.377707 1343785 host.go:66] Checking if "no-preload-363849" exists ...
I0328 22:07:56.378157 1343785 cli_runner.go:164] Run: docker container inspect no-preload-363849 --format={{.State.Status}}
I0328 22:07:56.400077 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:56.413221 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:56.443407 1343785 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0328 22:07:56.443427 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0328 22:07:56.443489 1343785 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-363849
I0328 22:07:56.444743 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:56.472393 1343785 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34559 SSHKeyPath:/home/jenkins/minikube-integration/17877-1145955/.minikube/machines/no-preload-363849/id_rsa Username:docker}
I0328 22:07:56.603058 1343785 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0328 22:07:56.623005 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:07:56.631866 1343785 node_ready.go:35] waiting up to 6m0s for node "no-preload-363849" to be "Ready" ...
I0328 22:07:56.701924 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0328 22:07:56.701952 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0328 22:07:56.708326 1343785 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0328 22:07:56.708348 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0328 22:07:56.716558 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:07:56.776499 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0328 22:07:56.776563 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0328 22:07:56.791089 1343785 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0328 22:07:56.791116 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0328 22:07:56.906688 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0328 22:07:56.906753 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
W0328 22:07:56.913260 1343785 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:56.913330 1343785 retry.go:31] will retry after 151.649039ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:56.923131 1343785 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:07:56.923195 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0328 22:07:56.964192 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0328 22:07:56.964231 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0328 22:07:56.982037 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0328 22:07:57.001821 1343785 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:57.001861 1343785 retry.go:31] will retry after 307.685852ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:57.032382 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0328 22:07:57.032413 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0328 22:07:57.065710 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0328 22:07:57.085247 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0328 22:07:57.085275 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0328 22:07:57.120033 1343785 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:57.120101 1343785 retry.go:31] will retry after 241.303163ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0328 22:07:57.136860 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0328 22:07:57.136888 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0328 22:07:57.185942 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0328 22:07:57.185968 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0328 22:07:57.218032 1343785 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:07:57.218059 1343785 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0328 22:07:57.238836 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0328 22:07:57.310060 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0328 22:07:57.362355 1343785 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0328 22:07:57.393698 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:07:59.891612 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:02.149786 1343785 node_ready.go:49] node "no-preload-363849" has status "Ready":"True"
I0328 22:08:02.149820 1343785 node_ready.go:38] duration metric: took 5.517880837s for node "no-preload-363849" to be "Ready" ...
I0328 22:08:02.149832 1343785 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0328 22:08:02.601578 1343785 pod_ready.go:78] waiting up to 6m0s for pod "coredns-7db6d8ff4d-58zbh" in "kube-system" namespace to be "Ready" ...
I0328 22:08:02.839993 1343785 pod_ready.go:92] pod "coredns-7db6d8ff4d-58zbh" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:02.840028 1343785 pod_ready.go:81] duration metric: took 238.412952ms for pod "coredns-7db6d8ff4d-58zbh" in "kube-system" namespace to be "Ready" ...
I0328 22:08:02.840045 1343785 pod_ready.go:78] waiting up to 6m0s for pod "etcd-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:03.105364 1343785 pod_ready.go:92] pod "etcd-no-preload-363849" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:03.105398 1343785 pod_ready.go:81] duration metric: took 265.344195ms for pod "etcd-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:03.105416 1343785 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:03.756256 1343785 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (6.690496709s)
I0328 22:08:04.049993 1343785 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (6.739886052s)
I0328 22:08:04.050225 1343785 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (6.811346395s)
I0328 22:08:04.050359 1343785 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (6.687969579s)
I0328 22:08:04.050385 1343785 addons.go:470] Verifying addon metrics-server=true in "no-preload-363849"
I0328 22:08:04.052343 1343785 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-363849 addons enable metrics-server
I0328 22:08:04.057216 1343785 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0328 22:08:02.393850 1338826 pod_ready.go:102] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:04.892201 1338826 pod_ready.go:92] pod "etcd-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:04.892227 1338826 pod_ready.go:81] duration metric: took 1m16.006325351s for pod "etcd-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.892259 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.897466 1338826 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:04.897491 1338826 pod_ready.go:81] duration metric: took 5.218747ms for pod "kube-apiserver-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.897503 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:04.059415 1343785 addons.go:505] duration metric: took 7.80597283s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0328 22:08:05.112621 1343785 pod_ready.go:102] pod "kube-apiserver-no-preload-363849" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:07.611848 1343785 pod_ready.go:102] pod "kube-apiserver-no-preload-363849" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:06.904157 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:09.403663 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:10.112775 1343785 pod_ready.go:102] pod "kube-apiserver-no-preload-363849" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:12.111943 1343785 pod_ready.go:92] pod "kube-apiserver-no-preload-363849" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:12.111967 1343785 pod_ready.go:81] duration metric: took 9.006543407s for pod "kube-apiserver-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.111978 1343785 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.118129 1343785 pod_ready.go:92] pod "kube-controller-manager-no-preload-363849" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:12.118157 1343785 pod_ready.go:81] duration metric: took 6.17042ms for pod "kube-controller-manager-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.118171 1343785 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-c2fkg" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.123666 1343785 pod_ready.go:92] pod "kube-proxy-c2fkg" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:12.123693 1343785 pod_ready.go:81] duration metric: took 5.514893ms for pod "kube-proxy-c2fkg" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.123706 1343785 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.137241 1343785 pod_ready.go:92] pod "kube-scheduler-no-preload-363849" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:12.137265 1343785 pod_ready.go:81] duration metric: took 13.551188ms for pod "kube-scheduler-no-preload-363849" in "kube-system" namespace to be "Ready" ...
I0328 22:08:12.137277 1343785 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace to be "Ready" ...
I0328 22:08:11.405298 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:13.904187 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:14.144059 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:16.643651 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:16.403711 1338826 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:17.909294 1338826 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.909370 1338826 pod_ready.go:81] duration metric: took 13.011857211s for pod "kube-controller-manager-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.909397 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-9vs8r" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.918637 1338826 pod_ready.go:92] pod "kube-proxy-9vs8r" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.918672 1338826 pod_ready.go:81] duration metric: took 9.258655ms for pod "kube-proxy-9vs8r" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.918685 1338826 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.927105 1338826 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace has status "Ready":"True"
I0328 22:08:17.927130 1338826 pod_ready.go:81] duration metric: took 8.437573ms for pod "kube-scheduler-old-k8s-version-633693" in "kube-system" namespace to be "Ready" ...
I0328 22:08:17.927142 1338826 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace to be "Ready" ...
I0328 22:08:19.942294 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:19.143418 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:21.144771 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:22.434712 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:24.435112 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:23.650528 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:26.144807 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:26.933676 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:28.935233 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:28.643298 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:30.643343 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:33.142890 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:31.434711 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:33.932938 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:35.144472 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:37.646250 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:35.933182 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:37.933531 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:40.434985 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:40.149566 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:42.643175 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:42.932957 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:44.933746 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:44.643416 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:46.643671 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:47.432638 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:49.433249 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:49.143851 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:51.144359 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:51.433961 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:53.434580 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:53.644020 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:56.144300 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:58.145689 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:55.932722 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:08:57.933867 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:00.434560 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:00.161198 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:02.643446 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:02.932551 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:04.932913 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:05.144438 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:07.643527 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:07.433550 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:09.433637 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:10.143586 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:12.144340 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:11.933587 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:14.434340 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:14.155912 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:16.643503 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:16.434764 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:18.932404 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:18.643897 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:21.144159 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:23.144703 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:20.933066 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:22.938118 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:25.432907 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:25.643119 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:27.644236 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:27.433591 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:29.933701 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:30.145427 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:32.645092 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:32.433501 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:34.932827 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:35.144156 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:37.643526 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:36.934724 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:39.433686 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:40.144030 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:42.146006 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:41.434345 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:43.434422 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:44.643631 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:47.152838 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:45.932966 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:47.933007 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:49.933857 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:49.643780 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:52.143925 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:52.433046 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:54.434259 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:54.643593 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:57.144203 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:56.933632 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:58.933976 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:09:59.643009 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:01.644333 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:01.434400 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:03.434444 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:04.143341 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:06.147812 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:05.932668 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:07.934860 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:09.935369 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:08.643466 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:10.643927 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:12.644512 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:12.434151 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:14.933434 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:15.143703 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:17.143976 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:16.934278 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:19.434252 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:19.643434 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:22.143905 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:21.434530 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:23.933010 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:24.643574 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:26.643657 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:26.433027 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:28.433948 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:29.143718 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:31.643754 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:30.933738 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:32.933910 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:35.432932 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:34.143981 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:36.643479 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:37.434118 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:39.441245 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:39.143486 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:41.143865 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:41.932963 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:43.933931 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:43.643481 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:46.143296 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:48.144234 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:46.434034 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:48.933457 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:50.643518 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:52.643870 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:51.432895 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:53.434727 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:55.440622 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:55.145055 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:57.643473 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:10:57.933693 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:00.435477 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:00.176261 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:02.643711 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:02.932933 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:04.933833 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:05.143659 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:07.643313 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:07.433346 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:09.433437 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:09.643536 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:11.644359 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:11.933095 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:13.933603 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:14.143848 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:16.643114 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:16.433215 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:18.933087 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:19.143585 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:21.144363 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:20.933409 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:23.434532 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:25.434708 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:23.643590 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:26.142875 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:27.932941 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:29.933456 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:28.642795 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:30.643048 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:32.643408 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:31.933872 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:34.433180 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:34.644530 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:37.143975 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:36.433308 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:38.433890 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:40.434164 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:39.643734 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:42.144506 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:42.932888 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:44.932980 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:44.643656 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:47.143743 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:46.933414 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:49.432985 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:49.144185 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:51.644630 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:51.433663 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:53.434707 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:54.144272 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:56.643147 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:55.933135 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:57.933821 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:00.434788 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:11:58.643245 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:00.644232 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:03.142946 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:02.933354 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:04.934434 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:05.143851 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:07.144042 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:07.433060 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:09.433317 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:09.144261 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:11.643843 1343785 pod_ready.go:102] pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:12.143791 1343785 pod_ready.go:81] duration metric: took 4m0.006499541s for pod "metrics-server-569cc877fc-pttn4" in "kube-system" namespace to be "Ready" ...
E0328 22:12:12.143820 1343785 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0328 22:12:12.143830 1343785 pod_ready.go:38] duration metric: took 4m9.993988098s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0328 22:12:12.143845 1343785 api_server.go:52] waiting for apiserver process to appear ...
I0328 22:12:12.143876 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:12.143943 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:12.202036 1343785 cri.go:89] found id: "2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:12.202062 1343785 cri.go:89] found id: ""
I0328 22:12:12.202070 1343785 logs.go:276] 1 containers: [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37]
I0328 22:12:12.202136 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.207197 1343785 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:12.207271 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:12.256751 1343785 cri.go:89] found id: "77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:12.256779 1343785 cri.go:89] found id: ""
I0328 22:12:12.256788 1343785 logs.go:276] 1 containers: [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4]
I0328 22:12:12.256845 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.260599 1343785 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:12.260677 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:12.300601 1343785 cri.go:89] found id: "ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:12.300625 1343785 cri.go:89] found id: ""
I0328 22:12:12.300634 1343785 logs.go:276] 1 containers: [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7]
I0328 22:12:12.300703 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.304280 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:12.304359 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:12.344780 1343785 cri.go:89] found id: "c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:12.344852 1343785 cri.go:89] found id: ""
I0328 22:12:12.344867 1343785 logs.go:276] 1 containers: [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5]
I0328 22:12:12.344942 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.349071 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:12.349208 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:12.393558 1343785 cri.go:89] found id: "02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:12.393578 1343785 cri.go:89] found id: ""
I0328 22:12:12.393586 1343785 logs.go:276] 1 containers: [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43]
I0328 22:12:12.393642 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.397333 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:12.397413 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:12.445506 1343785 cri.go:89] found id: "b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:12.445527 1343785 cri.go:89] found id: ""
I0328 22:12:12.445535 1343785 logs.go:276] 1 containers: [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208]
I0328 22:12:12.445602 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.449650 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:12.449753 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:12.491692 1343785 cri.go:89] found id: "2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:12.491713 1343785 cri.go:89] found id: ""
I0328 22:12:12.491721 1343785 logs.go:276] 1 containers: [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5]
I0328 22:12:12.491799 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.495254 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:12.495357 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:12.539841 1343785 cri.go:89] found id: "1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:12.539863 1343785 cri.go:89] found id: ""
I0328 22:12:12.539875 1343785 logs.go:276] 1 containers: [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15]
I0328 22:12:12.539951 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.543604 1343785 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:12.543673 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:12.583762 1343785 cri.go:89] found id: "4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:12.583834 1343785 cri.go:89] found id: "3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:12.583855 1343785 cri.go:89] found id: ""
I0328 22:12:12.583878 1343785 logs.go:276] 2 containers: [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a]
I0328 22:12:12.583971 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.587596 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:12.590988 1343785 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:12.591013 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:12.672553 1343785 logs.go:123] Gathering logs for kube-apiserver [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37] ...
I0328 22:12:12.672591 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:12.724719 1343785 logs.go:123] Gathering logs for coredns [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7] ...
I0328 22:12:12.724752 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:12.775723 1343785 logs.go:123] Gathering logs for kube-scheduler [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5] ...
I0328 22:12:12.775750 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:12.822037 1343785 logs.go:123] Gathering logs for kube-controller-manager [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208] ...
I0328 22:12:12.822064 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:12.944698 1343785 logs.go:123] Gathering logs for storage-provisioner [3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a] ...
I0328 22:12:12.944737 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:12.982978 1343785 logs.go:123] Gathering logs for storage-provisioner [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2] ...
I0328 22:12:12.983011 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:13.025340 1343785 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:13.025367 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:13.049286 1343785 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:13.049322 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:13.207531 1343785 logs.go:123] Gathering logs for etcd [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4] ...
I0328 22:12:13.207606 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:13.265231 1343785 logs.go:123] Gathering logs for kube-proxy [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43] ...
I0328 22:12:13.265264 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:13.303419 1343785 logs.go:123] Gathering logs for kubernetes-dashboard [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15] ...
I0328 22:12:13.303449 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:13.349553 1343785 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:13.349584 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:13.381587 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:13.381824 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:13.426377 1343785 logs.go:123] Gathering logs for container status ...
I0328 22:12:13.426416 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:13.481053 1343785 logs.go:123] Gathering logs for kindnet [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5] ...
I0328 22:12:13.481084 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:13.522348 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:13.522383 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:13.522442 1343785 out.go:239] X Problems detected in kubelet:
W0328 22:12:13.522460 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:13.522471 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:13.522480 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:13.522489 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:11.433605 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:13.437226 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:15.933636 1338826 pod_ready.go:102] pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace has status "Ready":"False"
I0328 22:12:17.933340 1338826 pod_ready.go:81] duration metric: took 4m0.006183924s for pod "metrics-server-9975d5f86-h5ts8" in "kube-system" namespace to be "Ready" ...
E0328 22:12:17.933366 1338826 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0328 22:12:17.933376 1338826 pod_ready.go:38] duration metric: took 5m30.183327024s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0328 22:12:17.933390 1338826 api_server.go:52] waiting for apiserver process to appear ...
I0328 22:12:17.933418 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:17.933480 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:17.975520 1338826 cri.go:89] found id: "aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:17.975543 1338826 cri.go:89] found id: ""
I0328 22:12:17.975551 1338826 logs.go:276] 1 containers: [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5]
I0328 22:12:17.975605 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:17.978975 1338826 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:17.979046 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:18.021754 1338826 cri.go:89] found id: "54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:18.021779 1338826 cri.go:89] found id: ""
I0328 22:12:18.021787 1338826 logs.go:276] 1 containers: [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263]
I0328 22:12:18.021845 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.025748 1338826 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:18.025846 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:18.069778 1338826 cri.go:89] found id: "9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:18.069809 1338826 cri.go:89] found id: ""
I0328 22:12:18.069819 1338826 logs.go:276] 1 containers: [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962]
I0328 22:12:18.069881 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.073970 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:18.074054 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:18.116221 1338826 cri.go:89] found id: "0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:18.116246 1338826 cri.go:89] found id: ""
I0328 22:12:18.116254 1338826 logs.go:276] 1 containers: [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b]
I0328 22:12:18.116315 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.120360 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:18.120438 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:18.160651 1338826 cri.go:89] found id: "99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:18.160672 1338826 cri.go:89] found id: ""
I0328 22:12:18.160681 1338826 logs.go:276] 1 containers: [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95]
I0328 22:12:18.160741 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.164797 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:18.164870 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:18.208058 1338826 cri.go:89] found id: "accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:18.208165 1338826 cri.go:89] found id: ""
I0328 22:12:18.208192 1338826 logs.go:276] 1 containers: [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591]
I0328 22:12:18.208295 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.212186 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:18.212308 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:18.257286 1338826 cri.go:89] found id: "f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:18.257353 1338826 cri.go:89] found id: ""
I0328 22:12:18.257367 1338826 logs.go:276] 1 containers: [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b]
I0328 22:12:18.257425 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.263454 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:18.263535 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:18.301918 1338826 cri.go:89] found id: "dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:18.301941 1338826 cri.go:89] found id: ""
I0328 22:12:18.301949 1338826 logs.go:276] 1 containers: [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d]
I0328 22:12:18.302004 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.305551 1338826 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:18.305626 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:18.346240 1338826 cri.go:89] found id: "dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:18.346264 1338826 cri.go:89] found id: "de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:18.346269 1338826 cri.go:89] found id: ""
I0328 22:12:18.346277 1338826 logs.go:276] 2 containers: [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0]
I0328 22:12:18.346333 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.349950 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:18.353205 1338826 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:18.353227 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:18.405413 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826315 742 reflector.go:138] object-"kube-system"/"storage-provisioner-token-htmqq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-htmqq" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.405657 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826559 742 reflector.go:138] object-"default"/"default-token-skqvg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-skqvg" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.405882 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826733 742 reflector.go:138] object-"kube-system"/"metrics-server-token-qkmwr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-qkmwr" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406094 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829336 742 reflector.go:138] object-"kube-system"/"coredns-token-zjvsj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zjvsj" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406306 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829576 742 reflector.go:138] object-"kube-system"/"kindnet-token-g4wkb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g4wkb" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406507 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829784 742 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406719 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829969 742 reflector.go:138] object-"kube-system"/"kube-proxy-token-l4tct": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l4tct" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.406925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.830130 742 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.415925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.527663 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.416174 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.814044 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.418221 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:04 old-k8s-version-633693 kubelet[742]: E0328 22:07:04.699349 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.418545 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:07 old-k8s-version-633693 kubelet[742]: E0328 22:07:07.707146 742 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-kddb6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-kddb6" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:18.421763 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:15 old-k8s-version-633693 kubelet[742]: E0328 22:07:15.934579 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.422221 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:16 old-k8s-version-633693 kubelet[742]: E0328 22:07:16.937591 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.422406 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:19 old-k8s-version-633693 kubelet[742]: E0328 22:07:19.733048 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.422861 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:22 old-k8s-version-633693 kubelet[742]: E0328 22:07:22.808225 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.424949 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:34 old-k8s-version-633693 kubelet[742]: E0328 22:07:34.698485 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.425546 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:36 old-k8s-version-633693 kubelet[742]: E0328 22:07:36.977259 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.425874 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:42 old-k8s-version-633693 kubelet[742]: E0328 22:07:42.808747 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.426059 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:48 old-k8s-version-633693 kubelet[742]: E0328 22:07:48.688107 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.426384 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:54 old-k8s-version-633693 kubelet[742]: E0328 22:07:54.687538 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.426568 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:02 old-k8s-version-633693 kubelet[742]: E0328 22:08:02.688287 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.427151 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:09 old-k8s-version-633693 kubelet[742]: E0328 22:08:09.025390 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.427477 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:12 old-k8s-version-633693 kubelet[742]: E0328 22:08:12.808271 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.427660 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:14 old-k8s-version-633693 kubelet[742]: E0328 22:08:14.687911 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.429727 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:25 old-k8s-version-633693 kubelet[742]: E0328 22:08:25.704436 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.430054 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:27 old-k8s-version-633693 kubelet[742]: E0328 22:08:27.687797 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.430237 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:39 old-k8s-version-633693 kubelet[742]: E0328 22:08:39.688581 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.430562 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:41 old-k8s-version-633693 kubelet[742]: E0328 22:08:41.687550 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.430787 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:51 old-k8s-version-633693 kubelet[742]: E0328 22:08:51.689572 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.431378 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:55 old-k8s-version-633693 kubelet[742]: E0328 22:08:55.094326 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.431561 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.687902 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.431886 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.808415 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432226 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:13 old-k8s-version-633693 kubelet[742]: E0328 22:09:13.689276 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432413 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:14 old-k8s-version-633693 kubelet[742]: E0328 22:09:14.688165 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.432741 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:24 old-k8s-version-633693 kubelet[742]: E0328 22:09:24.687486 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.432928 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:29 old-k8s-version-633693 kubelet[742]: E0328 22:09:29.688477 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.433254 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:35 old-k8s-version-633693 kubelet[742]: E0328 22:09:35.687479 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.433952 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:40 old-k8s-version-633693 kubelet[742]: E0328 22:09:40.688238 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.434277 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:48 old-k8s-version-633693 kubelet[742]: E0328 22:09:48.687433 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.436363 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:51 old-k8s-version-633693 kubelet[742]: E0328 22:09:51.697009 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:18.436691 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:02 old-k8s-version-633693 kubelet[742]: E0328 22:10:02.687437 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.436876 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:03 old-k8s-version-633693 kubelet[742]: E0328 22:10:03.688247 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.437462 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.214323 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.437646 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.688145 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.437971 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:22 old-k8s-version-633693 kubelet[742]: E0328 22:10:22.808176 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.438155 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:29 old-k8s-version-633693 kubelet[742]: E0328 22:10:29.688127 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.438480 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:33 old-k8s-version-633693 kubelet[742]: E0328 22:10:33.687521 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.438666 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:40 old-k8s-version-633693 kubelet[742]: E0328 22:10:40.688295 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.438993 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:46 old-k8s-version-633693 kubelet[742]: E0328 22:10:46.687410 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.439177 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:52 old-k8s-version-633693 kubelet[742]: E0328 22:10:52.688204 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.439502 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:57 old-k8s-version-633693 kubelet[742]: E0328 22:10:57.687778 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.439686 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:07 old-k8s-version-633693 kubelet[742]: E0328 22:11:07.688569 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.440010 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: E0328 22:11:10.687505 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.440203 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:21 old-k8s-version-633693 kubelet[742]: E0328 22:11:21.688528 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.440529 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: E0328 22:11:25.687833 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.440714 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:36 old-k8s-version-633693 kubelet[742]: E0328 22:11:36.687961 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.441289 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: E0328 22:11:39.687565 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.441475 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:51 old-k8s-version-633693 kubelet[742]: E0328 22:11:51.689147 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.441800 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.441983 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:18.442307 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.442634 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:18.442816 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:18.442826 1338826 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:18.442840 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:18.610600 1338826 logs.go:123] Gathering logs for coredns [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962] ...
I0328 22:12:18.610629 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:18.651719 1338826 logs.go:123] Gathering logs for storage-provisioner [de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0] ...
I0328 22:12:18.651747 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:18.692899 1338826 logs.go:123] Gathering logs for container status ...
I0328 22:12:18.692993 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:18.756410 1338826 logs.go:123] Gathering logs for kube-apiserver [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5] ...
I0328 22:12:18.756441 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:18.825430 1338826 logs.go:123] Gathering logs for kube-controller-manager [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591] ...
I0328 22:12:18.825468 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:18.915204 1338826 logs.go:123] Gathering logs for kindnet [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b] ...
I0328 22:12:18.915239 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:18.970831 1338826 logs.go:123] Gathering logs for kube-proxy [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95] ...
I0328 22:12:18.970914 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:19.019338 1338826 logs.go:123] Gathering logs for storage-provisioner [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723] ...
I0328 22:12:19.019365 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:19.064283 1338826 logs.go:123] Gathering logs for kubernetes-dashboard [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d] ...
I0328 22:12:19.064313 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:19.104471 1338826 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:19.104499 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:19.184921 1338826 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:19.184961 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:19.203985 1338826 logs.go:123] Gathering logs for etcd [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263] ...
I0328 22:12:19.204137 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:19.250334 1338826 logs.go:123] Gathering logs for kube-scheduler [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b] ...
I0328 22:12:19.250368 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:19.293923 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:19.293952 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:19.294030 1338826 out.go:239] X Problems detected in kubelet:
W0328 22:12:19.294045 1338826 out.go:239] Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294087 1338826 out.go:239] Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:19.294098 1338826 out.go:239] Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294105 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:19.294114 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:19.294121 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:19.294128 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:23.523244 1343785 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0328 22:12:23.535238 1343785 api_server.go:72] duration metric: took 4m27.2823195s to wait for apiserver process to appear ...
I0328 22:12:23.535268 1343785 api_server.go:88] waiting for apiserver healthz status ...
I0328 22:12:23.535302 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:23.535366 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:23.573014 1343785 cri.go:89] found id: "2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:23.573035 1343785 cri.go:89] found id: ""
I0328 22:12:23.573043 1343785 logs.go:276] 1 containers: [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37]
I0328 22:12:23.573101 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.576560 1343785 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:23.576641 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:23.621690 1343785 cri.go:89] found id: "77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:23.622946 1343785 cri.go:89] found id: ""
I0328 22:12:23.622957 1343785 logs.go:276] 1 containers: [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4]
I0328 22:12:23.623023 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.626834 1343785 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:23.626906 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:23.663733 1343785 cri.go:89] found id: "ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:23.663768 1343785 cri.go:89] found id: ""
I0328 22:12:23.663776 1343785 logs.go:276] 1 containers: [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7]
I0328 22:12:23.663846 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.667380 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:23.667450 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:23.705506 1343785 cri.go:89] found id: "c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:23.705527 1343785 cri.go:89] found id: ""
I0328 22:12:23.705535 1343785 logs.go:276] 1 containers: [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5]
I0328 22:12:23.705610 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.709096 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:23.709162 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:23.749488 1343785 cri.go:89] found id: "02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:23.749508 1343785 cri.go:89] found id: ""
I0328 22:12:23.749516 1343785 logs.go:276] 1 containers: [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43]
I0328 22:12:23.749595 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.753123 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:23.753206 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:23.796746 1343785 cri.go:89] found id: "b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:23.796771 1343785 cri.go:89] found id: ""
I0328 22:12:23.796779 1343785 logs.go:276] 1 containers: [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208]
I0328 22:12:23.796832 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.800250 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:23.800325 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:23.842130 1343785 cri.go:89] found id: "2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:23.842163 1343785 cri.go:89] found id: ""
I0328 22:12:23.842172 1343785 logs.go:276] 1 containers: [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5]
I0328 22:12:23.842233 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.846713 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:23.846791 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:23.884150 1343785 cri.go:89] found id: "1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:23.884179 1343785 cri.go:89] found id: ""
I0328 22:12:23.884187 1343785 logs.go:276] 1 containers: [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15]
I0328 22:12:23.884243 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.887848 1343785 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:23.887919 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:23.931035 1343785 cri.go:89] found id: "4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:23.931058 1343785 cri.go:89] found id: "3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:23.931063 1343785 cri.go:89] found id: ""
I0328 22:12:23.931070 1343785 logs.go:276] 2 containers: [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a]
I0328 22:12:23.931132 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.934849 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:23.938226 1343785 logs.go:123] Gathering logs for kube-apiserver [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37] ...
I0328 22:12:23.938253 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:23.988318 1343785 logs.go:123] Gathering logs for coredns [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7] ...
I0328 22:12:23.988350 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:24.036815 1343785 logs.go:123] Gathering logs for kube-scheduler [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5] ...
I0328 22:12:24.036844 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:24.079329 1343785 logs.go:123] Gathering logs for kube-proxy [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43] ...
I0328 22:12:24.079360 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:24.122361 1343785 logs.go:123] Gathering logs for etcd [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4] ...
I0328 22:12:24.122394 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:24.174268 1343785 logs.go:123] Gathering logs for kube-controller-manager [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208] ...
I0328 22:12:24.174301 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:24.234523 1343785 logs.go:123] Gathering logs for storage-provisioner [3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a] ...
I0328 22:12:24.234559 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:24.286185 1343785 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:24.286238 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:24.369922 1343785 logs.go:123] Gathering logs for container status ...
I0328 22:12:24.369997 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:24.414293 1343785 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:24.414320 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:24.444721 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:24.444981 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:24.490788 1343785 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:24.490819 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:24.514484 1343785 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:24.514511 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:24.641992 1343785 logs.go:123] Gathering logs for kindnet [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5] ...
I0328 22:12:24.642027 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:24.683454 1343785 logs.go:123] Gathering logs for kubernetes-dashboard [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15] ...
I0328 22:12:24.683561 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:24.724286 1343785 logs.go:123] Gathering logs for storage-provisioner [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2] ...
I0328 22:12:24.724314 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:24.760795 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:24.760818 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:24.760888 1343785 out.go:239] X Problems detected in kubelet:
W0328 22:12:24.760900 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:24.760907 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:24.760931 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:24.760943 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:29.295105 1338826 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0328 22:12:29.306579 1338826 api_server.go:72] duration metric: took 5m59.438575826s to wait for apiserver process to appear ...
I0328 22:12:29.306606 1338826 api_server.go:88] waiting for apiserver healthz status ...
I0328 22:12:29.306641 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:29.306705 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:29.345775 1338826 cri.go:89] found id: "aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:29.345797 1338826 cri.go:89] found id: ""
I0328 22:12:29.345805 1338826 logs.go:276] 1 containers: [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5]
I0328 22:12:29.345862 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.349735 1338826 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:29.349812 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:29.389469 1338826 cri.go:89] found id: "54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:29.389500 1338826 cri.go:89] found id: ""
I0328 22:12:29.389510 1338826 logs.go:276] 1 containers: [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263]
I0328 22:12:29.389587 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.393252 1338826 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:29.393335 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:29.435046 1338826 cri.go:89] found id: "9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:29.435068 1338826 cri.go:89] found id: ""
I0328 22:12:29.435076 1338826 logs.go:276] 1 containers: [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962]
I0328 22:12:29.435134 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.439053 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:29.439135 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:29.478063 1338826 cri.go:89] found id: "0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:29.478084 1338826 cri.go:89] found id: ""
I0328 22:12:29.478092 1338826 logs.go:276] 1 containers: [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b]
I0328 22:12:29.478148 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.481825 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:29.481896 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:29.524721 1338826 cri.go:89] found id: "99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:29.524784 1338826 cri.go:89] found id: ""
I0328 22:12:29.524806 1338826 logs.go:276] 1 containers: [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95]
I0328 22:12:29.524879 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.528593 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:29.528672 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:29.570444 1338826 cri.go:89] found id: "accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:29.570465 1338826 cri.go:89] found id: ""
I0328 22:12:29.570472 1338826 logs.go:276] 1 containers: [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591]
I0328 22:12:29.570548 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.574273 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:29.574347 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:29.615918 1338826 cri.go:89] found id: "f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:29.615941 1338826 cri.go:89] found id: ""
I0328 22:12:29.615948 1338826 logs.go:276] 1 containers: [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b]
I0328 22:12:29.616002 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.619941 1338826 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:29.620011 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:29.656396 1338826 cri.go:89] found id: "dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:29.656421 1338826 cri.go:89] found id: ""
I0328 22:12:29.656429 1338826 logs.go:276] 1 containers: [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d]
I0328 22:12:29.656484 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.660191 1338826 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:29.660260 1338826 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:29.697904 1338826 cri.go:89] found id: "dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:29.697933 1338826 cri.go:89] found id: "de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:29.697938 1338826 cri.go:89] found id: ""
I0328 22:12:29.697945 1338826 logs.go:276] 2 containers: [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0]
I0328 22:12:29.698002 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.701729 1338826 ssh_runner.go:195] Run: which crictl
I0328 22:12:29.705384 1338826 logs.go:123] Gathering logs for kube-proxy [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95] ...
I0328 22:12:29.705454 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95"
I0328 22:12:29.749991 1338826 logs.go:123] Gathering logs for storage-provisioner [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723] ...
I0328 22:12:29.750020 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723"
I0328 22:12:29.795978 1338826 logs.go:123] Gathering logs for storage-provisioner [de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0] ...
I0328 22:12:29.796009 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0"
I0328 22:12:29.859914 1338826 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:29.859942 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:29.880830 1338826 logs.go:123] Gathering logs for etcd [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263] ...
I0328 22:12:29.880857 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263"
I0328 22:12:29.935958 1338826 logs.go:123] Gathering logs for kubernetes-dashboard [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d] ...
I0328 22:12:29.935992 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d"
I0328 22:12:29.981822 1338826 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:29.981850 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:30.152830 1338826 logs.go:123] Gathering logs for kindnet [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b] ...
I0328 22:12:30.152868 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b"
I0328 22:12:30.199134 1338826 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:30.199177 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:30.283626 1338826 logs.go:123] Gathering logs for container status ...
I0328 22:12:30.283664 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:30.353160 1338826 logs.go:123] Gathering logs for kube-scheduler [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b] ...
I0328 22:12:30.353193 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b"
I0328 22:12:30.405400 1338826 logs.go:123] Gathering logs for kube-controller-manager [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591] ...
I0328 22:12:30.405432 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591"
I0328 22:12:30.506110 1338826 logs.go:123] Gathering logs for coredns [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962] ...
I0328 22:12:30.506147 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962"
I0328 22:12:30.564490 1338826 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:30.564518 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:30.618150 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826315 742 reflector.go:138] object-"kube-system"/"storage-provisioner-token-htmqq": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-htmqq" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618396 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826559 742 reflector.go:138] object-"default"/"default-token-skqvg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-skqvg" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618624 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.826733 742 reflector.go:138] object-"kube-system"/"metrics-server-token-qkmwr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-qkmwr" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.618835 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829336 742 reflector.go:138] object-"kube-system"/"coredns-token-zjvsj": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-zjvsj" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619044 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829576 742 reflector.go:138] object-"kube-system"/"kindnet-token-g4wkb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-g4wkb" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619249 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829784 742 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619463 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.829969 742 reflector.go:138] object-"kube-system"/"kube-proxy-token-l4tct": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-l4tct" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.619666 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:47 old-k8s-version-633693 kubelet[742]: E0328 22:06:47.830130 742 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.628872 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.527663 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.629070 1338826 logs.go:138] Found kubelet problem: Mar 28 22:06:50 old-k8s-version-633693 kubelet[742]: E0328 22:06:50.814044 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.631140 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:04 old-k8s-version-633693 kubelet[742]: E0328 22:07:04.699349 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.631470 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:07 old-k8s-version-633693 kubelet[742]: E0328 22:07:07.707146 742 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-kddb6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-kddb6" is forbidden: User "system:node:old-k8s-version-633693" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-633693' and this object
W0328 22:12:30.634771 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:15 old-k8s-version-633693 kubelet[742]: E0328 22:07:15.934579 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.635235 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:16 old-k8s-version-633693 kubelet[742]: E0328 22:07:16.937591 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.635423 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:19 old-k8s-version-633693 kubelet[742]: E0328 22:07:19.733048 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.635928 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:22 old-k8s-version-633693 kubelet[742]: E0328 22:07:22.808225 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.638464 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:34 old-k8s-version-633693 kubelet[742]: E0328 22:07:34.698485 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.639072 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:36 old-k8s-version-633693 kubelet[742]: E0328 22:07:36.977259 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.639400 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:42 old-k8s-version-633693 kubelet[742]: E0328 22:07:42.808747 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.639586 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:48 old-k8s-version-633693 kubelet[742]: E0328 22:07:48.688107 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.639926 1338826 logs.go:138] Found kubelet problem: Mar 28 22:07:54 old-k8s-version-633693 kubelet[742]: E0328 22:07:54.687538 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.640169 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:02 old-k8s-version-633693 kubelet[742]: E0328 22:08:02.688287 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.640768 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:09 old-k8s-version-633693 kubelet[742]: E0328 22:08:09.025390 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.641095 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:12 old-k8s-version-633693 kubelet[742]: E0328 22:08:12.808271 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.641278 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:14 old-k8s-version-633693 kubelet[742]: E0328 22:08:14.687911 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.643355 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:25 old-k8s-version-633693 kubelet[742]: E0328 22:08:25.704436 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.643711 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:27 old-k8s-version-633693 kubelet[742]: E0328 22:08:27.687797 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.643899 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:39 old-k8s-version-633693 kubelet[742]: E0328 22:08:39.688581 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.644248 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:41 old-k8s-version-633693 kubelet[742]: E0328 22:08:41.687550 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.645266 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:51 old-k8s-version-633693 kubelet[742]: E0328 22:08:51.689572 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.645867 1338826 logs.go:138] Found kubelet problem: Mar 28 22:08:55 old-k8s-version-633693 kubelet[742]: E0328 22:08:55.094326 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646051 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.687902 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.646397 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:02 old-k8s-version-633693 kubelet[742]: E0328 22:09:02.808415 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646732 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:13 old-k8s-version-633693 kubelet[742]: E0328 22:09:13.689276 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.646918 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:14 old-k8s-version-633693 kubelet[742]: E0328 22:09:14.688165 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.647245 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:24 old-k8s-version-633693 kubelet[742]: E0328 22:09:24.687486 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.647451 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:29 old-k8s-version-633693 kubelet[742]: E0328 22:09:29.688477 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.647780 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:35 old-k8s-version-633693 kubelet[742]: E0328 22:09:35.687479 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.648521 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:40 old-k8s-version-633693 kubelet[742]: E0328 22:09:40.688238 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.648899 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:48 old-k8s-version-633693 kubelet[742]: E0328 22:09:48.687433 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.651003 1338826 logs.go:138] Found kubelet problem: Mar 28 22:09:51 old-k8s-version-633693 kubelet[742]: E0328 22:09:51.697009 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0328 22:12:30.651336 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:02 old-k8s-version-633693 kubelet[742]: E0328 22:10:02.687437 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.651523 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:03 old-k8s-version-633693 kubelet[742]: E0328 22:10:03.688247 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.652119 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.214323 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.652326 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:16 old-k8s-version-633693 kubelet[742]: E0328 22:10:16.688145 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.652674 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:22 old-k8s-version-633693 kubelet[742]: E0328 22:10:22.808176 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.652864 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:29 old-k8s-version-633693 kubelet[742]: E0328 22:10:29.688127 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.653193 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:33 old-k8s-version-633693 kubelet[742]: E0328 22:10:33.687521 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.653377 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:40 old-k8s-version-633693 kubelet[742]: E0328 22:10:40.688295 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.653719 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:46 old-k8s-version-633693 kubelet[742]: E0328 22:10:46.687410 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.653906 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:52 old-k8s-version-633693 kubelet[742]: E0328 22:10:52.688204 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.654232 1338826 logs.go:138] Found kubelet problem: Mar 28 22:10:57 old-k8s-version-633693 kubelet[742]: E0328 22:10:57.687778 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.654416 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:07 old-k8s-version-633693 kubelet[742]: E0328 22:11:07.688569 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.654742 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: E0328 22:11:10.687505 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.654925 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:21 old-k8s-version-633693 kubelet[742]: E0328 22:11:21.688528 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.655250 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: E0328 22:11:25.687833 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.655433 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:36 old-k8s-version-633693 kubelet[742]: E0328 22:11:36.687961 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.656006 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: E0328 22:11:39.687565 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.656212 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:51 old-k8s-version-633693 kubelet[742]: E0328 22:11:51.689147 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.656539 1338826 logs.go:138] Found kubelet problem: Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.656739 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.657073 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.657397 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.657583 1338826 logs.go:138] Found kubelet problem: Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:30.657593 1338826 logs.go:123] Gathering logs for kube-apiserver [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5] ...
I0328 22:12:30.657607 1338826 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5"
I0328 22:12:30.726909 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:30.726942 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:30.727001 1338826 out.go:239] X Problems detected in kubelet:
W0328 22:12:30.727013 1338826 out.go:239] Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727021 1338826 out.go:239] Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0328 22:12:30.727028 1338826 out.go:239] Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727039 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
W0328 22:12:30.727048 1338826 out.go:239] Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0328 22:12:30.727062 1338826 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:30.727068 1338826 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:34.761967 1343785 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0328 22:12:34.771071 1343785 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0328 22:12:34.772106 1343785 api_server.go:141] control plane version: v1.30.0-beta.0
I0328 22:12:34.772131 1343785 api_server.go:131] duration metric: took 11.236856775s to wait for apiserver health ...
I0328 22:12:34.772140 1343785 system_pods.go:43] waiting for kube-system pods to appear ...
I0328 22:12:34.772169 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0328 22:12:34.772232 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0328 22:12:34.808896 1343785 cri.go:89] found id: "2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:34.808918 1343785 cri.go:89] found id: ""
I0328 22:12:34.808926 1343785 logs.go:276] 1 containers: [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37]
I0328 22:12:34.808982 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:34.814246 1343785 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0328 22:12:34.814323 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0328 22:12:34.862726 1343785 cri.go:89] found id: "77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:34.862749 1343785 cri.go:89] found id: ""
I0328 22:12:34.862756 1343785 logs.go:276] 1 containers: [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4]
I0328 22:12:34.862813 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:34.866259 1343785 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0328 22:12:34.866334 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0328 22:12:34.918324 1343785 cri.go:89] found id: "ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:34.918348 1343785 cri.go:89] found id: ""
I0328 22:12:34.918357 1343785 logs.go:276] 1 containers: [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7]
I0328 22:12:34.918414 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:34.922217 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0328 22:12:34.922301 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0328 22:12:34.963463 1343785 cri.go:89] found id: "c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:34.963485 1343785 cri.go:89] found id: ""
I0328 22:12:34.963492 1343785 logs.go:276] 1 containers: [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5]
I0328 22:12:34.963554 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:34.967897 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0328 22:12:34.967976 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0328 22:12:35.008964 1343785 cri.go:89] found id: "02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:35.008989 1343785 cri.go:89] found id: ""
I0328 22:12:35.008998 1343785 logs.go:276] 1 containers: [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43]
I0328 22:12:35.009072 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.014679 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0328 22:12:35.014762 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0328 22:12:35.055252 1343785 cri.go:89] found id: "b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:35.055276 1343785 cri.go:89] found id: ""
I0328 22:12:35.055285 1343785 logs.go:276] 1 containers: [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208]
I0328 22:12:35.055344 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.058889 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0328 22:12:35.058971 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0328 22:12:35.101541 1343785 cri.go:89] found id: "2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:35.101566 1343785 cri.go:89] found id: ""
I0328 22:12:35.101574 1343785 logs.go:276] 1 containers: [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5]
I0328 22:12:35.101632 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.105796 1343785 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0328 22:12:35.105877 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0328 22:12:35.150310 1343785 cri.go:89] found id: "4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:35.150335 1343785 cri.go:89] found id: "3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:35.150340 1343785 cri.go:89] found id: ""
I0328 22:12:35.150348 1343785 logs.go:276] 2 containers: [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a]
I0328 22:12:35.150436 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.154508 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.158266 1343785 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0328 22:12:35.158340 1343785 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0328 22:12:35.205344 1343785 cri.go:89] found id: "1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:35.205369 1343785 cri.go:89] found id: ""
I0328 22:12:35.205376 1343785 logs.go:276] 1 containers: [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15]
I0328 22:12:35.205458 1343785 ssh_runner.go:195] Run: which crictl
I0328 22:12:35.209503 1343785 logs.go:123] Gathering logs for dmesg ...
I0328 22:12:35.209576 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0328 22:12:35.228942 1343785 logs.go:123] Gathering logs for kube-scheduler [c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5] ...
I0328 22:12:35.228974 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c0438a8c8a7c6ee24b7206071fa6b63dd553fbf14f4fd558528c57917097cde5"
I0328 22:12:35.272370 1343785 logs.go:123] Gathering logs for storage-provisioner [3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a] ...
I0328 22:12:35.272403 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3bdeb47065970683bd1984cb42566138e7d30672fa0b08df7df8377ceb1c962a"
I0328 22:12:35.311726 1343785 logs.go:123] Gathering logs for describe nodes ...
I0328 22:12:35.311799 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.30.0-beta.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0328 22:12:35.440684 1343785 logs.go:123] Gathering logs for etcd [77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4] ...
I0328 22:12:35.440717 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77a89f4b79b5e13c02662c8ba6f6f3a922d205bf39b5c63ab0720b597e5ad2f4"
I0328 22:12:35.489277 1343785 logs.go:123] Gathering logs for kubernetes-dashboard [1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15] ...
I0328 22:12:35.489307 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1964cd4dcb928edc5fb3ff4cd4a8a1b6afca96dbe666d2a6c6308a962a04fe15"
I0328 22:12:35.535305 1343785 logs.go:123] Gathering logs for CRI-O ...
I0328 22:12:35.535381 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0328 22:12:35.613556 1343785 logs.go:123] Gathering logs for container status ...
I0328 22:12:35.613590 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0328 22:12:35.662071 1343785 logs.go:123] Gathering logs for kubelet ...
I0328 22:12:35.662100 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0328 22:12:35.690147 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:35.690377 1343785 logs.go:138] Found kubelet problem: Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:35.749307 1343785 logs.go:123] Gathering logs for kube-proxy [02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43] ...
I0328 22:12:35.749344 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 02c2f0a0149bac38e3cc7ba146663278ece69b0c2c72593414ea1aa8114deb43"
I0328 22:12:35.788679 1343785 logs.go:123] Gathering logs for kube-controller-manager [b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208] ...
I0328 22:12:35.788710 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4dfc8a81959e1e54f5ccd7dbce5e5b79e2e50ea3eb9c6ef60ed4cc3619a0208"
I0328 22:12:35.850080 1343785 logs.go:123] Gathering logs for kube-apiserver [2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37] ...
I0328 22:12:35.850120 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2802d01750454277c58463803cc59154ce9ed35544da7d3b40387123a0807b37"
I0328 22:12:35.922295 1343785 logs.go:123] Gathering logs for coredns [ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7] ...
I0328 22:12:35.922331 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ad1c78378fd080073e6d7b4adcb4ebcd3512cdf786702dc1c9708c29fd3c43b7"
I0328 22:12:35.964277 1343785 logs.go:123] Gathering logs for kindnet [2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5] ...
I0328 22:12:35.964304 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2629b02a9c3dc6762ccb0b6979179bee48869c9043cf2d708a2c66592fb3c3b5"
I0328 22:12:36.010621 1343785 logs.go:123] Gathering logs for storage-provisioner [4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2] ...
I0328 22:12:36.010658 1343785 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4638f483f335cf1696691ad4ba3f3dc515c5e7c24fb50964ab418406966667e2"
I0328 22:12:36.058876 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:36.058952 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0328 22:12:36.059039 1343785 out.go:239] X Problems detected in kubelet:
W0328 22:12:36.059084 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: W0328 22:08:17.200950 750 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
W0328 22:12:36.059135 1343785 out.go:239] Mar 28 22:08:17 no-preload-363849 kubelet[750]: E0328 22:08:17.201000 750 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-363849" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-363849' and this object
I0328 22:12:36.059189 1343785 out.go:304] Setting ErrFile to fd 2...
I0328 22:12:36.059209 1343785 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0328 22:12:40.727728 1338826 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0328 22:12:40.736874 1338826 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0328 22:12:40.739297 1338826 out.go:177]
W0328 22:12:40.741210 1338826 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0328 22:12:40.741248 1338826 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0328 22:12:40.741269 1338826 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0328 22:12:40.741277 1338826 out.go:239] *
W0328 22:12:40.742279 1338826 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0328 22:12:40.744622 1338826 out.go:177]
==> CRI-O <==
Mar 28 22:10:16 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:16.235825584Z" level=info msg="Removed container 3831dcc3bec2d0a81b186e95090c3fe03274886b562c0b66183b0c047d9b71e4: kubernetes-dashboard/dashboard-metrics-scraper-8d5bb5db8-z2tdh/dashboard-metrics-scraper" id=2b84a530-df89-4dc9-bd11-a318680e3831 name=/runtime.v1alpha2.RuntimeService/RemoveContainer
Mar 28 22:10:16 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:16.687666743Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=b5685d81-810a-4e4f-91e6-a2cf4ea5dc0f name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:16 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:16.687900825Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=b5685d81-810a-4e4f-91e6-a2cf4ea5dc0f name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:29 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:29.687500903Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=2113241f-1cd0-4543-81ee-d50ea80daf45 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:29 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:29.687763695Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=2113241f-1cd0-4543-81ee-d50ea80daf45 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:40 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:40.687786651Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=1dd46c99-c541-4616-8c72-02bb8266cdb7 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:40 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:40.688020701Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=1dd46c99-c541-4616-8c72-02bb8266cdb7 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:52 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:52.687517149Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=014d584d-f967-44ed-bae3-361ee4515587 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:10:52 old-k8s-version-633693 crio[624]: time="2024-03-28 22:10:52.687752996Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=014d584d-f967-44ed-bae3-361ee4515587 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:07 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:07.687733793Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=1c2ba994-cfc9-4389-bcf0-4b81929dd556 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:07 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:07.687968745Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=1c2ba994-cfc9-4389-bcf0-4b81929dd556 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:21 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:21.687483586Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=83a8b7f1-a71a-412b-9813-cc2c8d0290cf name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:21 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:21.687708569Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=83a8b7f1-a71a-412b-9813-cc2c8d0290cf name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:36 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:36.687486735Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=4aca201e-819c-4385-8324-4d7a6d2d6d3a name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:36 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:36.687718651Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=4aca201e-819c-4385-8324-4d7a6d2d6d3a name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:37 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:37.735687797Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.2" id=df9ebd51-7d04-4ace-ae9d-ae486e6df83f name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:37 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:37.735933178Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2a060e2e7101d419352bf82c613158587400be743482d9a537ec4a9d1b4eb93c,RepoTags:[k8s.gcr.io/pause:3.2 registry.k8s.io/pause:3.2],RepoDigests:[k8s.gcr.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f registry.k8s.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 registry.k8s.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f],Size_:489397,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=df9ebd51-7d04-4ace-ae9d-ae486e6df83f name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:51 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:51.688002610Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=d9bc7e68-96cb-47bb-94df-b5601762ab01 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:11:51 old-k8s-version-633693 crio[624]: time="2024-03-28 22:11:51.688912503Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=d9bc7e68-96cb-47bb-94df-b5601762ab01 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:02 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:02.687540585Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=fa883e51-fbf8-4209-9d57-35554b411001 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:02 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:02.687768218Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=fa883e51-fbf8-4209-9d57-35554b411001 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:17 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:17.688645888Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=4857a7aa-a3ed-490b-bac0-0b15c0ffec9d name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:17 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:17.689110968Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=4857a7aa-a3ed-490b-bac0-0b15c0ffec9d name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:30 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:30.688843471Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=82ec3e1c-b590-4205-8039-0e2c5386e7d4 name=/runtime.v1alpha2.ImageService/ImageStatus
Mar 28 22:12:30 old-k8s-version-633693 crio[624]: time="2024-03-28 22:12:30.689151588Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=82ec3e1c-b590-4205-8039-0e2c5386e7d4 name=/runtime.v1alpha2.ImageService/ImageStatus
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
cc607e36d8914 a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 2 minutes ago Exited dashboard-metrics-scraper 5 3457d31664405 dashboard-metrics-scraper-8d5bb5db8-z2tdh
dbebe9c9214b8 ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Running storage-provisioner 1 c93c82b167969 storage-provisioner
dcb78e1b74650 docker.io/kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93 5 minutes ago Running kubernetes-dashboard 0 117fb2e21ac1b kubernetes-dashboard-cd95d586-ckjdn
9a5cc20f063ad db91994f4ee8f894a1e8a6c1a76f615da8fc3c019300a3686291ce6fcbc57895 5 minutes ago Running coredns 0 b3bf2ede41d2a coredns-74ff55c5b-rq6t8
99963dd8c6223 25a5233254979d0678a2db1d15b76b73dc380d81bc5eed93916ba5638b3cd894 5 minutes ago Running kube-proxy 0 1ed9b3b46e264 kube-proxy-9vs8r
de9034878b9e7 ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Exited storage-provisioner 0 c93c82b167969 storage-provisioner
0ca17af26be7e 1611cd07b61d57dbbfebe6db242513fd51e1c02d20ba08af17a45837d86a8a8c 5 minutes ago Running busybox 0 16f14e1362953 busybox
f410c25189f5f 4740c1948d3fceb8d7dacc63033aa6299d80794ee4f4811539ec1081d9211f3d 5 minutes ago Running kindnet-cni 0 bc5a05d743cb5 kindnet-md4h4
aafa2b2860b7d 2c08bbbc02d3aa5dfbf4e79f15c0a61424049288917aa10364464ca1f7de7157 6 minutes ago Running kube-apiserver 0 622f934704238 kube-apiserver-old-k8s-version-633693
accf5dfc23ea0 1df8a2b116bd16f7070fd383a6769c8d644b365575e8ffa3e492b84e4f05fc74 6 minutes ago Running kube-controller-manager 0 6592fa42f41e8 kube-controller-manager-old-k8s-version-633693
54454ca60825d 05b738aa1bc6355db8a2ee8639f3631b908286e43f584a3d2ee0c472de033c28 6 minutes ago Running etcd 0 108fd5b0c4944 etcd-old-k8s-version-633693
0c1c9dee5f4ce e7605f88f17d6a4c3f083ef9c6f5f19b39f87e4d4406a05a8612b54a6ea57051 6 minutes ago Running kube-scheduler 0 80f6dddfa44ca kube-scheduler-old-k8s-version-633693
==> coredns [9a5cc20f063ad7f3789223916297cb6dbac2af4ce58e82d5bde29f4655036962] <==
I0328 22:07:21.732859 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-28 22:06:51.730641281 +0000 UTC m=+0.032486899) (total time: 30.002091471s):
Trace[2019727887]: [30.002091471s] [30.002091471s] END
E0328 22:07:21.736287 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0328 22:07:21.733123 1 trace.go:116] Trace[1427131847]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-28 22:06:51.731367528 +0000 UTC m=+0.033213155) (total time: 30.001735543s):
Trace[1427131847]: [30.001735543s] [30.001735543s] END
I0328 22:07:21.736222 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-03-28 22:06:51.731177293 +0000 UTC m=+0.033022912) (total time: 30.005007209s):
Trace[939984059]: [30.005007209s] [30.005007209s] END
E0328 22:07:21.736637 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
E0328 22:07:21.736625 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:52564 - 62559 "HINFO IN 6871333457023326095.2102777586334840463. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.027615718s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:52738 - 53427 "HINFO IN 7547922539308925481.753229652630310616. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.062798985s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> describe nodes <==
Name: old-k8s-version-633693
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-633693
kubernetes.io/os=linux
minikube.k8s.io/commit=2883ffbf70a3cdb38617e0fd1a9bb421b3d79967
minikube.k8s.io/name=old-k8s-version-633693
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_03_28T22_04_23_0700
minikube.k8s.io/version=v1.33.0-beta.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 28 Mar 2024 22:04:19 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-633693
AcquireTime: <unset>
RenewTime: Thu, 28 Mar 2024 22:12:40 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 28 Mar 2024 22:12:40 +0000 Thu, 28 Mar 2024 22:04:14 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 28 Mar 2024 22:12:40 +0000 Thu, 28 Mar 2024 22:04:14 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 28 Mar 2024 22:12:40 +0000 Thu, 28 Mar 2024 22:04:14 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 28 Mar 2024 22:12:40 +0000 Thu, 28 Mar 2024 22:05:21 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-633693
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022564Ki
pods: 110
System Info:
Machine ID: 5e11c1992c8e4398b6f23ee558d7ede7
System UUID: 50ff53cc-2070-40db-a887-822f52e8dbfa
Boot ID: 18dd0f92-d332-41a7-aacd-d07143d316b2
Kernel Version: 5.15.0-1056-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: cri-o://1.24.6
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m45s
kube-system coredns-74ff55c5b-rq6t8 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m3s
kube-system etcd-old-k8s-version-633693 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m11s
kube-system kindnet-md4h4 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m3s
kube-system kube-apiserver-old-k8s-version-633693 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m11s
kube-system kube-controller-manager-old-k8s-version-633693 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m11s
kube-system kube-proxy-9vs8r 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m3s
kube-system kube-scheduler-old-k8s-version-633693 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m11s
kube-system metrics-server-9975d5f86-h5ts8 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m34s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m1s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-z2tdh 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m35s
kubernetes-dashboard kubernetes-dashboard-cd95d586-ckjdn 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m35s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 8m11s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m11s kubelet Node old-k8s-version-633693 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m11s kubelet Node old-k8s-version-633693 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m11s kubelet Node old-k8s-version-633693 status is now: NodeHasSufficientPID
Normal Starting 8m1s kube-proxy Starting kube-proxy.
Normal NodeReady 7m21s kubelet Node old-k8s-version-633693 status is now: NodeReady
Normal Starting 6m5s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m5s (x8 over 6m5s) kubelet Node old-k8s-version-633693 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m5s (x8 over 6m5s) kubelet Node old-k8s-version-633693 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m5s (x8 over 6m5s) kubelet Node old-k8s-version-633693 status is now: NodeHasSufficientPID
Normal Starting 5m51s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.001077] FS-Cache: N-key=[8] 'eb405c0100000000'
[ +0.002412] FS-Cache: Duplicate cookie detected
[ +0.000778] FS-Cache: O-cookie c=0000004e [p=0000004b fl=226 nc=0 na=1]
[ +0.001057] FS-Cache: O-cookie d=00000000d4269778{9p.inode} n=000000005495c988
[ +0.001112] FS-Cache: O-key=[8] 'eb405c0100000000'
[ +0.000707] FS-Cache: N-cookie c=00000055 [p=0000004b fl=2 nc=0 na=1]
[ +0.000932] FS-Cache: N-cookie d=00000000d4269778{9p.inode} n=00000000b20bf3f1
[ +0.001084] FS-Cache: N-key=[8] 'eb405c0100000000'
[ +2.196103] FS-Cache: Duplicate cookie detected
[ +0.000725] FS-Cache: O-cookie c=0000004c [p=0000004b fl=226 nc=0 na=1]
[ +0.001008] FS-Cache: O-cookie d=00000000d4269778{9p.inode} n=00000000484a23fd
[ +0.001157] FS-Cache: O-key=[8] 'ea405c0100000000'
[ +0.000721] FS-Cache: N-cookie c=00000057 [p=0000004b fl=2 nc=0 na=1]
[ +0.001012] FS-Cache: N-cookie d=00000000d4269778{9p.inode} n=00000000d5b6d522
[ +0.001041] FS-Cache: N-key=[8] 'ea405c0100000000'
[ +0.350982] FS-Cache: Duplicate cookie detected
[ +0.000711] FS-Cache: O-cookie c=00000051 [p=0000004b fl=226 nc=0 na=1]
[ +0.001046] FS-Cache: O-cookie d=00000000d4269778{9p.inode} n=00000000baccf953
[ +0.001062] FS-Cache: O-key=[8] 'f0405c0100000000'
[ +0.000821] FS-Cache: N-cookie c=00000058 [p=0000004b fl=2 nc=0 na=1]
[ +0.001037] FS-Cache: N-cookie d=00000000d4269778{9p.inode} n=00000000d865c146
[ +0.001044] FS-Cache: N-key=[8] 'f0405c0100000000'
[Mar28 21:58] overlayfs: '/var/lib/containers/storage/overlay/l/7FOWQIVXOWACA56BLQVF4JJOLY' not a directory
[ +0.028912] overlayfs: '/var/lib/containers/storage/overlay/l/7FOWQIVXOWACA56BLQVF4JJOLY' not a directory
[ +0.101611] overlayfs: '/var/lib/containers/storage/overlay/l/7FOWQIVXOWACA56BLQVF4JJOLY' not a directory
==> etcd [54454ca60825d1d8afde9d3954763066222f962cd282dd4ae5776677f1d02263] <==
2024-03-28 22:08:36.503948 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:08:46.504037 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:08:56.504145 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:06.503982 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:16.507902 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:26.504056 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:36.504021 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:46.504117 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:09:56.503971 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:06.503971 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:16.503992 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:26.504043 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:36.504039 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:46.503949 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:10:56.503945 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:06.503934 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:16.504020 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:26.503949 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:36.504150 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:46.504010 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:11:56.503928 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:12:06.503936 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:12:16.504320 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:12:26.504018 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-03-28 22:12:36.503978 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
22:12:42 up 5:55, 0 users, load average: 0.28, 1.43, 2.06
Linux old-k8s-version-633693 5.15.0-1056-aws #61~20.04.1-Ubuntu SMP Wed Mar 13 17:45:04 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [f410c25189f5f56842f0d6f5e959b565b53ee6a8d5258db914882d1e38e9ab2b] <==
I0328 22:10:40.527418 1 main.go:227] handling current node
I0328 22:10:50.534515 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:10:50.534549 1 main.go:227] handling current node
I0328 22:11:00.549560 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:00.549591 1 main.go:227] handling current node
I0328 22:11:10.563873 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:10.563900 1 main.go:227] handling current node
I0328 22:11:20.573311 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:20.573349 1 main.go:227] handling current node
I0328 22:11:30.587875 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:30.587905 1 main.go:227] handling current node
I0328 22:11:40.602049 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:40.602077 1 main.go:227] handling current node
I0328 22:11:50.607661 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:11:50.607690 1 main.go:227] handling current node
I0328 22:12:00.620347 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:12:00.620375 1 main.go:227] handling current node
I0328 22:12:10.625288 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:12:10.625315 1 main.go:227] handling current node
I0328 22:12:20.630840 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:12:20.630869 1 main.go:227] handling current node
I0328 22:12:30.645087 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:12:30.645115 1 main.go:227] handling current node
I0328 22:12:40.652051 1 main.go:223] Handling node with IPs: map[192.168.76.2:{}]
I0328 22:12:40.652295 1 main.go:227] handling current node
==> kube-apiserver [aafa2b2860b7d02f5f63f8f79e578efc4b8a612e8845f3c01b98f89c881a05f5] <==
I0328 22:09:13.676511 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:09:13.676530 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0328 22:09:51.086043 1 client.go:360] parsed scheme: "passthrough"
I0328 22:09:51.086090 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:09:51.086100 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0328 22:09:52.391431 1 handler_proxy.go:102] no RequestInfo found in the context
E0328 22:09:52.391500 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0328 22:09:52.391507 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0328 22:10:28.319216 1 client.go:360] parsed scheme: "passthrough"
I0328 22:10:28.319266 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:10:28.319276 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0328 22:11:07.557977 1 client.go:360] parsed scheme: "passthrough"
I0328 22:11:07.558042 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:11:07.558051 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0328 22:11:38.387259 1 client.go:360] parsed scheme: "passthrough"
I0328 22:11:38.387307 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:11:38.387316 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0328 22:11:48.921809 1 handler_proxy.go:102] no RequestInfo found in the context
E0328 22:11:48.921983 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0328 22:11:48.921997 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0328 22:12:16.314600 1 client.go:360] parsed scheme: "passthrough"
I0328 22:12:16.314639 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0328 22:12:16.314648 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [accf5dfc23ea01d34e6a9b748711dcadd8f2af898f72bbc4348b539768308591] <==
W0328 22:08:14.932376 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:08:39.185632 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:08:46.582968 1 request.go:655] Throttling request took 1.04849168s, request: GET:https://192.168.76.2:8443/apis/authorization.k8s.io/v1?timeout=32s
W0328 22:08:47.434141 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:09:09.688625 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:09:19.084528 1 request.go:655] Throttling request took 1.048062666s, request: GET:https://192.168.76.2:8443/apis/networking.k8s.io/v1beta1?timeout=32s
W0328 22:09:19.942080 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:09:40.190880 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:09:51.592448 1 request.go:655] Throttling request took 1.04847541s, request: GET:https://192.168.76.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0328 22:09:52.443843 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:10:10.692737 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:10:24.094279 1 request.go:655] Throttling request took 1.048451825s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1?timeout=32s
W0328 22:10:24.945527 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:10:41.194506 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:10:56.595971 1 request.go:655] Throttling request took 1.048380995s, request: GET:https://192.168.76.2:8443/apis/authorization.k8s.io/v1beta1?timeout=32s
W0328 22:10:57.447420 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:11:11.696398 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:11:29.097825 1 request.go:655] Throttling request took 1.048417602s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0328 22:11:29.949243 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:11:42.198668 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:12:01.599601 1 request.go:655] Throttling request took 1.048262636s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0328 22:12:02.450926 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0328 22:12:12.700723 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0328 22:12:34.101321 1 request.go:655] Throttling request took 1.048180728s, request: GET:https://192.168.76.2:8443/apis/events.k8s.io/v1?timeout=32s
W0328 22:12:34.952876 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [99963dd8c6223acf3495ce9e9c35c7f4c45a97d49ba1435654dd656d82537a95] <==
I0328 22:04:41.239441 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0328 22:04:41.239543 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0328 22:04:41.251643 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0328 22:04:41.251732 1 server_others.go:185] Using iptables Proxier.
I0328 22:04:41.252027 1 server.go:650] Version: v1.20.0
I0328 22:04:41.252830 1 config.go:315] Starting service config controller
I0328 22:04:41.252846 1 shared_informer.go:240] Waiting for caches to sync for service config
I0328 22:04:41.252871 1 config.go:224] Starting endpoint slice config controller
I0328 22:04:41.252875 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0328 22:04:41.360645 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0328 22:04:41.360713 1 shared_informer.go:247] Caches are synced for service config
I0328 22:06:51.536416 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0328 22:06:51.536669 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0328 22:06:51.551158 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0328 22:06:51.551340 1 server_others.go:185] Using iptables Proxier.
I0328 22:06:51.552075 1 server.go:650] Version: v1.20.0
I0328 22:06:51.553898 1 config.go:315] Starting service config controller
I0328 22:06:51.553963 1 shared_informer.go:240] Waiting for caches to sync for service config
I0328 22:06:51.553987 1 config.go:224] Starting endpoint slice config controller
I0328 22:06:51.553991 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0328 22:06:51.660173 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0328 22:06:51.660401 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [0c1c9dee5f4ce24a9e91181634895b593f2aec0d21a11adbe16ddd1adce82d0b] <==
E0328 22:04:20.811852 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0328 22:04:20.959695 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
I0328 22:04:21.380035 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0328 22:06:41.191130 1 serving.go:331] Generated self-signed cert in-memory
W0328 22:06:47.663678 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0328 22:06:47.663705 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0328 22:06:47.663773 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0328 22:06:47.663780 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0328 22:06:47.825934 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0328 22:06:47.839769 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0328 22:06:47.840611 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0328 22:06:47.848851 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E0328 22:06:47.848623 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0328 22:06:47.879828 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0328 22:06:47.880581 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0328 22:06:47.881564 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0328 22:06:47.881680 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0328 22:06:47.881710 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0328 22:06:47.881889 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0328 22:06:47.882029 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0328 22:06:47.888143 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0328 22:06:47.892591 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0328 22:06:47.892690 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0328 22:06:47.904511 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0328 22:06:49.151631 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Mar 28 22:10:57 old-k8s-version-633693 kubelet[742]: E0328 22:10:57.687778 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:07 old-k8s-version-633693 kubelet[742]: E0328 22:11:07.688569 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: I0328 22:11:10.687074 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:11:10 old-k8s-version-633693 kubelet[742]: E0328 22:11:10.687505 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:21 old-k8s-version-633693 kubelet[742]: E0328 22:11:21.688528 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: I0328 22:11:25.687115 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:11:25 old-k8s-version-633693 kubelet[742]: E0328 22:11:25.687833 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:36 old-k8s-version-633693 kubelet[742]: E0328 22:11:36.687961 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:11:37 old-k8s-version-633693 kubelet[742]: E0328 22:11:37.743985 742 container_manager_linux.go:533] failed to find cgroups of kubelet - cpu and memory cgroup hierarchy not unified. cpu: /docker/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d, memory: /docker/fe636ffa3a97a0d6d85a45fcf40320435f4f1adeb9e541a502313138a659e23d/system.slice/kubelet.service
Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: I0328 22:11:39.687228 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:11:39 old-k8s-version-633693 kubelet[742]: E0328 22:11:39.687565 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:11:51 old-k8s-version-633693 kubelet[742]: E0328 22:11:51.689147 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: I0328 22:11:54.687105 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:11:54 old-k8s-version-633693 kubelet[742]: E0328 22:11:54.687465 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:02 old-k8s-version-633693 kubelet[742]: E0328 22:12:02.688202 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: I0328 22:12:06.687102 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:12:06 old-k8s-version-633693 kubelet[742]: E0328 22:12:06.687449 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: I0328 22:12:17.687398 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.687727 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:17 old-k8s-version-633693 kubelet[742]: E0328 22:12:17.689382 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:30 old-k8s-version-633693 kubelet[742]: I0328 22:12:30.687217 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:12:30 old-k8s-version-633693 kubelet[742]: E0328 22:12:30.687670 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
Mar 28 22:12:30 old-k8s-version-633693 kubelet[742]: E0328 22:12:30.689802 742 pod_workers.go:191] Error syncing pod fb7522bf-a2a3-485c-b715-79d144a23abd ("metrics-server-9975d5f86-h5ts8_kube-system(fb7522bf-a2a3-485c-b715-79d144a23abd)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Mar 28 22:12:41 old-k8s-version-633693 kubelet[742]: I0328 22:12:41.687199 742 scope.go:95] [topologymanager] RemoveContainer - Container ID: cc607e36d891433dcc76acec8be25b4ec9f6ac4b539a4302bb9b71c200e74ecc
Mar 28 22:12:41 old-k8s-version-633693 kubelet[742]: E0328 22:12:41.687544 742 pod_workers.go:191] Error syncing pod 48d62b74-b3d8-40ca-9a76-635584f8ffd5 ("dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-z2tdh_kubernetes-dashboard(48d62b74-b3d8-40ca-9a76-635584f8ffd5)"
==> kubernetes-dashboard [dcb78e1b7465099ed99741a848314cba3d027ecfab8e44b022dc85231ab9a26d] <==
2024/03/28 22:07:21 Using namespace: kubernetes-dashboard
2024/03/28 22:07:21 Using in-cluster config to connect to apiserver
2024/03/28 22:07:21 Using secret token for csrf signing
2024/03/28 22:07:21 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/03/28 22:07:21 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/03/28 22:07:21 Successful initial request to the apiserver, version: v1.20.0
2024/03/28 22:07:21 Generating JWE encryption key
2024/03/28 22:07:21 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/03/28 22:07:21 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/03/28 22:07:21 Initializing JWE encryption key from synchronized object
2024/03/28 22:07:21 Creating in-cluster Sidecar client
2024/03/28 22:07:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:07:21 Serving insecurely on HTTP port: 9090
2024/03/28 22:07:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:08:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:08:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:09:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:09:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:10:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:10:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:11:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:11:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:12:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/03/28 22:07:21 Starting overwatch
==> storage-provisioner [dbebe9c9214b8446dd7e1ba58af2dcdda839029f75a4df5d16fce02680f0f723] <==
I0328 22:07:22.068755 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0328 22:07:22.085681 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0328 22:07:22.085818 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0328 22:07:39.533702 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0328 22:07:39.533877 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-633693_55f8dc27-b24e-4917-9908-450b2a25a860!
I0328 22:07:39.534263 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"1b46d884-d612-44f4-b871-e29fcd11c9e4", APIVersion:"v1", ResourceVersion:"797", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-633693_55f8dc27-b24e-4917-9908-450b2a25a860 became leader
I0328 22:07:39.634313 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-633693_55f8dc27-b24e-4917-9908-450b2a25a860!
==> storage-provisioner [de9034878b9e7c8d2e40e141b1c21f2a76586667af994d8465e9601d126955e0] <==
I0328 22:05:26.391048 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0328 22:05:26.408457 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0328 22:05:26.408506 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0328 22:05:26.452948 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0328 22:05:26.453255 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-633693_3b8e925e-d2df-4db5-bed6-d59560f69ae6!
I0328 22:05:26.457828 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"1b46d884-d612-44f4-b871-e29fcd11c9e4", APIVersion:"v1", ResourceVersion:"489", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-633693_3b8e925e-d2df-4db5-bed6-d59560f69ae6 became leader
I0328 22:05:26.556969 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-633693_3b8e925e-d2df-4db5-bed6-d59560f69ae6!
I0328 22:06:51.402512 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0328 22:07:21.408539 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-633693 -n old-k8s-version-633693
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-633693 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-h5ts8
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-633693 describe pod metrics-server-9975d5f86-h5ts8
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-633693 describe pod metrics-server-9975d5f86-h5ts8: exit status 1 (88.919841ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-h5ts8" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-633693 describe pod metrics-server-9975d5f86-h5ts8: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (383.22s)