=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:254: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-999803 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:254: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-999803 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m14.019417741s)
-- stdout --
* [old-k8s-version-999803] minikube v1.35.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=20318
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/20318-888339/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/20318-888339/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.32.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.32.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-999803" primary control-plane node in "old-k8s-version-999803" cluster
* Pulling base image v0.0.46 ...
* Restarting existing docker container for "old-k8s-version-999803" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.24 ...
* Verifying Kubernetes components...
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-999803 addons enable metrics-server
* Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
-- /stdout --
** stderr **
I0127 12:09:23.286361 1099122 out.go:345] Setting OutFile to fd 1 ...
I0127 12:09:23.286558 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:09:23.286584 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:09:23.286606 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:09:23.286860 1099122 root.go:338] Updating PATH: /home/jenkins/minikube-integration/20318-888339/.minikube/bin
I0127 12:09:23.287286 1099122 out.go:352] Setting JSON to false
I0127 12:09:23.288289 1099122 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":17509,"bootTime":1737962255,"procs":200,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1075-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0127 12:09:23.288392 1099122 start.go:139] virtualization:
I0127 12:09:23.290852 1099122 out.go:177] * [old-k8s-version-999803] minikube v1.35.0 on Ubuntu 20.04 (arm64)
I0127 12:09:23.292203 1099122 out.go:177] - MINIKUBE_LOCATION=20318
I0127 12:09:23.292982 1099122 notify.go:220] Checking for updates...
I0127 12:09:23.295370 1099122 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0127 12:09:23.297468 1099122 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:09:23.298826 1099122 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/20318-888339/.minikube
I0127 12:09:23.300298 1099122 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0127 12:09:23.301848 1099122 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0127 12:09:23.304008 1099122 config.go:182] Loaded profile config "old-k8s-version-999803": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0127 12:09:23.306489 1099122 out.go:177] * Kubernetes 1.32.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.32.1
I0127 12:09:23.308010 1099122 driver.go:394] Setting default libvirt URI to qemu:///system
I0127 12:09:23.341499 1099122 docker.go:123] docker version: linux-27.5.1:Docker Engine - Community
I0127 12:09:23.341627 1099122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0127 12:09:23.415401 1099122 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:50 OomKillDisable:true NGoroutines:69 SystemTime:2025-01-27 12:09:23.403497453 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1075-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.5.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb Expected:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb} RuncCommit:{ID:v1.2.4-0-g6c52b3f Expected:v1.2.4-0-g6c52b3f} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.20.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.32.4]] Warnings:<nil>}}
I0127 12:09:23.415525 1099122 docker.go:318] overlay module found
I0127 12:09:23.418120 1099122 out.go:177] * Using the docker driver based on existing profile
I0127 12:09:23.420474 1099122 start.go:297] selected driver: docker
I0127 12:09:23.420493 1099122 start.go:901] validating driver "docker" against &{Name:old-k8s-version-999803 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-999803 Namespace:default APIServerHAVIP: APIServerName:minikubeCA A
PIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/je
nkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:09:23.420649 1099122 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0127 12:09:23.421433 1099122 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0127 12:09:23.505115 1099122 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:50 OomKillDisable:true NGoroutines:69 SystemTime:2025-01-27 12:09:23.4935221 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1075-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.5.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb Expected:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb} RuncCommit:{ID:v1.2.4-0-g6c52b3f Expected:v1.2.4-0-g6c52b3f} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErrors
:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.20.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.32.4]] Warnings:<nil>}}
I0127 12:09:23.505497 1099122 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0127 12:09:23.505529 1099122 cni.go:84] Creating CNI manager for ""
I0127 12:09:23.505571 1099122 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0127 12:09:23.505613 1099122 start.go:340] cluster config:
{Name:old-k8s-version-999803 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-999803 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:contai
nerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize
:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:09:23.508558 1099122 out.go:177] * Starting "old-k8s-version-999803" primary control-plane node in "old-k8s-version-999803" cluster
I0127 12:09:23.511228 1099122 cache.go:121] Beginning downloading kic base image for docker with containerd
I0127 12:09:23.513953 1099122 out.go:177] * Pulling base image v0.0.46 ...
I0127 12:09:23.516528 1099122 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0127 12:09:23.516594 1099122 preload.go:146] Found local preload: /home/jenkins/minikube-integration/20318-888339/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0127 12:09:23.516596 1099122 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 in local docker daemon
I0127 12:09:23.516603 1099122 cache.go:56] Caching tarball of preloaded images
I0127 12:09:23.516765 1099122 preload.go:172] Found /home/jenkins/minikube-integration/20318-888339/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0127 12:09:23.516776 1099122 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0127 12:09:23.516892 1099122 profile.go:143] Saving config to /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/config.json ...
I0127 12:09:23.545993 1099122 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 in local docker daemon, skipping pull
I0127 12:09:23.546020 1099122 cache.go:145] gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 exists in daemon, skipping load
I0127 12:09:23.546038 1099122 cache.go:227] Successfully downloaded all kic artifacts
I0127 12:09:23.546070 1099122 start.go:360] acquireMachinesLock for old-k8s-version-999803: {Name:mkb5231a94ab32a7dfe061b82488c125adaefd6b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:09:23.546136 1099122 start.go:364] duration metric: took 43.109µs to acquireMachinesLock for "old-k8s-version-999803"
I0127 12:09:23.546159 1099122 start.go:96] Skipping create...Using existing machine configuration
I0127 12:09:23.546171 1099122 fix.go:54] fixHost starting:
I0127 12:09:23.546437 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:23.567440 1099122 fix.go:112] recreateIfNeeded on old-k8s-version-999803: state=Stopped err=<nil>
W0127 12:09:23.567473 1099122 fix.go:138] unexpected machine state, will restart: <nil>
I0127 12:09:23.571027 1099122 out.go:177] * Restarting existing docker container for "old-k8s-version-999803" ...
I0127 12:09:23.573591 1099122 cli_runner.go:164] Run: docker start old-k8s-version-999803
I0127 12:09:23.938557 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:23.963783 1099122 kic.go:430] container "old-k8s-version-999803" state is running.
I0127 12:09:23.964166 1099122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-999803
I0127 12:09:23.988419 1099122 profile.go:143] Saving config to /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/config.json ...
I0127 12:09:23.988642 1099122 machine.go:93] provisionDockerMachine start ...
I0127 12:09:23.988697 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:24.016470 1099122 main.go:141] libmachine: Using SSH client type: native
I0127 12:09:24.016815 1099122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33857 <nil> <nil>}
I0127 12:09:24.016837 1099122 main.go:141] libmachine: About to run SSH command:
hostname
I0127 12:09:24.017596 1099122 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0127 12:09:27.148911 1099122 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-999803
I0127 12:09:27.148938 1099122 ubuntu.go:169] provisioning hostname "old-k8s-version-999803"
I0127 12:09:27.149017 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:27.182448 1099122 main.go:141] libmachine: Using SSH client type: native
I0127 12:09:27.182744 1099122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33857 <nil> <nil>}
I0127 12:09:27.182762 1099122 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-999803 && echo "old-k8s-version-999803" | sudo tee /etc/hostname
I0127 12:09:27.321909 1099122 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-999803
I0127 12:09:27.322049 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:27.346273 1099122 main.go:141] libmachine: Using SSH client type: native
I0127 12:09:27.346527 1099122 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33857 <nil> <nil>}
I0127 12:09:27.346555 1099122 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-999803' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-999803/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-999803' | sudo tee -a /etc/hosts;
fi
fi
I0127 12:09:27.472924 1099122 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0127 12:09:27.472966 1099122 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/20318-888339/.minikube CaCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/20318-888339/.minikube}
I0127 12:09:27.472988 1099122 ubuntu.go:177] setting up certificates
I0127 12:09:27.472998 1099122 provision.go:84] configureAuth start
I0127 12:09:27.473097 1099122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-999803
I0127 12:09:27.503556 1099122 provision.go:143] copyHostCerts
I0127 12:09:27.503619 1099122 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem, removing ...
I0127 12:09:27.503627 1099122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem
I0127 12:09:27.503698 1099122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem (1082 bytes)
I0127 12:09:27.503797 1099122 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem, removing ...
I0127 12:09:27.503802 1099122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem
I0127 12:09:27.503830 1099122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem (1123 bytes)
I0127 12:09:27.503887 1099122 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem, removing ...
I0127 12:09:27.503892 1099122 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem
I0127 12:09:27.503948 1099122 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem (1675 bytes)
I0127 12:09:27.504000 1099122 provision.go:117] generating server cert: /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-999803 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-999803]
I0127 12:09:28.346559 1099122 provision.go:177] copyRemoteCerts
I0127 12:09:28.346675 1099122 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0127 12:09:28.346748 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:28.374775 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:28.474505 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0127 12:09:28.503951 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0127 12:09:28.536434 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0127 12:09:28.583474 1099122 provision.go:87] duration metric: took 1.110460237s to configureAuth
I0127 12:09:28.583502 1099122 ubuntu.go:193] setting minikube options for container-runtime
I0127 12:09:28.583694 1099122 config.go:182] Loaded profile config "old-k8s-version-999803": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0127 12:09:28.583707 1099122 machine.go:96] duration metric: took 4.595057547s to provisionDockerMachine
I0127 12:09:28.583715 1099122 start.go:293] postStartSetup for "old-k8s-version-999803" (driver="docker")
I0127 12:09:28.583725 1099122 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0127 12:09:28.583779 1099122 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0127 12:09:28.583823 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:28.623674 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:28.729593 1099122 ssh_runner.go:195] Run: cat /etc/os-release
I0127 12:09:28.737167 1099122 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0127 12:09:28.737204 1099122 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0127 12:09:28.737215 1099122 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0127 12:09:28.737222 1099122 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0127 12:09:28.737235 1099122 filesync.go:126] Scanning /home/jenkins/minikube-integration/20318-888339/.minikube/addons for local assets ...
I0127 12:09:28.737295 1099122 filesync.go:126] Scanning /home/jenkins/minikube-integration/20318-888339/.minikube/files for local assets ...
I0127 12:09:28.737372 1099122 filesync.go:149] local asset: /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem -> 8937152.pem in /etc/ssl/certs
I0127 12:09:28.737475 1099122 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0127 12:09:28.749110 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem --> /etc/ssl/certs/8937152.pem (1708 bytes)
I0127 12:09:28.800541 1099122 start.go:296] duration metric: took 216.810668ms for postStartSetup
I0127 12:09:28.800623 1099122 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0127 12:09:28.800680 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:28.846652 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:28.966791 1099122 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0127 12:09:28.986523 1099122 fix.go:56] duration metric: took 5.440348943s for fixHost
I0127 12:09:28.986551 1099122 start.go:83] releasing machines lock for "old-k8s-version-999803", held for 5.440403367s
I0127 12:09:28.986632 1099122 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-999803
I0127 12:09:29.043546 1099122 ssh_runner.go:195] Run: cat /version.json
I0127 12:09:29.043609 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:29.043609 1099122 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0127 12:09:29.043677 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:29.091513 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:29.101587 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:29.377301 1099122 ssh_runner.go:195] Run: systemctl --version
I0127 12:09:29.382181 1099122 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0127 12:09:29.389631 1099122 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0127 12:09:29.409254 1099122 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0127 12:09:29.409328 1099122 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0127 12:09:29.420280 1099122 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0127 12:09:29.420301 1099122 start.go:495] detecting cgroup driver to use...
I0127 12:09:29.420332 1099122 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0127 12:09:29.420386 1099122 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0127 12:09:29.441810 1099122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0127 12:09:29.468770 1099122 docker.go:217] disabling cri-docker service (if available) ...
I0127 12:09:29.468862 1099122 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0127 12:09:29.489484 1099122 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0127 12:09:29.524146 1099122 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0127 12:09:29.742846 1099122 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0127 12:09:30.032138 1099122 docker.go:233] disabling docker service ...
I0127 12:09:30.032219 1099122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0127 12:09:30.111880 1099122 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0127 12:09:30.171407 1099122 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0127 12:09:30.314282 1099122 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0127 12:09:30.434053 1099122 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0127 12:09:30.453712 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0127 12:09:30.474448 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0127 12:09:30.487544 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0127 12:09:30.497807 1099122 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0127 12:09:30.497877 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0127 12:09:30.525869 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0127 12:09:30.553630 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0127 12:09:30.577099 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0127 12:09:30.605591 1099122 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0127 12:09:30.617164 1099122 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0127 12:09:30.632417 1099122 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0127 12:09:30.644125 1099122 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0127 12:09:30.654945 1099122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:09:30.787916 1099122 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0127 12:09:31.017179 1099122 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0127 12:09:31.017248 1099122 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0127 12:09:31.024802 1099122 start.go:563] Will wait 60s for crictl version
I0127 12:09:31.024869 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:09:31.028937 1099122 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0127 12:09:31.100051 1099122 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.24
RuntimeApiVersion: v1
I0127 12:09:31.100131 1099122 ssh_runner.go:195] Run: containerd --version
I0127 12:09:31.146811 1099122 ssh_runner.go:195] Run: containerd --version
I0127 12:09:31.173670 1099122 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.24 ...
I0127 12:09:31.175120 1099122 cli_runner.go:164] Run: docker network inspect old-k8s-version-999803 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0127 12:09:31.211498 1099122 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0127 12:09:31.215529 1099122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0127 12:09:31.226955 1099122 kubeadm.go:883] updating cluster {Name:old-k8s-version-999803 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-999803 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[]
APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-h
ost Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0127 12:09:31.227069 1099122 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0127 12:09:31.227131 1099122 ssh_runner.go:195] Run: sudo crictl images --output json
I0127 12:09:31.282226 1099122 containerd.go:627] all images are preloaded for containerd runtime.
I0127 12:09:31.282246 1099122 containerd.go:534] Images already preloaded, skipping extraction
I0127 12:09:31.282304 1099122 ssh_runner.go:195] Run: sudo crictl images --output json
I0127 12:09:31.340833 1099122 containerd.go:627] all images are preloaded for containerd runtime.
I0127 12:09:31.340901 1099122 cache_images.go:84] Images are preloaded, skipping loading
I0127 12:09:31.340922 1099122 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 containerd true true} ...
I0127 12:09:31.341084 1099122 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-999803 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-999803 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0127 12:09:31.341182 1099122 ssh_runner.go:195] Run: sudo crictl info
I0127 12:09:31.409559 1099122 cni.go:84] Creating CNI manager for ""
I0127 12:09:31.409581 1099122 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0127 12:09:31.409596 1099122 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0127 12:09:31.409618 1099122 kubeadm.go:189] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-999803 NodeName:old-k8s-version-999803 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0127 12:09:31.409742 1099122 kubeadm.go:195] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-999803"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0127 12:09:31.409807 1099122 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0127 12:09:31.419450 1099122 binaries.go:44] Found k8s binaries, skipping transfer
I0127 12:09:31.419571 1099122 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0127 12:09:31.429011 1099122 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0127 12:09:31.448157 1099122 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0127 12:09:31.467193 1099122 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0127 12:09:31.485896 1099122 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0127 12:09:31.489817 1099122 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0127 12:09:31.500650 1099122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:09:31.604587 1099122 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0127 12:09:31.621052 1099122 certs.go:68] Setting up /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803 for IP: 192.168.76.2
I0127 12:09:31.621120 1099122 certs.go:194] generating shared ca certs ...
I0127 12:09:31.621149 1099122 certs.go:226] acquiring lock for ca certs: {Name:mke15f79704ae0e83f911aa0e3f9c4b862da9341 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:09:31.621347 1099122 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/20318-888339/.minikube/ca.key
I0127 12:09:31.621433 1099122 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.key
I0127 12:09:31.621459 1099122 certs.go:256] generating profile certs ...
I0127 12:09:31.621604 1099122 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/client.key
I0127 12:09:31.621712 1099122 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/apiserver.key.10aba280
I0127 12:09:31.621799 1099122 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/proxy-client.key
I0127 12:09:31.622072 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715.pem (1338 bytes)
W0127 12:09:31.622128 1099122 certs.go:480] ignoring /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715_empty.pem, impossibly tiny 0 bytes
I0127 12:09:31.622165 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem (1679 bytes)
I0127 12:09:31.622215 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem (1082 bytes)
I0127 12:09:31.622273 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem (1123 bytes)
I0127 12:09:31.622333 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem (1675 bytes)
I0127 12:09:31.622413 1099122 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem (1708 bytes)
I0127 12:09:31.623141 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0127 12:09:31.706726 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0127 12:09:31.799931 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0127 12:09:31.834885 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0127 12:09:31.861207 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0127 12:09:31.891039 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0127 12:09:31.916614 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0127 12:09:31.943387 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/old-k8s-version-999803/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0127 12:09:31.975203 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0127 12:09:32.003272 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715.pem --> /usr/share/ca-certificates/893715.pem (1338 bytes)
I0127 12:09:32.034780 1099122 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem --> /usr/share/ca-certificates/8937152.pem (1708 bytes)
I0127 12:09:32.068572 1099122 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0127 12:09:32.087431 1099122 ssh_runner.go:195] Run: openssl version
I0127 12:09:32.094084 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0127 12:09:32.103576 1099122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0127 12:09:32.107900 1099122 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jan 27 11:23 /usr/share/ca-certificates/minikubeCA.pem
I0127 12:09:32.107971 1099122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0127 12:09:32.115400 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0127 12:09:32.124714 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/893715.pem && ln -fs /usr/share/ca-certificates/893715.pem /etc/ssl/certs/893715.pem"
I0127 12:09:32.134981 1099122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/893715.pem
I0127 12:09:32.139173 1099122 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jan 27 11:31 /usr/share/ca-certificates/893715.pem
I0127 12:09:32.139236 1099122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/893715.pem
I0127 12:09:32.147615 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/893715.pem /etc/ssl/certs/51391683.0"
I0127 12:09:32.158978 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/8937152.pem && ln -fs /usr/share/ca-certificates/8937152.pem /etc/ssl/certs/8937152.pem"
I0127 12:09:32.170491 1099122 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/8937152.pem
I0127 12:09:32.174348 1099122 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jan 27 11:31 /usr/share/ca-certificates/8937152.pem
I0127 12:09:32.174479 1099122 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/8937152.pem
I0127 12:09:32.183063 1099122 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/8937152.pem /etc/ssl/certs/3ec20f2e.0"
I0127 12:09:32.193473 1099122 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0127 12:09:32.198197 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0127 12:09:32.206517 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0127 12:09:32.217890 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0127 12:09:32.225231 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0127 12:09:32.232441 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0127 12:09:32.239625 1099122 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0127 12:09:32.246630 1099122 kubeadm.go:392] StartCluster: {Name:old-k8s-version-999803 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-999803 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] API
ServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host
Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:09:32.246792 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0127 12:09:32.246883 1099122 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0127 12:09:32.306653 1099122 cri.go:89] found id: "006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:09:32.306729 1099122 cri.go:89] found id: "92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:09:32.306748 1099122 cri.go:89] found id: "ad07ff871a074a1b4a7dabc075bef03722ba87f36572ade75097c7f0336caa7b"
I0127 12:09:32.306773 1099122 cri.go:89] found id: "69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:09:32.306817 1099122 cri.go:89] found id: "f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:09:32.306874 1099122 cri.go:89] found id: "24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:09:32.306932 1099122 cri.go:89] found id: "8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:09:32.306957 1099122 cri.go:89] found id: "2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:09:32.306975 1099122 cri.go:89] found id: ""
I0127 12:09:32.307053 1099122 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
W0127 12:09:32.328064 1099122 kubeadm.go:399] unpause failed: list paused: runc: sudo runc --root /run/containerd/runc/k8s.io list -f json: Process exited with status 1
stdout:
stderr:
time="2025-01-27T12:09:32Z" level=error msg="open /run/containerd/runc/k8s.io: no such file or directory"
I0127 12:09:32.328191 1099122 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0127 12:09:32.343347 1099122 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0127 12:09:32.343418 1099122 kubeadm.go:593] restartPrimaryControlPlane start ...
I0127 12:09:32.343515 1099122 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0127 12:09:32.355944 1099122 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0127 12:09:32.356480 1099122 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-999803" does not appear in /home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:09:32.356650 1099122 kubeconfig.go:62] /home/jenkins/minikube-integration/20318-888339/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-999803" cluster setting kubeconfig missing "old-k8s-version-999803" context setting]
I0127 12:09:32.357075 1099122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/20318-888339/kubeconfig: {Name:mk75ddd380b783b9f157e482ffdcc29dbd635876 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:09:32.358768 1099122 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0127 12:09:32.369137 1099122 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0127 12:09:32.369218 1099122 kubeadm.go:597] duration metric: took 25.779057ms to restartPrimaryControlPlane
I0127 12:09:32.369242 1099122 kubeadm.go:394] duration metric: took 122.620377ms to StartCluster
I0127 12:09:32.369287 1099122 settings.go:142] acquiring lock: {Name:mk8e4620a376eeb900823ad35149c0dd6d301c83 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:09:32.369378 1099122 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:09:32.370062 1099122 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/20318-888339/kubeconfig: {Name:mk75ddd380b783b9f157e482ffdcc29dbd635876 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:09:32.370310 1099122 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0127 12:09:32.370844 1099122 config.go:182] Loaded profile config "old-k8s-version-999803": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0127 12:09:32.370818 1099122 addons.go:511] enable addons start: toEnable=map[ambassador:false amd-gpu-device-plugin:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0127 12:09:32.371172 1099122 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-999803"
I0127 12:09:32.371209 1099122 addons.go:238] Setting addon storage-provisioner=true in "old-k8s-version-999803"
W0127 12:09:32.371245 1099122 addons.go:247] addon storage-provisioner should already be in state true
I0127 12:09:32.371290 1099122 host.go:66] Checking if "old-k8s-version-999803" exists ...
I0127 12:09:32.371886 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:32.372063 1099122 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-999803"
I0127 12:09:32.372113 1099122 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-999803"
I0127 12:09:32.372433 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:32.372811 1099122 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-999803"
I0127 12:09:32.372830 1099122 addons.go:238] Setting addon metrics-server=true in "old-k8s-version-999803"
W0127 12:09:32.372836 1099122 addons.go:247] addon metrics-server should already be in state true
I0127 12:09:32.372858 1099122 host.go:66] Checking if "old-k8s-version-999803" exists ...
I0127 12:09:32.373465 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:32.373766 1099122 addons.go:69] Setting dashboard=true in profile "old-k8s-version-999803"
I0127 12:09:32.373803 1099122 addons.go:238] Setting addon dashboard=true in "old-k8s-version-999803"
W0127 12:09:32.373838 1099122 addons.go:247] addon dashboard should already be in state true
I0127 12:09:32.373882 1099122 host.go:66] Checking if "old-k8s-version-999803" exists ...
I0127 12:09:32.374389 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:32.376460 1099122 out.go:177] * Verifying Kubernetes components...
I0127 12:09:32.384448 1099122 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:09:32.426417 1099122 addons.go:238] Setting addon default-storageclass=true in "old-k8s-version-999803"
W0127 12:09:32.426441 1099122 addons.go:247] addon default-storageclass should already be in state true
I0127 12:09:32.426466 1099122 host.go:66] Checking if "old-k8s-version-999803" exists ...
I0127 12:09:32.426879 1099122 cli_runner.go:164] Run: docker container inspect old-k8s-version-999803 --format={{.State.Status}}
I0127 12:09:32.457567 1099122 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0127 12:09:32.458953 1099122 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0127 12:09:32.461015 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0127 12:09:32.461134 1099122 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0127 12:09:32.461217 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:32.466895 1099122 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0127 12:09:32.466957 1099122 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0127 12:09:32.468136 1099122 addons.go:435] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:09:32.468155 1099122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0127 12:09:32.468288 1099122 addons.go:435] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0127 12:09:32.468296 1099122 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0127 12:09:32.468580 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:32.469050 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:32.510920 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:32.521468 1099122 addons.go:435] installing /etc/kubernetes/addons/storageclass.yaml
I0127 12:09:32.521489 1099122 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0127 12:09:32.521551 1099122 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-999803
I0127 12:09:32.572078 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:32.584656 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:32.594969 1099122 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33857 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/old-k8s-version-999803/id_rsa Username:docker}
I0127 12:09:32.613717 1099122 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0127 12:09:32.662584 1099122 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-999803" to be "Ready" ...
I0127 12:09:32.709374 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0127 12:09:32.709397 1099122 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0127 12:09:32.763316 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0127 12:09:32.763414 1099122 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0127 12:09:32.769985 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0127 12:09:32.788867 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:09:32.811845 1099122 addons.go:435] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0127 12:09:32.811871 1099122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0127 12:09:32.860268 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0127 12:09:32.860293 1099122 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0127 12:09:32.861629 1099122 addons.go:435] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0127 12:09:32.861648 1099122 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0127 12:09:32.922819 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0127 12:09:32.922844 1099122 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0127 12:09:32.931379 1099122 addons.go:435] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:09:32.931405 1099122 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0127 12:09:33.009749 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-role.yaml
I0127 12:09:33.009777 1099122 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0127 12:09:33.028136 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:09:33.072011 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0127 12:09:33.072040 1099122 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0127 12:09:33.092674 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.092709 1099122 retry.go:31] will retry after 215.749644ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:33.092747 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.092760 1099122 retry.go:31] will retry after 342.392591ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.129246 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0127 12:09:33.129274 1099122 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0127 12:09:33.204763 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0127 12:09:33.204788 1099122 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
W0127 12:09:33.230805 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.230839 1099122 retry.go:31] will retry after 324.502248ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.242298 1099122 addons.go:435] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:33.242327 1099122 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0127 12:09:33.260523 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:33.309108 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0127 12:09:33.362004 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.362038 1099122 retry.go:31] will retry after 248.439965ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.436221 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0127 12:09:33.441825 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.441860 1099122 retry.go:31] will retry after 338.007574ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:33.535294 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.535329 1099122 retry.go:31] will retry after 200.606528ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.555554 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:09:33.610898 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0127 12:09:33.653675 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.653760 1099122 retry.go:31] will retry after 525.967005ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:33.733542 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.733576 1099122 retry.go:31] will retry after 423.596219ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.736824 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:09:33.779998 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0127 12:09:33.841591 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.841621 1099122 retry.go:31] will retry after 364.299343ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:33.927860 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:33.927896 1099122 retry.go:31] will retry after 661.414616ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.158342 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:34.180175 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:09:34.207000 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0127 12:09:34.280344 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.280385 1099122 retry.go:31] will retry after 722.451237ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:34.457040 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.457076 1099122 retry.go:31] will retry after 657.542567ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:34.461158 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.461198 1099122 retry.go:31] will retry after 854.797077ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.589545 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0127 12:09:34.663242 1099122 node_ready.go:53] error getting node "old-k8s-version-999803": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-999803": dial tcp 192.168.76.2:8443: connect: connection refused
W0127 12:09:34.678391 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:34.678431 1099122 retry.go:31] will retry after 732.310994ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.004235 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:35.115552 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0127 12:09:35.120846 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.120930 1099122 retry.go:31] will retry after 725.801172ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:35.216756 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.216839 1099122 retry.go:31] will retry after 1.144933552s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.317064 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:09:35.411423 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0127 12:09:35.421626 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.421711 1099122 retry.go:31] will retry after 1.353882074s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0127 12:09:35.509535 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.509620 1099122 retry.go:31] will retry after 1.490857259s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.847537 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0127 12:09:35.952169 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:35.952211 1099122 retry.go:31] will retry after 1.510852016s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:36.362267 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0127 12:09:36.478170 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:36.478255 1099122 retry.go:31] will retry after 741.738408ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:36.664100 1099122 node_ready.go:53] error getting node "old-k8s-version-999803": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-999803": dial tcp 192.168.76.2:8443: connect: connection refused
I0127 12:09:36.776557 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0127 12:09:36.894820 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:36.894858 1099122 retry.go:31] will retry after 2.217654455s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.001180 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0127 12:09:37.148389 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.148419 1099122 retry.go:31] will retry after 1.456107325s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.220218 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0127 12:09:37.321405 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.321446 1099122 retry.go:31] will retry after 2.422198697s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.463841 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0127 12:09:37.566522 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:37.566561 1099122 retry.go:31] will retry after 1.4666102s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:38.605130 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0127 12:09:38.714899 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:38.714938 1099122 retry.go:31] will retry after 3.82266818s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.033394 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:39.112893 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0127 12:09:39.153075 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.153113 1099122 retry.go:31] will retry after 1.892643033s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.163692 1099122 node_ready.go:53] error getting node "old-k8s-version-999803": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-999803": dial tcp 192.168.76.2:8443: connect: connection refused
W0127 12:09:39.243540 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.243574 1099122 retry.go:31] will retry after 3.934943475s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.744525 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0127 12:09:39.849927 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:39.849961 1099122 retry.go:31] will retry after 3.112229576s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:41.046747 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0127 12:09:41.295684 1099122 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:41.295721 1099122 retry.go:31] will retry after 5.587518335s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0127 12:09:42.537834 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0127 12:09:42.963287 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:09:43.179108 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:09:46.885295 1099122 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:09:51.665223 1099122 node_ready.go:53] error getting node "old-k8s-version-999803": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-999803": net/http: TLS handshake timeout
I0127 12:09:53.290513 1099122 node_ready.go:49] node "old-k8s-version-999803" has status "Ready":"True"
I0127 12:09:53.290545 1099122 node_ready.go:38] duration metric: took 20.627877462s for node "old-k8s-version-999803" to be "Ready" ...
I0127 12:09:53.290557 1099122 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0127 12:09:53.613480 1099122 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-8pc5m" in "kube-system" namespace to be "Ready" ...
I0127 12:09:53.683299 1099122 pod_ready.go:93] pod "coredns-74ff55c5b-8pc5m" in "kube-system" namespace has status "Ready":"True"
I0127 12:09:53.683339 1099122 pod_ready.go:82] duration metric: took 69.822535ms for pod "coredns-74ff55c5b-8pc5m" in "kube-system" namespace to be "Ready" ...
I0127 12:09:53.683353 1099122 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:09:53.726483 1099122 pod_ready.go:93] pod "etcd-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:09:53.726515 1099122 pod_ready.go:82] duration metric: took 43.154155ms for pod "etcd-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:09:53.726531 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:09:54.510253 1099122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (11.972361714s)
I0127 12:09:55.431146 1099122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (12.467809108s)
I0127 12:09:55.431337 1099122 addons.go:479] Verifying addon metrics-server=true in "old-k8s-version-999803"
I0127 12:09:55.431261 1099122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (12.252120921s)
I0127 12:09:55.547233 1099122 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (8.661886894s)
I0127 12:09:55.550367 1099122 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-999803 addons enable metrics-server
I0127 12:09:55.553376 1099122 out.go:177] * Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
I0127 12:09:55.556384 1099122 addons.go:514] duration metric: took 23.185560997s for enable addons: enabled=[default-storageclass metrics-server storage-provisioner dashboard]
I0127 12:09:55.756379 1099122 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:09:58.232516 1099122 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:00.303835 1099122 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:02.732144 1099122 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:03.733402 1099122 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:03.733430 1099122 pod_ready.go:82] duration metric: took 10.006890665s for pod "kube-apiserver-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:10:03.733443 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:10:05.740555 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:07.741384 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:10.239805 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:12.240144 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:14.240905 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:16.742205 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:18.742415 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:21.249112 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:23.740749 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:25.741758 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:27.758055 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:30.239728 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:32.239988 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:34.740785 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:36.741352 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:39.240630 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:41.740112 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:43.741550 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:45.741618 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:47.745656 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:50.240430 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:52.240688 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:54.741996 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:57.239927 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:59.240275 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:01.241298 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:03.740357 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:06.239980 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:08.741190 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:10.741646 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:11.741804 1099122 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:11.741830 1099122 pod_ready.go:82] duration metric: took 1m8.008379088s for pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.741843 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-nt2l9" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.747091 1099122 pod_ready.go:93] pod "kube-proxy-nt2l9" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:11.747117 1099122 pod_ready.go:82] duration metric: took 5.26649ms for pod "kube-proxy-nt2l9" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.747129 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:13.753603 1099122 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:13.753625 1099122 pod_ready.go:82] duration metric: took 2.006488609s for pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:13.753636 1099122 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace to be "Ready" ...
I0127 12:11:15.760059 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:17.760104 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:20.259362 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:22.259503 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:24.265513 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:26.759839 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:29.259521 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:31.759761 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:33.760790 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:36.259938 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:38.260402 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:40.260435 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:42.261372 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:44.759807 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:46.760139 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:49.259135 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:51.260408 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:53.761606 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:56.262295 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:58.760329 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:00.760445 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:03.259143 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:05.260247 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:07.759197 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:09.760214 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:11.760909 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:14.259817 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:16.260506 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:18.766515 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:21.259066 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:23.259646 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:25.259764 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:27.259886 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:29.758956 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:31.765446 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:34.259539 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:36.259690 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:38.259810 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:40.265756 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:42.759695 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:44.760001 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:47.264085 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:49.759794 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:51.760814 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:54.260573 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:56.261649 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:58.760562 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:01.260514 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:03.760846 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:06.259692 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:08.259809 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:10.264009 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:12.760961 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:15.259543 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:17.259672 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:19.266685 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:21.760466 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:23.760739 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:26.260232 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:28.759672 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:30.759737 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:32.760624 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:35.260200 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:37.760079 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:39.760771 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:42.261258 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:44.760471 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:47.260741 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:49.760653 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:51.761150 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:54.260064 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:56.260168 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:58.759553 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:00.760997 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:03.259192 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:05.260842 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:07.759902 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:09.760696 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:11.761676 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:14.259176 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:16.260538 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:18.760162 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:20.761412 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:23.259212 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:25.259378 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:27.260207 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:29.761136 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:32.259719 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:34.260064 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:36.760335 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:38.760390 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:41.260552 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:43.260596 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:45.305357 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:47.761135 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:49.762322 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:52.262267 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:54.760292 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:57.260208 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:59.760442 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:01.760706 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:04.260789 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:06.759923 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:08.761236 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:10.761497 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:13.259288 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:13.754136 1099122 pod_ready.go:82] duration metric: took 4m0.000481802s for pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace to be "Ready" ...
E0127 12:15:13.754168 1099122 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0127 12:15:13.754179 1099122 pod_ready.go:39] duration metric: took 5m20.463611828s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0127 12:15:13.754196 1099122 api_server.go:52] waiting for apiserver process to appear ...
I0127 12:15:13.754230 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:13.754302 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:13.794220 1099122 cri.go:89] found id: "709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:13.794243 1099122 cri.go:89] found id: "f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:13.794248 1099122 cri.go:89] found id: ""
I0127 12:15:13.794255 1099122 logs.go:282] 2 containers: [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e]
I0127 12:15:13.794338 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.797981 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.801452 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:13.801523 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:13.839152 1099122 cri.go:89] found id: "8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:13.839179 1099122 cri.go:89] found id: "2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:13.839185 1099122 cri.go:89] found id: ""
I0127 12:15:13.839192 1099122 logs.go:282] 2 containers: [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1]
I0127 12:15:13.839249 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.842927 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.846323 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:13.846397 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:13.884753 1099122 cri.go:89] found id: "66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:13.884776 1099122 cri.go:89] found id: "006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:13.884781 1099122 cri.go:89] found id: ""
I0127 12:15:13.884787 1099122 logs.go:282] 2 containers: [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863]
I0127 12:15:13.884849 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.888585 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.892544 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:13.892620 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:13.935201 1099122 cri.go:89] found id: "15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:13.935265 1099122 cri.go:89] found id: "8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:13.935275 1099122 cri.go:89] found id: ""
I0127 12:15:13.935282 1099122 logs.go:282] 2 containers: [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b]
I0127 12:15:13.935348 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.938912 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.942321 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:13.942420 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:13.982013 1099122 cri.go:89] found id: "244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:13.982037 1099122 cri.go:89] found id: "69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:13.982042 1099122 cri.go:89] found id: ""
I0127 12:15:13.982049 1099122 logs.go:282] 2 containers: [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d]
I0127 12:15:13.982107 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.985808 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.989196 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:13.989297 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:14.034072 1099122 cri.go:89] found id: "35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:14.034096 1099122 cri.go:89] found id: "24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:14.034103 1099122 cri.go:89] found id: ""
I0127 12:15:14.034110 1099122 logs.go:282] 2 containers: [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84]
I0127 12:15:14.034175 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.038229 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.041981 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:14.042087 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:14.096637 1099122 cri.go:89] found id: "8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:14.096662 1099122 cri.go:89] found id: "92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:14.096667 1099122 cri.go:89] found id: ""
I0127 12:15:14.096674 1099122 logs.go:282] 2 containers: [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f]
I0127 12:15:14.096735 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.100700 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.104367 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:14.104440 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:14.142295 1099122 cri.go:89] found id: "d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:14.142318 1099122 cri.go:89] found id: ""
I0127 12:15:14.142341 1099122 logs.go:282] 1 containers: [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6]
I0127 12:15:14.142395 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.145773 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:14.145852 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:14.184211 1099122 cri.go:89] found id: "1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:14.184233 1099122 cri.go:89] found id: "072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:14.184239 1099122 cri.go:89] found id: ""
I0127 12:15:14.184246 1099122 logs.go:282] 2 containers: [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4]
I0127 12:15:14.184300 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.187804 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.191031 1099122 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:14.191103 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:14.210127 1099122 logs.go:123] Gathering logs for etcd [2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1] ...
I0127 12:15:14.210159 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:14.250486 1099122 logs.go:123] Gathering logs for kube-controller-manager [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d] ...
I0127 12:15:14.250517 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:14.310627 1099122 logs.go:123] Gathering logs for kubernetes-dashboard [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6] ...
I0127 12:15:14.310663 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:14.357158 1099122 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:14.357187 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:14.421142 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.065326 663 reflector.go:138] object-"kube-system"/"metrics-server-token-827qh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-827qh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421398 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.068245 663 reflector.go:138] object-"kube-system"/"kindnet-token-jxc27": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jxc27" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421622 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.072508 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-pzcvk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-pzcvk" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421857 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.092394 663 reflector.go:138] object-"kube-system"/"coredns-token-m2lsh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2lsh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422066 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.102861 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422281 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.106409 663 reflector.go:138] object-"default"/"default-token-pmbfm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-pmbfm" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422509 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135445 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-bbnlz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-bbnlz" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422714 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135737 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.431944 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:55 old-k8s-version-999803 kubelet[663]: E0127 12:09:55.695476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.432135 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:56 old-k8s-version-999803 kubelet[663]: E0127 12:09:56.587407 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.434973 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:08 old-k8s-version-999803 kubelet[663]: E0127 12:10:08.310539 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.436943 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:18 old-k8s-version-999803 kubelet[663]: E0127 12:10:18.685285 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.437410 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:19 old-k8s-version-999803 kubelet[663]: E0127 12:10:19.697364 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.437932 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:23 old-k8s-version-999803 kubelet[663]: E0127 12:10:23.289410 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.438369 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:27 old-k8s-version-999803 kubelet[663]: E0127 12:10:27.725508 663 pod_workers.go:191] Error syncing pod f73574be-9aec-4a33-ac88-97d900488a22 ("storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"
W0127 12:15:14.439000 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:29 old-k8s-version-999803 kubelet[663]: E0127 12:10:29.743840 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.441866 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:35 old-k8s-version-999803 kubelet[663]: E0127 12:10:35.298365 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.442236 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:38 old-k8s-version-999803 kubelet[663]: E0127 12:10:38.666640 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.442554 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:46 old-k8s-version-999803 kubelet[663]: E0127 12:10:46.289557 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.443148 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:53 old-k8s-version-999803 kubelet[663]: E0127 12:10:53.813459 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.443334 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:57 old-k8s-version-999803 kubelet[663]: E0127 12:10:57.289244 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.443693 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:58 old-k8s-version-999803 kubelet[663]: E0127 12:10:58.662563 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.443879 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.289633 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.444208 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.290664 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.446683 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:21 old-k8s-version-999803 kubelet[663]: E0127 12:11:21.295133 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.447018 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:25 old-k8s-version-999803 kubelet[663]: E0127 12:11:25.289132 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.447202 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:36 old-k8s-version-999803 kubelet[663]: E0127 12:11:36.291476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.447862 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:40 old-k8s-version-999803 kubelet[663]: E0127 12:11:40.940519 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448197 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:48 old-k8s-version-999803 kubelet[663]: E0127 12:11:48.662783 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448380 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:50 old-k8s-version-999803 kubelet[663]: E0127 12:11:50.289726 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.448722 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.288851 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448905 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.289979 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.449125 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:14 old-k8s-version-999803 kubelet[663]: E0127 12:12:14.295027 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.449483 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:18 old-k8s-version-999803 kubelet[663]: E0127 12:12:18.288843 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.449672 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:26 old-k8s-version-999803 kubelet[663]: E0127 12:12:26.289650 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.450001 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:30 old-k8s-version-999803 kubelet[663]: E0127 12:12:30.289329 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.450184 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:37 old-k8s-version-999803 kubelet[663]: E0127 12:12:37.289235 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.450509 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:44 old-k8s-version-999803 kubelet[663]: E0127 12:12:44.288931 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.452938 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:50 old-k8s-version-999803 kubelet[663]: E0127 12:12:50.299810 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.453274 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:57 old-k8s-version-999803 kubelet[663]: E0127 12:12:57.288813 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.453460 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:03 old-k8s-version-999803 kubelet[663]: E0127 12:13:03.289391 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.454055 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:11 old-k8s-version-999803 kubelet[663]: E0127 12:13:11.199586 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.454245 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:14 old-k8s-version-999803 kubelet[663]: E0127 12:13:14.289503 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.454570 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:18 old-k8s-version-999803 kubelet[663]: E0127 12:13:18.662572 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.454756 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:29 old-k8s-version-999803 kubelet[663]: E0127 12:13:29.289301 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.455083 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:31 old-k8s-version-999803 kubelet[663]: E0127 12:13:31.288795 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.455286 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.289460 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.455615 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.290963 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.455808 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:53 old-k8s-version-999803 kubelet[663]: E0127 12:13:53.289152 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.456133 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:57 old-k8s-version-999803 kubelet[663]: E0127 12:13:57.288829 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.456320 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:07 old-k8s-version-999803 kubelet[663]: E0127 12:14:07.289177 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.456652 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: E0127 12:14:12.289332 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.456835 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:21 old-k8s-version-999803 kubelet[663]: E0127 12:14:21.289169 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.457166 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: E0127 12:14:26.288817 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.457351 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:32 old-k8s-version-999803 kubelet[663]: E0127 12:14:32.289382 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.457676 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: E0127 12:14:37.288718 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.457860 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.458184 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.458369 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.458694 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.458877 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:14.458888 1099122 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:14.458906 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:14.600554 1099122 logs.go:123] Gathering logs for coredns [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab] ...
I0127 12:15:14.600589 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:14.642894 1099122 logs.go:123] Gathering logs for kube-controller-manager [24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84] ...
I0127 12:15:14.642924 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:14.708414 1099122 logs.go:123] Gathering logs for etcd [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9] ...
I0127 12:15:14.708446 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:14.750297 1099122 logs.go:123] Gathering logs for coredns [006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863] ...
I0127 12:15:14.750327 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:14.787004 1099122 logs.go:123] Gathering logs for kube-scheduler [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47] ...
I0127 12:15:14.787031 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:14.828360 1099122 logs.go:123] Gathering logs for kube-proxy [69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d] ...
I0127 12:15:14.828389 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:14.873162 1099122 logs.go:123] Gathering logs for kindnet [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026] ...
I0127 12:15:14.873189 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:14.921463 1099122 logs.go:123] Gathering logs for kindnet [92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f] ...
I0127 12:15:14.921495 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:14.974595 1099122 logs.go:123] Gathering logs for storage-provisioner [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609] ...
I0127 12:15:14.974623 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:15.034157 1099122 logs.go:123] Gathering logs for storage-provisioner [072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4] ...
I0127 12:15:15.034189 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:15.077801 1099122 logs.go:123] Gathering logs for kube-apiserver [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f] ...
I0127 12:15:15.077829 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:15.150398 1099122 logs.go:123] Gathering logs for kube-apiserver [f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e] ...
I0127 12:15:15.150475 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:15.219967 1099122 logs.go:123] Gathering logs for kube-scheduler [8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b] ...
I0127 12:15:15.220021 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:15.267003 1099122 logs.go:123] Gathering logs for kube-proxy [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56] ...
I0127 12:15:15.267075 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:15.305261 1099122 logs.go:123] Gathering logs for containerd ...
I0127 12:15:15.305350 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:15.379123 1099122 logs.go:123] Gathering logs for container status ...
I0127 12:15:15.379163 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:15.437988 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:15.438018 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:15.438072 1099122 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0127 12:15:15.438088 1099122 out.go:270] Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:15.438101 1099122 out.go:270] Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:15.438126 1099122 out.go:270] Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:15.438132 1099122 out.go:270] Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:15.438137 1099122 out.go:270] Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:15.438142 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:15.438151 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:25.440470 1099122 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0127 12:15:25.452605 1099122 api_server.go:72] duration metric: took 5m53.082233775s to wait for apiserver process to appear ...
I0127 12:15:25.452629 1099122 api_server.go:88] waiting for apiserver healthz status ...
I0127 12:15:25.452667 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:25.452724 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:25.494090 1099122 cri.go:89] found id: "709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:25.494115 1099122 cri.go:89] found id: "f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:25.494121 1099122 cri.go:89] found id: ""
I0127 12:15:25.494128 1099122 logs.go:282] 2 containers: [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e]
I0127 12:15:25.494189 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.497645 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.500895 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:25.500968 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:25.542357 1099122 cri.go:89] found id: "8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:25.542442 1099122 cri.go:89] found id: "2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:25.542451 1099122 cri.go:89] found id: ""
I0127 12:15:25.542460 1099122 logs.go:282] 2 containers: [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1]
I0127 12:15:25.542525 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.548254 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.552119 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:25.552193 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:25.627449 1099122 cri.go:89] found id: "66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:25.627471 1099122 cri.go:89] found id: "006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:25.627476 1099122 cri.go:89] found id: ""
I0127 12:15:25.627484 1099122 logs.go:282] 2 containers: [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863]
I0127 12:15:25.627539 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.631955 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.635615 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:25.635695 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:25.686029 1099122 cri.go:89] found id: "15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:25.686052 1099122 cri.go:89] found id: "8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:25.686057 1099122 cri.go:89] found id: ""
I0127 12:15:25.686063 1099122 logs.go:282] 2 containers: [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b]
I0127 12:15:25.686121 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.691005 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.696361 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:25.696439 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:25.735204 1099122 cri.go:89] found id: "244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:25.735228 1099122 cri.go:89] found id: "69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:25.735233 1099122 cri.go:89] found id: ""
I0127 12:15:25.735246 1099122 logs.go:282] 2 containers: [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d]
I0127 12:15:25.735318 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.738739 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.742012 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:25.742080 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:25.783705 1099122 cri.go:89] found id: "35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:25.783728 1099122 cri.go:89] found id: "24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:25.783733 1099122 cri.go:89] found id: ""
I0127 12:15:25.783740 1099122 logs.go:282] 2 containers: [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84]
I0127 12:15:25.783798 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.787402 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.790806 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:25.790883 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:25.830016 1099122 cri.go:89] found id: "8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:25.830038 1099122 cri.go:89] found id: "92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:25.830043 1099122 cri.go:89] found id: ""
I0127 12:15:25.830050 1099122 logs.go:282] 2 containers: [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f]
I0127 12:15:25.830108 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.834070 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.837688 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:25.837767 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:25.880360 1099122 cri.go:89] found id: "1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:25.880381 1099122 cri.go:89] found id: "072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:25.880386 1099122 cri.go:89] found id: ""
I0127 12:15:25.880394 1099122 logs.go:282] 2 containers: [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4]
I0127 12:15:25.880459 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.884176 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.888084 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:25.888159 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:25.929160 1099122 cri.go:89] found id: "d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:25.929185 1099122 cri.go:89] found id: ""
I0127 12:15:25.929193 1099122 logs.go:282] 1 containers: [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6]
I0127 12:15:25.929257 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.934269 1099122 logs.go:123] Gathering logs for kindnet [92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f] ...
I0127 12:15:25.934297 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:25.978515 1099122 logs.go:123] Gathering logs for kube-scheduler [8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b] ...
I0127 12:15:25.978546 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:26.023365 1099122 logs.go:123] Gathering logs for kube-proxy [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56] ...
I0127 12:15:26.023397 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:26.074845 1099122 logs.go:123] Gathering logs for kube-proxy [69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d] ...
I0127 12:15:26.074873 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:26.126098 1099122 logs.go:123] Gathering logs for container status ...
I0127 12:15:26.126177 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:26.170530 1099122 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:26.170571 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:26.190451 1099122 logs.go:123] Gathering logs for coredns [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab] ...
I0127 12:15:26.190479 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:26.240411 1099122 logs.go:123] Gathering logs for storage-provisioner [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609] ...
I0127 12:15:26.240439 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:26.285457 1099122 logs.go:123] Gathering logs for kindnet [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026] ...
I0127 12:15:26.285491 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:26.332919 1099122 logs.go:123] Gathering logs for containerd ...
I0127 12:15:26.332949 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:26.397666 1099122 logs.go:123] Gathering logs for kube-apiserver [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f] ...
I0127 12:15:26.397707 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:26.459604 1099122 logs.go:123] Gathering logs for etcd [2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1] ...
I0127 12:15:26.459638 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:26.511966 1099122 logs.go:123] Gathering logs for kube-scheduler [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47] ...
I0127 12:15:26.512123 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:26.553498 1099122 logs.go:123] Gathering logs for etcd [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9] ...
I0127 12:15:26.553568 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:26.619235 1099122 logs.go:123] Gathering logs for coredns [006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863] ...
I0127 12:15:26.619267 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:26.662492 1099122 logs.go:123] Gathering logs for kube-controller-manager [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d] ...
I0127 12:15:26.662523 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:26.717270 1099122 logs.go:123] Gathering logs for kube-controller-manager [24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84] ...
I0127 12:15:26.717303 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:26.789783 1099122 logs.go:123] Gathering logs for storage-provisioner [072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4] ...
I0127 12:15:26.789827 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:26.841274 1099122 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:26.841302 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:26.899918 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.065326 663 reflector.go:138] object-"kube-system"/"metrics-server-token-827qh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-827qh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900177 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.068245 663 reflector.go:138] object-"kube-system"/"kindnet-token-jxc27": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jxc27" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900398 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.072508 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-pzcvk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-pzcvk" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900685 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.092394 663 reflector.go:138] object-"kube-system"/"coredns-token-m2lsh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2lsh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900893 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.102861 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901112 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.106409 663 reflector.go:138] object-"default"/"default-token-pmbfm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-pmbfm" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901342 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135445 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-bbnlz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-bbnlz" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901547 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135737 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.910804 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:55 old-k8s-version-999803 kubelet[663]: E0127 12:09:55.695476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.910996 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:56 old-k8s-version-999803 kubelet[663]: E0127 12:09:56.587407 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.913762 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:08 old-k8s-version-999803 kubelet[663]: E0127 12:10:08.310539 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.915693 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:18 old-k8s-version-999803 kubelet[663]: E0127 12:10:18.685285 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.916152 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:19 old-k8s-version-999803 kubelet[663]: E0127 12:10:19.697364 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.916671 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:23 old-k8s-version-999803 kubelet[663]: E0127 12:10:23.289410 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.917136 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:27 old-k8s-version-999803 kubelet[663]: E0127 12:10:27.725508 663 pod_workers.go:191] Error syncing pod f73574be-9aec-4a33-ac88-97d900488a22 ("storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"
W0127 12:15:26.917721 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:29 old-k8s-version-999803 kubelet[663]: E0127 12:10:29.743840 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.920519 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:35 old-k8s-version-999803 kubelet[663]: E0127 12:10:35.298365 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.920848 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:38 old-k8s-version-999803 kubelet[663]: E0127 12:10:38.666640 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.921170 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:46 old-k8s-version-999803 kubelet[663]: E0127 12:10:46.289557 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.921758 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:53 old-k8s-version-999803 kubelet[663]: E0127 12:10:53.813459 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.921942 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:57 old-k8s-version-999803 kubelet[663]: E0127 12:10:57.289244 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.922272 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:58 old-k8s-version-999803 kubelet[663]: E0127 12:10:58.662563 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.922455 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.289633 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.922880 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.290664 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.925368 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:21 old-k8s-version-999803 kubelet[663]: E0127 12:11:21.295133 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.925709 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:25 old-k8s-version-999803 kubelet[663]: E0127 12:11:25.289132 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.925894 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:36 old-k8s-version-999803 kubelet[663]: E0127 12:11:36.291476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.926506 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:40 old-k8s-version-999803 kubelet[663]: E0127 12:11:40.940519 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.926833 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:48 old-k8s-version-999803 kubelet[663]: E0127 12:11:48.662783 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.927017 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:50 old-k8s-version-999803 kubelet[663]: E0127 12:11:50.289726 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.927348 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.288851 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.927543 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.289979 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.927729 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:14 old-k8s-version-999803 kubelet[663]: E0127 12:12:14.295027 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.928059 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:18 old-k8s-version-999803 kubelet[663]: E0127 12:12:18.288843 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.928244 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:26 old-k8s-version-999803 kubelet[663]: E0127 12:12:26.289650 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.928570 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:30 old-k8s-version-999803 kubelet[663]: E0127 12:12:30.289329 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.928753 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:37 old-k8s-version-999803 kubelet[663]: E0127 12:12:37.289235 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.929138 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:44 old-k8s-version-999803 kubelet[663]: E0127 12:12:44.288931 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.931588 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:50 old-k8s-version-999803 kubelet[663]: E0127 12:12:50.299810 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.931921 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:57 old-k8s-version-999803 kubelet[663]: E0127 12:12:57.288813 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.932106 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:03 old-k8s-version-999803 kubelet[663]: E0127 12:13:03.289391 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.932704 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:11 old-k8s-version-999803 kubelet[663]: E0127 12:13:11.199586 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.932890 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:14 old-k8s-version-999803 kubelet[663]: E0127 12:13:14.289503 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.933222 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:18 old-k8s-version-999803 kubelet[663]: E0127 12:13:18.662572 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.933406 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:29 old-k8s-version-999803 kubelet[663]: E0127 12:13:29.289301 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.933735 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:31 old-k8s-version-999803 kubelet[663]: E0127 12:13:31.288795 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.933921 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.289460 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.934248 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.290963 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.934432 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:53 old-k8s-version-999803 kubelet[663]: E0127 12:13:53.289152 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.934759 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:57 old-k8s-version-999803 kubelet[663]: E0127 12:13:57.288829 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.934942 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:07 old-k8s-version-999803 kubelet[663]: E0127 12:14:07.289177 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.935267 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: E0127 12:14:12.289332 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.935452 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:21 old-k8s-version-999803 kubelet[663]: E0127 12:14:21.289169 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.935784 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: E0127 12:14:26.288817 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.935967 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:32 old-k8s-version-999803 kubelet[663]: E0127 12:14:32.289382 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.936292 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: E0127 12:14:37.288718 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.936477 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.936802 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.936988 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.937320 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.937506 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.937836 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.938019 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:26.938029 1099122 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:26.938044 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:27.111830 1099122 logs.go:123] Gathering logs for kube-apiserver [f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e] ...
I0127 12:15:27.111864 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:27.165850 1099122 logs.go:123] Gathering logs for kubernetes-dashboard [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6] ...
I0127 12:15:27.165886 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:27.211532 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:27.211556 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:27.211643 1099122 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0127 12:15:27.211657 1099122 out.go:270] Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:27.211679 1099122 out.go:270] Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:27.211711 1099122 out.go:270] Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:27.211720 1099122 out.go:270] Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:27.211727 1099122 out.go:270] Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:27.211737 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:27.211744 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:37.213669 1099122 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0127 12:15:37.226947 1099122 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0127 12:15:37.229972 1099122 out.go:201]
W0127 12:15:37.232459 1099122 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0127 12:15:37.232495 1099122 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0127 12:15:37.232520 1099122 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0127 12:15:37.232529 1099122 out.go:270] *
*
W0127 12:15:37.233525 1099122 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0127 12:15:37.237108 1099122 out.go:201]
** /stderr **
start_stop_delete_test.go:257: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-999803 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-999803
helpers_test.go:235: (dbg) docker inspect old-k8s-version-999803:
-- stdout --
[
{
"Id": "677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15",
"Created": "2025-01-27T12:06:42.163950524Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1099337,
"ExitCode": 0,
"Error": "",
"StartedAt": "2025-01-27T12:09:23.726651729Z",
"FinishedAt": "2025-01-27T12:09:22.635251532Z"
},
"Image": "sha256:0434cf58b6dbace281e5de753aa4b2e3fe33dc9a3be53021531403743c3f155a",
"ResolvConfPath": "/var/lib/docker/containers/677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15/hostname",
"HostsPath": "/var/lib/docker/containers/677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15/hosts",
"LogPath": "/var/lib/docker/containers/677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15/677d3676ac1fbe8f2697346fc0eac2d7484b857cb067ddc8cb41a960e04edf15-json.log",
"Name": "/old-k8s-version-999803",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-999803:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-999803",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/c0fcf769388dfba01188e437d9f3a77650f13f130de7eb4c70f4f60472407768-init/diff:/var/lib/docker/overlay2/027cb12703497bfe682a04123361dc92cd40ae4c78d3ee9eafeedefee7ad1bd7/diff",
"MergedDir": "/var/lib/docker/overlay2/c0fcf769388dfba01188e437d9f3a77650f13f130de7eb4c70f4f60472407768/merged",
"UpperDir": "/var/lib/docker/overlay2/c0fcf769388dfba01188e437d9f3a77650f13f130de7eb4c70f4f60472407768/diff",
"WorkDir": "/var/lib/docker/overlay2/c0fcf769388dfba01188e437d9f3a77650f13f130de7eb4c70f4f60472407768/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-999803",
"Source": "/var/lib/docker/volumes/old-k8s-version-999803/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-999803",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-999803",
"name.minikube.sigs.k8s.io": "old-k8s-version-999803",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "5f4d1a9bef0aedd6cd51c57ae0d895ec6b9107e1526207734103f13b24ede6ea",
"SandboxKey": "/var/run/docker/netns/5f4d1a9bef0a",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33857"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33858"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33861"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33859"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33860"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-999803": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "b4e46dc8ac51fed9012b81352b7879f1f93fd5c5934f073610fd15fd0544b6d6",
"EndpointID": "223d6f73923c0ac688cc4d02596f5afe549c03199e4350abfc72029a3a44ada7",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-999803",
"677d3676ac1f"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-999803 -n old-k8s-version-999803
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-999803 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-999803 logs -n 25: (2.138170418s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| ssh | -p cilium-106238 sudo | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | |
| | containerd config dump | | | | | |
| ssh | -p cilium-106238 sudo | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | |
| | systemctl status crio --all | | | | | |
| | --full --no-pager | | | | | |
| ssh | -p cilium-106238 sudo | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | |
| | systemctl cat crio --no-pager | | | | | |
| ssh | -p cilium-106238 sudo find | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p cilium-106238 sudo crio | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | |
| | config | | | | | |
| delete | -p cilium-106238 | cilium-106238 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | 27 Jan 25 12:05 UTC |
| start | -p cert-expiration-972837 | cert-expiration-972837 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | 27 Jan 25 12:06 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-947488 | force-systemd-env-947488 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | 27 Jan 25 12:05 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-947488 | force-systemd-env-947488 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | 27 Jan 25 12:05 UTC |
| start | -p cert-options-429275 | cert-options-429275 | jenkins | v1.35.0 | 27 Jan 25 12:05 UTC | 27 Jan 25 12:06 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-429275 ssh | cert-options-429275 | jenkins | v1.35.0 | 27 Jan 25 12:06 UTC | 27 Jan 25 12:06 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-429275 -- sudo | cert-options-429275 | jenkins | v1.35.0 | 27 Jan 25 12:06 UTC | 27 Jan 25 12:06 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-429275 | cert-options-429275 | jenkins | v1.35.0 | 27 Jan 25 12:06 UTC | 27 Jan 25 12:06 UTC |
| start | -p old-k8s-version-999803 | old-k8s-version-999803 | jenkins | v1.35.0 | 27 Jan 25 12:06 UTC | 27 Jan 25 12:08 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-972837 | cert-expiration-972837 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:09 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| addons | enable metrics-server -p old-k8s-version-999803 | old-k8s-version-999803 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:09 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| delete | -p cert-expiration-972837 | cert-expiration-972837 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:09 UTC |
| stop | -p old-k8s-version-999803 | old-k8s-version-999803 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:09 UTC |
| | --alsologtostderr -v=3 | | | | | |
| start | -p no-preload-835765 | no-preload-835765 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:10 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.32.1 | | | | | |
| addons | enable dashboard -p old-k8s-version-999803 | old-k8s-version-999803 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | 27 Jan 25 12:09 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-999803 | old-k8s-version-999803 | jenkins | v1.35.0 | 27 Jan 25 12:09 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-835765 | no-preload-835765 | jenkins | v1.35.0 | 27 Jan 25 12:10 UTC | 27 Jan 25 12:10 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-835765 | no-preload-835765 | jenkins | v1.35.0 | 27 Jan 25 12:10 UTC | 27 Jan 25 12:10 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-835765 | no-preload-835765 | jenkins | v1.35.0 | 27 Jan 25 12:10 UTC | 27 Jan 25 12:10 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-835765 | no-preload-835765 | jenkins | v1.35.0 | 27 Jan 25 12:10 UTC | 27 Jan 25 12:15 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.32.1 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2025/01/27 12:10:38
Running on machine: ip-172-31-29-130
Binary: Built with gc go1.23.4 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0127 12:10:38.803047 1104534 out.go:345] Setting OutFile to fd 1 ...
I0127 12:10:38.803261 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:10:38.803287 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:10:38.803306 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:10:38.803676 1104534 root.go:338] Updating PATH: /home/jenkins/minikube-integration/20318-888339/.minikube/bin
I0127 12:10:38.804192 1104534 out.go:352] Setting JSON to false
I0127 12:10:38.805735 1104534 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":17584,"bootTime":1737962255,"procs":240,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1075-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0127 12:10:38.805850 1104534 start.go:139] virtualization:
I0127 12:10:38.808874 1104534 out.go:177] * [no-preload-835765] minikube v1.35.0 on Ubuntu 20.04 (arm64)
I0127 12:10:38.812232 1104534 out.go:177] - MINIKUBE_LOCATION=20318
I0127 12:10:38.812359 1104534 notify.go:220] Checking for updates...
I0127 12:10:38.817591 1104534 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0127 12:10:38.820375 1104534 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:10:38.822943 1104534 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/20318-888339/.minikube
I0127 12:10:38.825500 1104534 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0127 12:10:38.828053 1104534 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0127 12:10:38.831436 1104534 config.go:182] Loaded profile config "no-preload-835765": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.32.1
I0127 12:10:38.832050 1104534 driver.go:394] Setting default libvirt URI to qemu:///system
I0127 12:10:38.863133 1104534 docker.go:123] docker version: linux-27.5.1:Docker Engine - Community
I0127 12:10:38.863258 1104534 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0127 12:10:38.937831 1104534 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:32 OomKillDisable:true NGoroutines:53 SystemTime:2025-01-27 12:10:38.922663769 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1075-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.5.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb Expected:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb} RuncCommit:{ID:v1.2.4-0-g6c52b3f Expected:v1.2.4-0-g6c52b3f} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.20.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.32.4]] Warnings:<nil>}}
I0127 12:10:38.937948 1104534 docker.go:318] overlay module found
I0127 12:10:38.941535 1104534 out.go:177] * Using the docker driver based on existing profile
I0127 12:10:38.944224 1104534 start.go:297] selected driver: docker
I0127 12:10:38.944287 1104534 start.go:901] validating driver "docker" against &{Name:no-preload-835765 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.32.1 ClusterName:no-preload-835765 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerN
ames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.32.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/mi
nikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:10:38.944448 1104534 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0127 12:10:38.945332 1104534 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0127 12:10:39.006329 1104534 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:32 OomKillDisable:true NGoroutines:53 SystemTime:2025-01-27 12:10:38.996157859 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1075-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214835200 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.5.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb Expected:bcc810d6b9066471b0b6fa75f557a15a1cbf31bb} RuncCommit:{ID:v1.2.4-0-g6c52b3f Expected:v1.2.4-0-g6c52b3f} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.20.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.32.4]] Warnings:<nil>}}
I0127 12:10:39.006779 1104534 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0127 12:10:39.006806 1104534 cni.go:84] Creating CNI manager for ""
I0127 12:10:39.006848 1104534 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0127 12:10:39.006897 1104534 start.go:340] cluster config:
{Name:no-preload-835765 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.32.1 ClusterName:no-preload-835765 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRIS
ocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.32.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 M
ountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:10:39.025134 1104534 out.go:177] * Starting "no-preload-835765" primary control-plane node in "no-preload-835765" cluster
I0127 12:10:39.027812 1104534 cache.go:121] Beginning downloading kic base image for docker with containerd
I0127 12:10:39.030533 1104534 out.go:177] * Pulling base image v0.0.46 ...
I0127 12:10:39.033294 1104534 image.go:81] Checking for gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 in local docker daemon
I0127 12:10:39.033258 1104534 preload.go:131] Checking if preload exists for k8s version v1.32.1 and runtime containerd
I0127 12:10:39.033520 1104534 profile.go:143] Saving config to /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/config.json ...
I0127 12:10:39.033856 1104534 cache.go:107] acquiring lock: {Name:mk292ee9ed5b43a9c61611c969682fac3248b2f1 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.033941 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0127 12:10:39.033950 1104534 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 100.511µs
I0127 12:10:39.033959 1104534 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0127 12:10:39.033970 1104534 cache.go:107] acquiring lock: {Name:mka6724bad1f9cf143ea552dbb28a2755f784c65 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034008 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.32.1 exists
I0127 12:10:39.034014 1104534 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.32.1" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.32.1" took 45.341µs
I0127 12:10:39.034020 1104534 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.32.1 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.32.1 succeeded
I0127 12:10:39.034029 1104534 cache.go:107] acquiring lock: {Name:mk1b9eab94843a4f62d36fad35d75d33bd51a03e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034061 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.32.1 exists
I0127 12:10:39.034066 1104534 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.32.1" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.32.1" took 37.981µs
I0127 12:10:39.034072 1104534 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.32.1 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.32.1 succeeded
I0127 12:10:39.034081 1104534 cache.go:107] acquiring lock: {Name:mk44a6dcc398514c761e3b9818a69e8bfbed5335 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034112 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.32.1 exists
I0127 12:10:39.034117 1104534 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.32.1" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.32.1" took 37.627µs
I0127 12:10:39.034123 1104534 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.32.1 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.32.1 succeeded
I0127 12:10:39.034135 1104534 cache.go:107] acquiring lock: {Name:mkeba9e5dabb87864f54b2e7d8405ff976418755 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034160 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.32.1 exists
I0127 12:10:39.034170 1104534 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.32.1" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.32.1" took 34.321µs
I0127 12:10:39.034175 1104534 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.32.1 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.32.1 succeeded
I0127 12:10:39.034186 1104534 cache.go:107] acquiring lock: {Name:mkde6de3bc5db18a92ec49ef53a0c1340f50c6cf Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034211 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0127 12:10:39.034220 1104534 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 31.04µs
I0127 12:10:39.034225 1104534 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0127 12:10:39.034234 1104534 cache.go:107] acquiring lock: {Name:mkdf68f1ad8fd9f55ae85b3a69ce9b122c0e1f65 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034259 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.16-0 exists
I0127 12:10:39.034264 1104534 cache.go:96] cache image "registry.k8s.io/etcd:3.5.16-0" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.16-0" took 31.162µs
I0127 12:10:39.034269 1104534 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.16-0 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.16-0 succeeded
I0127 12:10:39.034278 1104534 cache.go:107] acquiring lock: {Name:mkc31583c11dfe3a18725ed06b477e250589266f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.034302 1104534 cache.go:115] /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 exists
I0127 12:10:39.034307 1104534 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.3" -> "/home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3" took 30.03µs
I0127 12:10:39.034312 1104534 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.3 -> /home/jenkins/minikube-integration/20318-888339/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 succeeded
I0127 12:10:39.034317 1104534 cache.go:87] Successfully saved all images to host disk.
I0127 12:10:39.056936 1104534 image.go:100] Found gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 in local docker daemon, skipping pull
I0127 12:10:39.056962 1104534 cache.go:145] gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 exists in daemon, skipping load
I0127 12:10:39.056983 1104534 cache.go:227] Successfully downloaded all kic artifacts
I0127 12:10:39.057006 1104534 start.go:360] acquireMachinesLock for no-preload-835765: {Name:mkc979a89666647f77f4b877447000ec99cc9e8d Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0127 12:10:39.057086 1104534 start.go:364] duration metric: took 59.436µs to acquireMachinesLock for "no-preload-835765"
I0127 12:10:39.057111 1104534 start.go:96] Skipping create...Using existing machine configuration
I0127 12:10:39.057125 1104534 fix.go:54] fixHost starting:
I0127 12:10:39.057394 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:39.074673 1104534 fix.go:112] recreateIfNeeded on no-preload-835765: state=Stopped err=<nil>
W0127 12:10:39.074703 1104534 fix.go:138] unexpected machine state, will restart: <nil>
I0127 12:10:39.077693 1104534 out.go:177] * Restarting existing docker container for "no-preload-835765" ...
I0127 12:10:39.240630 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:41.740112 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:39.081358 1104534 cli_runner.go:164] Run: docker start no-preload-835765
I0127 12:10:39.412477 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:39.432039 1104534 kic.go:430] container "no-preload-835765" state is running.
I0127 12:10:39.432409 1104534 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-835765
I0127 12:10:39.459975 1104534 profile.go:143] Saving config to /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/config.json ...
I0127 12:10:39.460205 1104534 machine.go:93] provisionDockerMachine start ...
I0127 12:10:39.460281 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:39.480477 1104534 main.go:141] libmachine: Using SSH client type: native
I0127 12:10:39.480735 1104534 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33862 <nil> <nil>}
I0127 12:10:39.480745 1104534 main.go:141] libmachine: About to run SSH command:
hostname
I0127 12:10:39.481659 1104534 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0127 12:10:42.609324 1104534 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-835765
I0127 12:10:42.609352 1104534 ubuntu.go:169] provisioning hostname "no-preload-835765"
I0127 12:10:42.609414 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:42.628230 1104534 main.go:141] libmachine: Using SSH client type: native
I0127 12:10:42.628478 1104534 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33862 <nil> <nil>}
I0127 12:10:42.628496 1104534 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-835765 && echo "no-preload-835765" | sudo tee /etc/hostname
I0127 12:10:42.770642 1104534 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-835765
I0127 12:10:42.770737 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:42.789875 1104534 main.go:141] libmachine: Using SSH client type: native
I0127 12:10:42.790136 1104534 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x4132a0] 0x415ae0 <nil> [] 0s} 127.0.0.1 33862 <nil> <nil>}
I0127 12:10:42.790159 1104534 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-835765' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-835765/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-835765' | sudo tee -a /etc/hosts;
fi
fi
I0127 12:10:42.913316 1104534 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0127 12:10:42.913346 1104534 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/20318-888339/.minikube CaCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/20318-888339/.minikube}
I0127 12:10:42.913370 1104534 ubuntu.go:177] setting up certificates
I0127 12:10:42.913380 1104534 provision.go:84] configureAuth start
I0127 12:10:42.913445 1104534 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-835765
I0127 12:10:42.933903 1104534 provision.go:143] copyHostCerts
I0127 12:10:42.933972 1104534 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem, removing ...
I0127 12:10:42.933981 1104534 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem
I0127 12:10:42.934062 1104534 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/ca.pem (1082 bytes)
I0127 12:10:42.934169 1104534 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem, removing ...
I0127 12:10:42.934179 1104534 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem
I0127 12:10:42.934207 1104534 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/cert.pem (1123 bytes)
I0127 12:10:42.934276 1104534 exec_runner.go:144] found /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem, removing ...
I0127 12:10:42.934286 1104534 exec_runner.go:203] rm: /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem
I0127 12:10:42.934309 1104534 exec_runner.go:151] cp: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/20318-888339/.minikube/key.pem (1675 bytes)
I0127 12:10:42.934379 1104534 provision.go:117] generating server cert: /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem org=jenkins.no-preload-835765 san=[127.0.0.1 192.168.85.2 localhost minikube no-preload-835765]
I0127 12:10:43.477185 1104534 provision.go:177] copyRemoteCerts
I0127 12:10:43.477355 1104534 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0127 12:10:43.477476 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:43.495765 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:43.586538 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0127 12:10:43.619605 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0127 12:10:43.646951 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0127 12:10:43.681462 1104534 provision.go:87] duration metric: took 768.06575ms to configureAuth
I0127 12:10:43.681493 1104534 ubuntu.go:193] setting minikube options for container-runtime
I0127 12:10:43.681698 1104534 config.go:182] Loaded profile config "no-preload-835765": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.32.1
I0127 12:10:43.681719 1104534 machine.go:96] duration metric: took 4.2214979s to provisionDockerMachine
I0127 12:10:43.681728 1104534 start.go:293] postStartSetup for "no-preload-835765" (driver="docker")
I0127 12:10:43.681745 1104534 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0127 12:10:43.681798 1104534 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0127 12:10:43.681843 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:43.699040 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:43.791620 1104534 ssh_runner.go:195] Run: cat /etc/os-release
I0127 12:10:43.795348 1104534 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0127 12:10:43.795387 1104534 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0127 12:10:43.795398 1104534 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0127 12:10:43.795405 1104534 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0127 12:10:43.795416 1104534 filesync.go:126] Scanning /home/jenkins/minikube-integration/20318-888339/.minikube/addons for local assets ...
I0127 12:10:43.795479 1104534 filesync.go:126] Scanning /home/jenkins/minikube-integration/20318-888339/.minikube/files for local assets ...
I0127 12:10:43.795563 1104534 filesync.go:149] local asset: /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem -> 8937152.pem in /etc/ssl/certs
I0127 12:10:43.795666 1104534 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0127 12:10:43.805879 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem --> /etc/ssl/certs/8937152.pem (1708 bytes)
I0127 12:10:43.832485 1104534 start.go:296] duration metric: took 150.734927ms for postStartSetup
I0127 12:10:43.832563 1104534 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0127 12:10:43.832611 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:43.850375 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:43.938544 1104534 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0127 12:10:43.943488 1104534 fix.go:56] duration metric: took 4.886362521s for fixHost
I0127 12:10:43.943514 1104534 start.go:83] releasing machines lock for "no-preload-835765", held for 4.886414918s
I0127 12:10:43.943585 1104534 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-835765
I0127 12:10:43.961374 1104534 ssh_runner.go:195] Run: cat /version.json
I0127 12:10:43.961453 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:43.961374 1104534 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0127 12:10:43.961540 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:43.981835 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:43.989140 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:44.228184 1104534 ssh_runner.go:195] Run: systemctl --version
I0127 12:10:44.232718 1104534 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0127 12:10:44.237306 1104534 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0127 12:10:44.256278 1104534 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0127 12:10:44.256385 1104534 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0127 12:10:44.265277 1104534 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0127 12:10:44.265359 1104534 start.go:495] detecting cgroup driver to use...
I0127 12:10:44.265399 1104534 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0127 12:10:44.265460 1104534 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0127 12:10:44.279509 1104534 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0127 12:10:44.296653 1104534 docker.go:217] disabling cri-docker service (if available) ...
I0127 12:10:44.296774 1104534 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0127 12:10:44.310240 1104534 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0127 12:10:44.324914 1104534 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0127 12:10:44.418938 1104534 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0127 12:10:44.504491 1104534 docker.go:233] disabling docker service ...
I0127 12:10:44.504571 1104534 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0127 12:10:44.519281 1104534 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0127 12:10:44.532313 1104534 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0127 12:10:44.627377 1104534 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0127 12:10:44.721340 1104534 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0127 12:10:44.736126 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0127 12:10:44.756045 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0127 12:10:44.767553 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0127 12:10:44.777471 1104534 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0127 12:10:44.777596 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0127 12:10:44.790792 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0127 12:10:44.801343 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0127 12:10:44.812744 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0127 12:10:44.826111 1104534 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0127 12:10:44.835823 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0127 12:10:44.846915 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0127 12:10:44.859280 1104534 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0127 12:10:44.870391 1104534 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0127 12:10:44.880582 1104534 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0127 12:10:44.889420 1104534 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:10:44.974974 1104534 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0127 12:10:45.206192 1104534 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0127 12:10:45.206368 1104534 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0127 12:10:45.212092 1104534 start.go:563] Will wait 60s for crictl version
I0127 12:10:45.212221 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:10:45.216766 1104534 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0127 12:10:45.285758 1104534 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.24
RuntimeApiVersion: v1
I0127 12:10:45.285851 1104534 ssh_runner.go:195] Run: containerd --version
I0127 12:10:45.312643 1104534 ssh_runner.go:195] Run: containerd --version
I0127 12:10:45.349400 1104534 out.go:177] * Preparing Kubernetes v1.32.1 on containerd 1.7.24 ...
I0127 12:10:45.352258 1104534 cli_runner.go:164] Run: docker network inspect no-preload-835765 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0127 12:10:45.373997 1104534 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0127 12:10:45.378566 1104534 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0127 12:10:45.390499 1104534 kubeadm.go:883] updating cluster {Name:no-preload-835765 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.32.1 ClusterName:no-preload-835765 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServer
IPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.32.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount
9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0127 12:10:45.390622 1104534 preload.go:131] Checking if preload exists for k8s version v1.32.1 and runtime containerd
I0127 12:10:45.390673 1104534 ssh_runner.go:195] Run: sudo crictl images --output json
I0127 12:10:45.431522 1104534 containerd.go:627] all images are preloaded for containerd runtime.
I0127 12:10:45.431547 1104534 cache_images.go:84] Images are preloaded, skipping loading
I0127 12:10:45.431556 1104534 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.32.1 containerd true true} ...
I0127 12:10:45.431662 1104534 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.32.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=no-preload-835765 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.32.1 ClusterName:no-preload-835765 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0127 12:10:45.431728 1104534 ssh_runner.go:195] Run: sudo crictl info
I0127 12:10:45.470564 1104534 cni.go:84] Creating CNI manager for ""
I0127 12:10:45.470638 1104534 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0127 12:10:45.470668 1104534 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0127 12:10:45.470720 1104534 kubeadm.go:189] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.32.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-835765 NodeName:no-preload-835765 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP
ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0127 12:10:45.470894 1104534 kubeadm.go:195] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "no-preload-835765"
kubeletExtraArgs:
- name: "node-ip"
value: "192.168.85.2"
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
- name: "enable-admission-plugins"
value: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
- name: "allocate-node-cidrs"
value: "true"
- name: "leader-elect"
value: "false"
scheduler:
extraArgs:
- name: "leader-elect"
value: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
- name: "proxy-refresh-interval"
value: "70000"
kubernetesVersion: v1.32.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0127 12:10:45.471004 1104534 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.32.1
I0127 12:10:45.483523 1104534 binaries.go:44] Found k8s binaries, skipping transfer
I0127 12:10:45.483626 1104534 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0127 12:10:45.493178 1104534 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (321 bytes)
I0127 12:10:45.511350 1104534 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0127 12:10:45.530108 1104534 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2307 bytes)
I0127 12:10:45.550896 1104534 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0127 12:10:45.554600 1104534 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0127 12:10:45.565730 1104534 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:10:45.651598 1104534 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0127 12:10:45.666456 1104534 certs.go:68] Setting up /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765 for IP: 192.168.85.2
I0127 12:10:45.666480 1104534 certs.go:194] generating shared ca certs ...
I0127 12:10:45.666495 1104534 certs.go:226] acquiring lock for ca certs: {Name:mke15f79704ae0e83f911aa0e3f9c4b862da9341 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:10:45.666626 1104534 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/20318-888339/.minikube/ca.key
I0127 12:10:45.666682 1104534 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.key
I0127 12:10:45.666694 1104534 certs.go:256] generating profile certs ...
I0127 12:10:45.666787 1104534 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/client.key
I0127 12:10:45.666850 1104534 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/apiserver.key.a1be0775
I0127 12:10:45.666903 1104534 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/proxy-client.key
I0127 12:10:45.667026 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715.pem (1338 bytes)
W0127 12:10:45.667058 1104534 certs.go:480] ignoring /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715_empty.pem, impossibly tiny 0 bytes
I0127 12:10:45.667071 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca-key.pem (1679 bytes)
I0127 12:10:45.667118 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/ca.pem (1082 bytes)
I0127 12:10:45.667153 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/cert.pem (1123 bytes)
I0127 12:10:45.667178 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/certs/key.pem (1675 bytes)
I0127 12:10:45.667228 1104534 certs.go:484] found cert: /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem (1708 bytes)
I0127 12:10:45.668430 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0127 12:10:45.699255 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0127 12:10:45.725238 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0127 12:10:45.753007 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0127 12:10:45.780111 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0127 12:10:45.819608 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0127 12:10:45.854794 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0127 12:10:45.902427 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/profiles/no-preload-835765/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0127 12:10:45.937357 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/files/etc/ssl/certs/8937152.pem --> /usr/share/ca-certificates/8937152.pem (1708 bytes)
I0127 12:10:45.964302 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0127 12:10:45.992805 1104534 ssh_runner.go:362] scp /home/jenkins/minikube-integration/20318-888339/.minikube/certs/893715.pem --> /usr/share/ca-certificates/893715.pem (1338 bytes)
I0127 12:10:46.028102 1104534 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0127 12:10:46.050120 1104534 ssh_runner.go:195] Run: openssl version
I0127 12:10:46.057725 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0127 12:10:46.068624 1104534 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0127 12:10:46.072491 1104534 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Jan 27 11:23 /usr/share/ca-certificates/minikubeCA.pem
I0127 12:10:46.072560 1104534 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0127 12:10:46.080487 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0127 12:10:46.097736 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/893715.pem && ln -fs /usr/share/ca-certificates/893715.pem /etc/ssl/certs/893715.pem"
I0127 12:10:46.107964 1104534 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/893715.pem
I0127 12:10:46.111851 1104534 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Jan 27 11:31 /usr/share/ca-certificates/893715.pem
I0127 12:10:46.111920 1104534 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/893715.pem
I0127 12:10:46.119607 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/893715.pem /etc/ssl/certs/51391683.0"
I0127 12:10:46.128975 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/8937152.pem && ln -fs /usr/share/ca-certificates/8937152.pem /etc/ssl/certs/8937152.pem"
I0127 12:10:46.139301 1104534 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/8937152.pem
I0127 12:10:46.143015 1104534 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Jan 27 11:31 /usr/share/ca-certificates/8937152.pem
I0127 12:10:46.143110 1104534 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/8937152.pem
I0127 12:10:46.150353 1104534 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/8937152.pem /etc/ssl/certs/3ec20f2e.0"
I0127 12:10:46.159785 1104534 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0127 12:10:46.163389 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0127 12:10:46.170391 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0127 12:10:46.178687 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0127 12:10:46.186905 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0127 12:10:46.194508 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0127 12:10:46.201357 1104534 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0127 12:10:46.208238 1104534 kubeadm.go:392] StartCluster: {Name:no-preload-835765 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.46@sha256:fd2d445ddcc33ebc5c6b68a17e6219ea207ce63c005095ea1525296da2d1a279 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.32.1 ClusterName:no-preload-835765 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs
:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.32.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PV
ersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0127 12:10:46.208339 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0127 12:10:46.208410 1104534 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0127 12:10:46.252505 1104534 cri.go:89] found id: "9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:10:46.252530 1104534 cri.go:89] found id: "b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:10:46.252538 1104534 cri.go:89] found id: "0f07c42b866dc5ced085b913bc0032794fa31172869fd8b902b40cdea9c9c1e0"
I0127 12:10:46.252550 1104534 cri.go:89] found id: "b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:10:46.252554 1104534 cri.go:89] found id: "e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:10:46.252557 1104534 cri.go:89] found id: "d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:10:46.252561 1104534 cri.go:89] found id: "1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:10:46.252564 1104534 cri.go:89] found id: "e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:10:46.252567 1104534 cri.go:89] found id: ""
I0127 12:10:46.252622 1104534 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
W0127 12:10:46.265812 1104534 kubeadm.go:399] unpause failed: list paused: runc: sudo runc --root /run/containerd/runc/k8s.io list -f json: Process exited with status 1
stdout:
stderr:
time="2025-01-27T12:10:46Z" level=error msg="open /run/containerd/runc/k8s.io: no such file or directory"
I0127 12:10:46.265892 1104534 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0127 12:10:46.274846 1104534 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0127 12:10:46.274868 1104534 kubeadm.go:593] restartPrimaryControlPlane start ...
I0127 12:10:46.274945 1104534 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0127 12:10:46.284259 1104534 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0127 12:10:46.284870 1104534 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-835765" does not appear in /home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:10:46.285177 1104534 kubeconfig.go:62] /home/jenkins/minikube-integration/20318-888339/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-835765" cluster setting kubeconfig missing "no-preload-835765" context setting]
I0127 12:10:46.286476 1104534 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/20318-888339/kubeconfig: {Name:mk75ddd380b783b9f157e482ffdcc29dbd635876 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:10:46.287931 1104534 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0127 12:10:46.315257 1104534 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0127 12:10:46.315293 1104534 kubeadm.go:597] duration metric: took 40.420055ms to restartPrimaryControlPlane
I0127 12:10:46.315304 1104534 kubeadm.go:394] duration metric: took 107.076173ms to StartCluster
I0127 12:10:46.315319 1104534 settings.go:142] acquiring lock: {Name:mk8e4620a376eeb900823ad35149c0dd6d301c83 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:10:46.315382 1104534 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/20318-888339/kubeconfig
I0127 12:10:46.316309 1104534 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/20318-888339/kubeconfig: {Name:mk75ddd380b783b9f157e482ffdcc29dbd635876 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0127 12:10:46.316518 1104534 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.32.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0127 12:10:46.316849 1104534 config.go:182] Loaded profile config "no-preload-835765": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.32.1
I0127 12:10:46.316914 1104534 addons.go:511] enable addons start: toEnable=map[ambassador:false amd-gpu-device-plugin:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0127 12:10:46.316991 1104534 addons.go:69] Setting storage-provisioner=true in profile "no-preload-835765"
I0127 12:10:46.317005 1104534 addons.go:238] Setting addon storage-provisioner=true in "no-preload-835765"
W0127 12:10:46.317189 1104534 addons.go:247] addon storage-provisioner should already be in state true
I0127 12:10:46.317222 1104534 host.go:66] Checking if "no-preload-835765" exists ...
I0127 12:10:46.317156 1104534 addons.go:69] Setting dashboard=true in profile "no-preload-835765"
I0127 12:10:46.317413 1104534 addons.go:238] Setting addon dashboard=true in "no-preload-835765"
W0127 12:10:46.317422 1104534 addons.go:247] addon dashboard should already be in state true
I0127 12:10:46.317475 1104534 host.go:66] Checking if "no-preload-835765" exists ...
I0127 12:10:46.317693 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:46.318083 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:46.317171 1104534 addons.go:69] Setting default-storageclass=true in profile "no-preload-835765"
I0127 12:10:46.318385 1104534 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-835765"
I0127 12:10:46.319257 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:46.317179 1104534 addons.go:69] Setting metrics-server=true in profile "no-preload-835765"
I0127 12:10:46.322096 1104534 addons.go:238] Setting addon metrics-server=true in "no-preload-835765"
W0127 12:10:46.322124 1104534 addons.go:247] addon metrics-server should already be in state true
I0127 12:10:46.322182 1104534 host.go:66] Checking if "no-preload-835765" exists ...
I0127 12:10:46.322678 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:46.326200 1104534 out.go:177] * Verifying Kubernetes components...
I0127 12:10:46.337564 1104534 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0127 12:10:46.384223 1104534 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0127 12:10:46.388099 1104534 addons.go:435] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:10:46.388123 1104534 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0127 12:10:46.388192 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:46.413657 1104534 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0127 12:10:46.417054 1104534 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0127 12:10:46.419619 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0127 12:10:46.419643 1104534 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0127 12:10:46.419713 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:46.422453 1104534 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0127 12:10:43.741550 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:45.741618 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:47.745656 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:46.423512 1104534 addons.go:238] Setting addon default-storageclass=true in "no-preload-835765"
W0127 12:10:46.424719 1104534 addons.go:247] addon default-storageclass should already be in state true
I0127 12:10:46.424752 1104534 host.go:66] Checking if "no-preload-835765" exists ...
I0127 12:10:46.425445 1104534 cli_runner.go:164] Run: docker container inspect no-preload-835765 --format={{.State.Status}}
I0127 12:10:46.426237 1104534 addons.go:435] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0127 12:10:46.426258 1104534 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0127 12:10:46.426312 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:46.434763 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:46.486009 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:46.486248 1104534 addons.go:435] installing /etc/kubernetes/addons/storageclass.yaml
I0127 12:10:46.486262 1104534 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0127 12:10:46.486332 1104534 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-835765
I0127 12:10:46.488421 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:46.519103 1104534 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33862 SSHKeyPath:/home/jenkins/minikube-integration/20318-888339/.minikube/machines/no-preload-835765/id_rsa Username:docker}
I0127 12:10:46.544668 1104534 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0127 12:10:46.592618 1104534 node_ready.go:35] waiting up to 6m0s for node "no-preload-835765" to be "Ready" ...
I0127 12:10:46.716165 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0127 12:10:46.716186 1104534 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0127 12:10:46.753719 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:10:46.813417 1104534 addons.go:435] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0127 12:10:46.813485 1104534 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0127 12:10:46.834049 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0127 12:10:46.834129 1104534 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0127 12:10:46.835362 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0127 12:10:46.909466 1104534 addons.go:435] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0127 12:10:46.909568 1104534 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0127 12:10:46.969073 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0127 12:10:46.969144 1104534 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0127 12:10:47.066287 1104534 addons.go:435] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0127 12:10:47.066370 1104534 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0127 12:10:47.122712 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0127 12:10:47.122795 1104534 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0127 12:10:47.206676 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-role.yaml
I0127 12:10:47.206701 1104534 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0127 12:10:47.287967 1104534 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0127 12:10:47.288055 1104534 retry.go:31] will retry after 323.375103ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0127 12:10:47.315759 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0127 12:10:47.397621 1104534 addons.go:461] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0127 12:10:47.397704 1104534 retry.go:31] will retry after 338.042784ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0127 12:10:47.408297 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0127 12:10:47.408371 1104534 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0127 12:10:47.513762 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0127 12:10:47.513788 1104534 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0127 12:10:47.591628 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0127 12:10:47.591654 1104534 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0127 12:10:47.611581 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0127 12:10:47.714382 1104534 addons.go:435] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:10:47.714419 1104534 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0127 12:10:47.736808 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0127 12:10:47.874952 1104534 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0127 12:10:50.240430 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:52.240688 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:51.803948 1104534 node_ready.go:49] node "no-preload-835765" has status "Ready":"True"
I0127 12:10:51.803976 1104534 node_ready.go:38] duration metric: took 5.211318882s for node "no-preload-835765" to be "Ready" ...
I0127 12:10:51.803986 1104534 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0127 12:10:51.883904 1104534 pod_ready.go:79] waiting up to 6m0s for pod "coredns-668d6bf9bc-8wfgw" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.924217 1104534 pod_ready.go:93] pod "coredns-668d6bf9bc-8wfgw" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:51.924245 1104534 pod_ready.go:82] duration metric: took 40.305515ms for pod "coredns-668d6bf9bc-8wfgw" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.924257 1104534 pod_ready.go:79] waiting up to 6m0s for pod "etcd-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.955276 1104534 pod_ready.go:93] pod "etcd-no-preload-835765" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:51.955302 1104534 pod_ready.go:82] duration metric: took 31.037455ms for pod "etcd-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.955320 1104534 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.969520 1104534 pod_ready.go:93] pod "kube-apiserver-no-preload-835765" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:51.969547 1104534 pod_ready.go:82] duration metric: took 14.219366ms for pod "kube-apiserver-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.969559 1104534 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.983523 1104534 pod_ready.go:93] pod "kube-controller-manager-no-preload-835765" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:51.983551 1104534 pod_ready.go:82] duration metric: took 13.982695ms for pod "kube-controller-manager-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:51.983564 1104534 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-6j77q" in "kube-system" namespace to be "Ready" ...
I0127 12:10:52.015527 1104534 pod_ready.go:93] pod "kube-proxy-6j77q" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:52.015555 1104534 pod_ready.go:82] duration metric: took 31.98318ms for pod "kube-proxy-6j77q" in "kube-system" namespace to be "Ready" ...
I0127 12:10:52.015568 1104534 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:52.408574 1104534 pod_ready.go:93] pod "kube-scheduler-no-preload-835765" in "kube-system" namespace has status "Ready":"True"
I0127 12:10:52.408601 1104534 pod_ready.go:82] duration metric: took 393.023075ms for pod "kube-scheduler-no-preload-835765" in "kube-system" namespace to be "Ready" ...
I0127 12:10:52.408615 1104534 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace to be "Ready" ...
I0127 12:10:54.416302 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:55.325625 1104534 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (8.009779225s)
I0127 12:10:55.325661 1104534 addons.go:479] Verifying addon metrics-server=true in "no-preload-835765"
I0127 12:10:55.363443 1104534 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (7.751820329s)
I0127 12:10:55.363502 1104534 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (7.626650792s)
I0127 12:10:55.363589 1104534 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.32.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.488605964s)
I0127 12:10:55.366673 1104534 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-835765 addons enable metrics-server
I0127 12:10:55.372353 1104534 out.go:177] * Enabled addons: metrics-server, storage-provisioner, dashboard, default-storageclass
I0127 12:10:54.741996 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:57.239927 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:55.374967 1104534 addons.go:514] duration metric: took 9.058048039s for enable addons: enabled=[metrics-server storage-provisioner dashboard default-storageclass]
I0127 12:10:56.936967 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:59.240275 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:01.241298 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:10:59.414939 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:01.416389 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:03.426544 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:03.740357 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:06.239980 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:05.915405 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:07.915533 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:08.741190 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:10.741646 1099122 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:11.741804 1099122 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:11.741830 1099122 pod_ready.go:82] duration metric: took 1m8.008379088s for pod "kube-controller-manager-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.741843 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-nt2l9" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.747091 1099122 pod_ready.go:93] pod "kube-proxy-nt2l9" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:11.747117 1099122 pod_ready.go:82] duration metric: took 5.26649ms for pod "kube-proxy-nt2l9" in "kube-system" namespace to be "Ready" ...
I0127 12:11:11.747129 1099122 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:10.415644 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:12.914568 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:13.753603 1099122 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace has status "Ready":"True"
I0127 12:11:13.753625 1099122 pod_ready.go:82] duration metric: took 2.006488609s for pod "kube-scheduler-old-k8s-version-999803" in "kube-system" namespace to be "Ready" ...
I0127 12:11:13.753636 1099122 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace to be "Ready" ...
I0127 12:11:15.760059 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:17.760104 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:15.414556 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:17.415296 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:20.259362 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:22.259503 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:19.915237 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:22.414659 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:24.265513 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:26.759839 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:24.414703 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:26.415033 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:29.259521 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:31.759761 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:28.915350 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:31.415056 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:33.415268 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:33.760790 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:36.259938 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:38.260402 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:35.415391 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:37.914181 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:40.260435 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:42.261372 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:39.914398 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:41.915143 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:44.759807 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:46.760139 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:44.416783 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:46.914380 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:49.259135 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:51.260408 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:48.915266 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:51.416394 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:53.761606 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:56.262295 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:53.914602 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:55.915025 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:58.414880 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:11:58.760329 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:00.760445 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:03.259143 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:00.416034 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:02.915057 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:05.260247 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:07.759197 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:05.414974 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:07.415077 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:09.760214 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:11.760909 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:09.914864 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:11.915239 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:14.259817 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:16.260506 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:14.414612 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:16.414845 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:18.766515 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:21.259066 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:23.259646 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:18.914824 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:21.414566 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:25.259764 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:27.259886 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:23.915105 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:26.414708 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:28.415044 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:29.758956 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:31.765446 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:30.914568 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:32.915072 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:34.259539 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:36.259690 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:38.259810 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:35.414995 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:37.917578 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:40.265756 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:42.759695 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:40.414897 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:42.914760 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:44.760001 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:47.264085 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:45.414031 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:47.414286 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:49.759794 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:51.760814 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:49.414937 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:51.415236 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:54.260573 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:56.261649 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:53.914997 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:55.915775 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:58.414565 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:12:58.760562 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:01.260514 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:00.414786 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:02.415180 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:03.760846 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:06.259692 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:08.259809 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:04.415454 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:06.415535 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:10.264009 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:12.760961 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:08.914755 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:11.414542 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:13.418303 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:15.259543 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:17.259672 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:15.914779 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:17.915250 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:19.266685 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:21.760466 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:20.414958 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:22.914725 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:23.760739 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:26.260232 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:24.915208 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:27.415392 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:28.759672 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:30.759737 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:32.760624 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:29.914226 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:31.920853 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:35.260200 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:37.760079 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:34.415357 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:36.415527 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:39.760771 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:42.261258 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:38.915074 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:40.915321 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:42.919024 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:44.760471 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:47.260741 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:45.414581 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:47.415198 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:49.760653 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:51.761150 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:49.914817 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:52.414019 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:54.260064 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:56.260168 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:54.414449 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:56.914336 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:58.759553 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:00.760997 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:03.259192 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:13:58.914511 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:00.914932 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:02.915016 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:05.260842 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:07.759902 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:04.916117 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:07.414717 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:09.760696 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:11.761676 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:09.914790 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:11.914897 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:14.259176 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:16.260538 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:13.915152 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:15.915220 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:17.916494 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:18.760162 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:20.761412 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:23.259212 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:20.417172 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:22.914389 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:25.259378 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:27.260207 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:25.414391 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:27.914990 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:29.761136 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:32.259719 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:30.415746 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:32.914428 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:34.260064 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:36.760335 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:34.924137 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:37.414702 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:38.760390 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:41.260552 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:43.260596 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:39.415266 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:41.914587 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:45.305357 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:47.761135 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:43.915452 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:46.415069 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:49.762322 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:52.262267 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:48.914614 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:51.415286 1104534 pod_ready.go:103] pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:52.409492 1104534 pod_ready.go:82] duration metric: took 4m0.000857266s for pod "metrics-server-f79f97bbb-kzh9d" in "kube-system" namespace to be "Ready" ...
E0127 12:14:52.409551 1104534 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0127 12:14:52.409563 1104534 pod_ready.go:39] duration metric: took 4m0.605564558s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0127 12:14:52.409581 1104534 api_server.go:52] waiting for apiserver process to appear ...
I0127 12:14:52.409615 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:14:52.409692 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:14:52.468089 1104534 cri.go:89] found id: "1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:14:52.468113 1104534 cri.go:89] found id: "1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:14:52.468118 1104534 cri.go:89] found id: ""
I0127 12:14:52.468126 1104534 logs.go:282] 2 containers: [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e]
I0127 12:14:52.468180 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.472041 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.475897 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:14:52.475966 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:14:52.521693 1104534 cri.go:89] found id: "483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:14:52.521713 1104534 cri.go:89] found id: "e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:14:52.521718 1104534 cri.go:89] found id: ""
I0127 12:14:52.521725 1104534 logs.go:282] 2 containers: [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38]
I0127 12:14:52.521783 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.526901 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.530874 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:14:52.530943 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:14:52.581693 1104534 cri.go:89] found id: "509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:14:52.581718 1104534 cri.go:89] found id: "9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:14:52.581724 1104534 cri.go:89] found id: ""
I0127 12:14:52.581732 1104534 logs.go:282] 2 containers: [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af]
I0127 12:14:52.581788 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.585405 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.588979 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:14:52.589105 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:14:52.628610 1104534 cri.go:89] found id: "9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:14:52.628646 1104534 cri.go:89] found id: "d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:14:52.628650 1104534 cri.go:89] found id: ""
I0127 12:14:52.628657 1104534 logs.go:282] 2 containers: [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6]
I0127 12:14:52.628717 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.632373 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.635897 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:14:52.635973 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:14:52.674828 1104534 cri.go:89] found id: "cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:14:52.674902 1104534 cri.go:89] found id: "b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:14:52.674915 1104534 cri.go:89] found id: ""
I0127 12:14:52.674924 1104534 logs.go:282] 2 containers: [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7]
I0127 12:14:52.674993 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.678483 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.681899 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:14:52.681984 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:14:52.722129 1104534 cri.go:89] found id: "32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:14:52.722152 1104534 cri.go:89] found id: "e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:14:52.722156 1104534 cri.go:89] found id: ""
I0127 12:14:52.722164 1104534 logs.go:282] 2 containers: [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487]
I0127 12:14:52.722243 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.726009 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.729362 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:14:52.729431 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:14:52.781432 1104534 cri.go:89] found id: "fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:14:52.781458 1104534 cri.go:89] found id: "b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:14:52.781463 1104534 cri.go:89] found id: ""
I0127 12:14:52.781471 1104534 logs.go:282] 2 containers: [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9]
I0127 12:14:52.781542 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.785249 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.790278 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:14:52.790398 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:14:52.837147 1104534 cri.go:89] found id: "cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:14:52.837167 1104534 cri.go:89] found id: ""
I0127 12:14:52.837175 1104534 logs.go:282] 1 containers: [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5]
I0127 12:14:52.837230 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.842098 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:14:52.842166 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:14:52.886268 1104534 cri.go:89] found id: "f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:14:52.886298 1104534 cri.go:89] found id: "14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:14:52.886304 1104534 cri.go:89] found id: ""
I0127 12:14:52.886311 1104534 logs.go:282] 2 containers: [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37]
I0127 12:14:52.886381 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.890198 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:14:52.893680 1104534 logs.go:123] Gathering logs for kube-controller-manager [e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487] ...
I0127 12:14:52.893701 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:14:52.971483 1104534 logs.go:123] Gathering logs for kindnet [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6] ...
I0127 12:14:52.971516 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:14:53.026229 1104534 logs.go:123] Gathering logs for storage-provisioner [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333] ...
I0127 12:14:53.026262 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:14:53.074822 1104534 logs.go:123] Gathering logs for storage-provisioner [14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37] ...
I0127 12:14:53.074854 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:14:53.120261 1104534 logs.go:123] Gathering logs for container status ...
I0127 12:14:53.120289 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:14:53.167689 1104534 logs.go:123] Gathering logs for describe nodes ...
I0127 12:14:53.167718 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.32.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:14:53.313357 1104534 logs.go:123] Gathering logs for coredns [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25] ...
I0127 12:14:53.313387 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:14:53.355258 1104534 logs.go:123] Gathering logs for kube-proxy [b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7] ...
I0127 12:14:53.355296 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:14:53.398790 1104534 logs.go:123] Gathering logs for kube-controller-manager [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191] ...
I0127 12:14:53.398820 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:14:53.467891 1104534 logs.go:123] Gathering logs for kindnet [b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9] ...
I0127 12:14:53.467926 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:14:53.516544 1104534 logs.go:123] Gathering logs for dmesg ...
I0127 12:14:53.516576 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:14:53.535166 1104534 logs.go:123] Gathering logs for kube-apiserver [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c] ...
I0127 12:14:53.535196 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:14:53.595276 1104534 logs.go:123] Gathering logs for kube-apiserver [1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e] ...
I0127 12:14:53.595317 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:14:53.647355 1104534 logs.go:123] Gathering logs for coredns [9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af] ...
I0127 12:14:53.647390 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:14:53.687311 1104534 logs.go:123] Gathering logs for kube-scheduler [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863] ...
I0127 12:14:53.687339 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:14:53.724606 1104534 logs.go:123] Gathering logs for kube-scheduler [d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6] ...
I0127 12:14:53.724634 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:14:53.788367 1104534 logs.go:123] Gathering logs for kube-proxy [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388] ...
I0127 12:14:53.788405 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:14:54.760292 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:57.260208 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:14:53.833338 1104534 logs.go:123] Gathering logs for kubernetes-dashboard [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5] ...
I0127 12:14:53.833416 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:14:53.878744 1104534 logs.go:123] Gathering logs for kubelet ...
I0127 12:14:53.878826 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:14:53.937189 1104534 logs.go:138] Found kubelet problem: Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:14:53.970519 1104534 logs.go:123] Gathering logs for etcd [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f] ...
I0127 12:14:53.970556 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:14:54.019375 1104534 logs.go:123] Gathering logs for etcd [e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38] ...
I0127 12:14:54.019407 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:14:54.078194 1104534 logs.go:123] Gathering logs for containerd ...
I0127 12:14:54.078224 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:14:54.151084 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:14:54.151122 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:14:54.151198 1104534 out.go:270] X Problems detected in kubelet:
W0127 12:14:54.151209 1104534 out.go:270] Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:14:54.151215 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:14:54.151230 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:14:59.760442 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:01.760706 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:04.260789 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:06.759923 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:04.153767 1104534 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0127 12:15:04.165808 1104534 api_server.go:72] duration metric: took 4m17.84925442s to wait for apiserver process to appear ...
I0127 12:15:04.165833 1104534 api_server.go:88] waiting for apiserver healthz status ...
I0127 12:15:04.165867 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:04.165926 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:04.206360 1104534 cri.go:89] found id: "1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:15:04.206379 1104534 cri.go:89] found id: "1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:15:04.206384 1104534 cri.go:89] found id: ""
I0127 12:15:04.206391 1104534 logs.go:282] 2 containers: [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e]
I0127 12:15:04.206446 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.210163 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.213563 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:04.213630 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:04.265694 1104534 cri.go:89] found id: "483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:15:04.265716 1104534 cri.go:89] found id: "e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:15:04.265721 1104534 cri.go:89] found id: ""
I0127 12:15:04.265728 1104534 logs.go:282] 2 containers: [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38]
I0127 12:15:04.265796 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.269778 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.273247 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:04.273325 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:04.325226 1104534 cri.go:89] found id: "509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:15:04.325246 1104534 cri.go:89] found id: "9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:15:04.325250 1104534 cri.go:89] found id: ""
I0127 12:15:04.325257 1104534 logs.go:282] 2 containers: [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af]
I0127 12:15:04.325313 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.329161 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.332948 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:04.333120 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:04.376122 1104534 cri.go:89] found id: "9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:15:04.376144 1104534 cri.go:89] found id: "d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:15:04.376148 1104534 cri.go:89] found id: ""
I0127 12:15:04.376155 1104534 logs.go:282] 2 containers: [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6]
I0127 12:15:04.376214 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.379955 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.383516 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:04.383602 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:04.421817 1104534 cri.go:89] found id: "cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:15:04.421892 1104534 cri.go:89] found id: "b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:15:04.421904 1104534 cri.go:89] found id: ""
I0127 12:15:04.421912 1104534 logs.go:282] 2 containers: [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7]
I0127 12:15:04.421983 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.427341 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.431155 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:04.431250 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:04.469681 1104534 cri.go:89] found id: "32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:15:04.469703 1104534 cri.go:89] found id: "e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:15:04.469708 1104534 cri.go:89] found id: ""
I0127 12:15:04.469715 1104534 logs.go:282] 2 containers: [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487]
I0127 12:15:04.469818 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.473314 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.476616 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:04.476707 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:04.526341 1104534 cri.go:89] found id: "fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:15:04.526361 1104534 cri.go:89] found id: "b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:15:04.526366 1104534 cri.go:89] found id: ""
I0127 12:15:04.526373 1104534 logs.go:282] 2 containers: [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9]
I0127 12:15:04.526429 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.530577 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.534415 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:04.534511 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:04.580996 1104534 cri.go:89] found id: "f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:15:04.581023 1104534 cri.go:89] found id: "14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:15:04.581066 1104534 cri.go:89] found id: ""
I0127 12:15:04.581073 1104534 logs.go:282] 2 containers: [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37]
I0127 12:15:04.581130 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.584898 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.588862 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:04.588937 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:04.630840 1104534 cri.go:89] found id: "cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:15:04.630862 1104534 cri.go:89] found id: ""
I0127 12:15:04.630869 1104534 logs.go:282] 1 containers: [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5]
I0127 12:15:04.630943 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:04.634596 1104534 logs.go:123] Gathering logs for kube-apiserver [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c] ...
I0127 12:15:04.634620 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:15:04.702788 1104534 logs.go:123] Gathering logs for coredns [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25] ...
I0127 12:15:04.702826 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:15:04.750031 1104534 logs.go:123] Gathering logs for kindnet [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6] ...
I0127 12:15:04.750128 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:15:04.805002 1104534 logs.go:123] Gathering logs for containerd ...
I0127 12:15:04.805124 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:04.874471 1104534 logs.go:123] Gathering logs for storage-provisioner [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333] ...
I0127 12:15:04.874505 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:15:04.915226 1104534 logs.go:123] Gathering logs for storage-provisioner [14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37] ...
I0127 12:15:04.915253 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:15:04.965916 1104534 logs.go:123] Gathering logs for kubernetes-dashboard [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5] ...
I0127 12:15:04.965943 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:15:05.010093 1104534 logs.go:123] Gathering logs for kube-apiserver [1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e] ...
I0127 12:15:05.010197 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:15:05.064758 1104534 logs.go:123] Gathering logs for coredns [9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af] ...
I0127 12:15:05.064797 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:15:05.122869 1104534 logs.go:123] Gathering logs for kube-proxy [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388] ...
I0127 12:15:05.122939 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:15:05.165959 1104534 logs.go:123] Gathering logs for kube-controller-manager [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191] ...
I0127 12:15:05.165995 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:15:05.259097 1104534 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:05.259176 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:05.279066 1104534 logs.go:123] Gathering logs for kube-scheduler [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863] ...
I0127 12:15:05.279149 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:15:05.327280 1104534 logs.go:123] Gathering logs for kube-proxy [b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7] ...
I0127 12:15:05.327350 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:15:05.377189 1104534 logs.go:123] Gathering logs for kindnet [b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9] ...
I0127 12:15:05.377220 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:15:05.440485 1104534 logs.go:123] Gathering logs for kube-scheduler [d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6] ...
I0127 12:15:05.440514 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:15:05.506648 1104534 logs.go:123] Gathering logs for kube-controller-manager [e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487] ...
I0127 12:15:05.506681 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:15:05.583948 1104534 logs.go:123] Gathering logs for container status ...
I0127 12:15:05.583988 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:05.635807 1104534 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:05.635878 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:05.690586 1104534 logs.go:138] Found kubelet problem: Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:15:05.725729 1104534 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:05.725781 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.32.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:05.860466 1104534 logs.go:123] Gathering logs for etcd [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f] ...
I0127 12:15:05.860496 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:15:05.931631 1104534 logs.go:123] Gathering logs for etcd [e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38] ...
I0127 12:15:05.931676 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:15:05.991931 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:05.991962 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:05.992019 1104534 out.go:270] X Problems detected in kubelet:
W0127 12:15:05.992036 1104534 out.go:270] Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:15:05.992042 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:05.992048 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:08.761236 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:10.761497 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:13.259288 1099122 pod_ready.go:103] pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace has status "Ready":"False"
I0127 12:15:13.754136 1099122 pod_ready.go:82] duration metric: took 4m0.000481802s for pod "metrics-server-9975d5f86-qzhdd" in "kube-system" namespace to be "Ready" ...
E0127 12:15:13.754168 1099122 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0127 12:15:13.754179 1099122 pod_ready.go:39] duration metric: took 5m20.463611828s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0127 12:15:13.754196 1099122 api_server.go:52] waiting for apiserver process to appear ...
I0127 12:15:13.754230 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:13.754302 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:13.794220 1099122 cri.go:89] found id: "709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:13.794243 1099122 cri.go:89] found id: "f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:13.794248 1099122 cri.go:89] found id: ""
I0127 12:15:13.794255 1099122 logs.go:282] 2 containers: [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e]
I0127 12:15:13.794338 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.797981 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.801452 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:13.801523 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:13.839152 1099122 cri.go:89] found id: "8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:13.839179 1099122 cri.go:89] found id: "2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:13.839185 1099122 cri.go:89] found id: ""
I0127 12:15:13.839192 1099122 logs.go:282] 2 containers: [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1]
I0127 12:15:13.839249 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.842927 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.846323 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:13.846397 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:13.884753 1099122 cri.go:89] found id: "66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:13.884776 1099122 cri.go:89] found id: "006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:13.884781 1099122 cri.go:89] found id: ""
I0127 12:15:13.884787 1099122 logs.go:282] 2 containers: [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863]
I0127 12:15:13.884849 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.888585 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.892544 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:13.892620 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:13.935201 1099122 cri.go:89] found id: "15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:13.935265 1099122 cri.go:89] found id: "8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:13.935275 1099122 cri.go:89] found id: ""
I0127 12:15:13.935282 1099122 logs.go:282] 2 containers: [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b]
I0127 12:15:13.935348 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.938912 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.942321 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:13.942420 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:13.982013 1099122 cri.go:89] found id: "244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:13.982037 1099122 cri.go:89] found id: "69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:13.982042 1099122 cri.go:89] found id: ""
I0127 12:15:13.982049 1099122 logs.go:282] 2 containers: [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d]
I0127 12:15:13.982107 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.985808 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:13.989196 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:13.989297 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:14.034072 1099122 cri.go:89] found id: "35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:14.034096 1099122 cri.go:89] found id: "24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:14.034103 1099122 cri.go:89] found id: ""
I0127 12:15:14.034110 1099122 logs.go:282] 2 containers: [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84]
I0127 12:15:14.034175 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.038229 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.041981 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:14.042087 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:14.096637 1099122 cri.go:89] found id: "8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:14.096662 1099122 cri.go:89] found id: "92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:14.096667 1099122 cri.go:89] found id: ""
I0127 12:15:14.096674 1099122 logs.go:282] 2 containers: [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f]
I0127 12:15:14.096735 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.100700 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.104367 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:14.104440 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:14.142295 1099122 cri.go:89] found id: "d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:14.142318 1099122 cri.go:89] found id: ""
I0127 12:15:14.142341 1099122 logs.go:282] 1 containers: [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6]
I0127 12:15:14.142395 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.145773 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:14.145852 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:14.184211 1099122 cri.go:89] found id: "1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:14.184233 1099122 cri.go:89] found id: "072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:14.184239 1099122 cri.go:89] found id: ""
I0127 12:15:14.184246 1099122 logs.go:282] 2 containers: [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4]
I0127 12:15:14.184300 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.187804 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:14.191031 1099122 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:14.191103 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:14.210127 1099122 logs.go:123] Gathering logs for etcd [2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1] ...
I0127 12:15:14.210159 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:14.250486 1099122 logs.go:123] Gathering logs for kube-controller-manager [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d] ...
I0127 12:15:14.250517 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:14.310627 1099122 logs.go:123] Gathering logs for kubernetes-dashboard [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6] ...
I0127 12:15:14.310663 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:14.357158 1099122 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:14.357187 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:14.421142 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.065326 663 reflector.go:138] object-"kube-system"/"metrics-server-token-827qh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-827qh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421398 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.068245 663 reflector.go:138] object-"kube-system"/"kindnet-token-jxc27": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jxc27" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421622 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.072508 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-pzcvk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-pzcvk" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.421857 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.092394 663 reflector.go:138] object-"kube-system"/"coredns-token-m2lsh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2lsh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422066 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.102861 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422281 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.106409 663 reflector.go:138] object-"default"/"default-token-pmbfm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-pmbfm" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422509 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135445 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-bbnlz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-bbnlz" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.422714 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135737 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:14.431944 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:55 old-k8s-version-999803 kubelet[663]: E0127 12:09:55.695476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.432135 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:56 old-k8s-version-999803 kubelet[663]: E0127 12:09:56.587407 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.434973 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:08 old-k8s-version-999803 kubelet[663]: E0127 12:10:08.310539 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.436943 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:18 old-k8s-version-999803 kubelet[663]: E0127 12:10:18.685285 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.437410 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:19 old-k8s-version-999803 kubelet[663]: E0127 12:10:19.697364 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.437932 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:23 old-k8s-version-999803 kubelet[663]: E0127 12:10:23.289410 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.438369 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:27 old-k8s-version-999803 kubelet[663]: E0127 12:10:27.725508 663 pod_workers.go:191] Error syncing pod f73574be-9aec-4a33-ac88-97d900488a22 ("storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"
W0127 12:15:14.439000 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:29 old-k8s-version-999803 kubelet[663]: E0127 12:10:29.743840 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.441866 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:35 old-k8s-version-999803 kubelet[663]: E0127 12:10:35.298365 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.442236 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:38 old-k8s-version-999803 kubelet[663]: E0127 12:10:38.666640 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.442554 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:46 old-k8s-version-999803 kubelet[663]: E0127 12:10:46.289557 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.443148 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:53 old-k8s-version-999803 kubelet[663]: E0127 12:10:53.813459 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.443334 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:57 old-k8s-version-999803 kubelet[663]: E0127 12:10:57.289244 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.443693 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:58 old-k8s-version-999803 kubelet[663]: E0127 12:10:58.662563 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.443879 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.289633 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.444208 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.290664 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.446683 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:21 old-k8s-version-999803 kubelet[663]: E0127 12:11:21.295133 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.447018 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:25 old-k8s-version-999803 kubelet[663]: E0127 12:11:25.289132 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.447202 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:36 old-k8s-version-999803 kubelet[663]: E0127 12:11:36.291476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.447862 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:40 old-k8s-version-999803 kubelet[663]: E0127 12:11:40.940519 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448197 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:48 old-k8s-version-999803 kubelet[663]: E0127 12:11:48.662783 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448380 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:50 old-k8s-version-999803 kubelet[663]: E0127 12:11:50.289726 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.448722 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.288851 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.448905 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.289979 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.449125 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:14 old-k8s-version-999803 kubelet[663]: E0127 12:12:14.295027 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.449483 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:18 old-k8s-version-999803 kubelet[663]: E0127 12:12:18.288843 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.449672 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:26 old-k8s-version-999803 kubelet[663]: E0127 12:12:26.289650 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.450001 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:30 old-k8s-version-999803 kubelet[663]: E0127 12:12:30.289329 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.450184 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:37 old-k8s-version-999803 kubelet[663]: E0127 12:12:37.289235 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.450509 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:44 old-k8s-version-999803 kubelet[663]: E0127 12:12:44.288931 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.452938 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:50 old-k8s-version-999803 kubelet[663]: E0127 12:12:50.299810 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:14.453274 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:57 old-k8s-version-999803 kubelet[663]: E0127 12:12:57.288813 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.453460 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:03 old-k8s-version-999803 kubelet[663]: E0127 12:13:03.289391 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.454055 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:11 old-k8s-version-999803 kubelet[663]: E0127 12:13:11.199586 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.454245 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:14 old-k8s-version-999803 kubelet[663]: E0127 12:13:14.289503 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.454570 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:18 old-k8s-version-999803 kubelet[663]: E0127 12:13:18.662572 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.454756 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:29 old-k8s-version-999803 kubelet[663]: E0127 12:13:29.289301 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.455083 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:31 old-k8s-version-999803 kubelet[663]: E0127 12:13:31.288795 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.455286 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.289460 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.455615 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.290963 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.455808 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:53 old-k8s-version-999803 kubelet[663]: E0127 12:13:53.289152 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.456133 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:57 old-k8s-version-999803 kubelet[663]: E0127 12:13:57.288829 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.456320 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:07 old-k8s-version-999803 kubelet[663]: E0127 12:14:07.289177 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.456652 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: E0127 12:14:12.289332 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.456835 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:21 old-k8s-version-999803 kubelet[663]: E0127 12:14:21.289169 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.457166 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: E0127 12:14:26.288817 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.457351 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:32 old-k8s-version-999803 kubelet[663]: E0127 12:14:32.289382 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.457676 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: E0127 12:14:37.288718 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.457860 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.458184 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.458369 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:14.458694 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:14.458877 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:14.458888 1099122 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:14.458906 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:14.600554 1099122 logs.go:123] Gathering logs for coredns [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab] ...
I0127 12:15:14.600589 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:14.642894 1099122 logs.go:123] Gathering logs for kube-controller-manager [24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84] ...
I0127 12:15:14.642924 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:14.708414 1099122 logs.go:123] Gathering logs for etcd [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9] ...
I0127 12:15:14.708446 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:14.750297 1099122 logs.go:123] Gathering logs for coredns [006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863] ...
I0127 12:15:14.750327 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:14.787004 1099122 logs.go:123] Gathering logs for kube-scheduler [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47] ...
I0127 12:15:14.787031 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:14.828360 1099122 logs.go:123] Gathering logs for kube-proxy [69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d] ...
I0127 12:15:14.828389 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:14.873162 1099122 logs.go:123] Gathering logs for kindnet [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026] ...
I0127 12:15:14.873189 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:14.921463 1099122 logs.go:123] Gathering logs for kindnet [92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f] ...
I0127 12:15:14.921495 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:14.974595 1099122 logs.go:123] Gathering logs for storage-provisioner [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609] ...
I0127 12:15:14.974623 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:15.034157 1099122 logs.go:123] Gathering logs for storage-provisioner [072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4] ...
I0127 12:15:15.034189 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:15.077801 1099122 logs.go:123] Gathering logs for kube-apiserver [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f] ...
I0127 12:15:15.077829 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:15.150398 1099122 logs.go:123] Gathering logs for kube-apiserver [f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e] ...
I0127 12:15:15.150475 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:15.219967 1099122 logs.go:123] Gathering logs for kube-scheduler [8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b] ...
I0127 12:15:15.220021 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:15.267003 1099122 logs.go:123] Gathering logs for kube-proxy [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56] ...
I0127 12:15:15.267075 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:15.305261 1099122 logs.go:123] Gathering logs for containerd ...
I0127 12:15:15.305350 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:15.379123 1099122 logs.go:123] Gathering logs for container status ...
I0127 12:15:15.379163 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:15.437988 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:15.438018 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:15.438072 1099122 out.go:270] X Problems detected in kubelet:
W0127 12:15:15.438088 1099122 out.go:270] Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:15.438101 1099122 out.go:270] Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:15.438126 1099122 out.go:270] Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:15.438132 1099122 out.go:270] Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:15.438137 1099122 out.go:270] Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:15.438142 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:15.438151 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:15.993224 1104534 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0127 12:15:16.001545 1104534 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0127 12:15:16.002581 1104534 api_server.go:141] control plane version: v1.32.1
I0127 12:15:16.002607 1104534 api_server.go:131] duration metric: took 11.836766659s to wait for apiserver health ...
I0127 12:15:16.002615 1104534 system_pods.go:43] waiting for kube-system pods to appear ...
I0127 12:15:16.002637 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:16.002699 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:16.041659 1104534 cri.go:89] found id: "1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:15:16.041685 1104534 cri.go:89] found id: "1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:15:16.041691 1104534 cri.go:89] found id: ""
I0127 12:15:16.041698 1104534 logs.go:282] 2 containers: [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e]
I0127 12:15:16.041761 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.045565 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.049192 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:16.049269 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:16.103251 1104534 cri.go:89] found id: "483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:15:16.103275 1104534 cri.go:89] found id: "e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:15:16.103285 1104534 cri.go:89] found id: ""
I0127 12:15:16.103293 1104534 logs.go:282] 2 containers: [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38]
I0127 12:15:16.103358 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.107119 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.110762 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:16.110835 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:16.150856 1104534 cri.go:89] found id: "509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:15:16.150878 1104534 cri.go:89] found id: "9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:15:16.150882 1104534 cri.go:89] found id: ""
I0127 12:15:16.150889 1104534 logs.go:282] 2 containers: [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af]
I0127 12:15:16.150947 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.155002 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.158414 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:16.158489 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:16.194786 1104534 cri.go:89] found id: "9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:15:16.194807 1104534 cri.go:89] found id: "d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:15:16.194812 1104534 cri.go:89] found id: ""
I0127 12:15:16.194819 1104534 logs.go:282] 2 containers: [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6]
I0127 12:15:16.194902 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.198584 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.202276 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:16.202371 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:16.249547 1104534 cri.go:89] found id: "cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:15:16.249615 1104534 cri.go:89] found id: "b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:15:16.249633 1104534 cri.go:89] found id: ""
I0127 12:15:16.249655 1104534 logs.go:282] 2 containers: [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7]
I0127 12:15:16.249747 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.253544 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.257480 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:16.257554 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:16.313355 1104534 cri.go:89] found id: "32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:15:16.313428 1104534 cri.go:89] found id: "e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:15:16.313447 1104534 cri.go:89] found id: ""
I0127 12:15:16.313468 1104534 logs.go:282] 2 containers: [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487]
I0127 12:15:16.313569 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.317728 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.321603 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:16.321703 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:16.361633 1104534 cri.go:89] found id: "fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:15:16.361711 1104534 cri.go:89] found id: "b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:15:16.361723 1104534 cri.go:89] found id: ""
I0127 12:15:16.361732 1104534 logs.go:282] 2 containers: [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9]
I0127 12:15:16.361791 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.365379 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.368518 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:16.368599 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:16.407080 1104534 cri.go:89] found id: "f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:15:16.407103 1104534 cri.go:89] found id: "14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:15:16.407108 1104534 cri.go:89] found id: ""
I0127 12:15:16.407115 1104534 logs.go:282] 2 containers: [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37]
I0127 12:15:16.407175 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.410860 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.414418 1104534 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:16.414513 1104534 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:16.474028 1104534 cri.go:89] found id: "cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:15:16.474100 1104534 cri.go:89] found id: ""
I0127 12:15:16.474115 1104534 logs.go:282] 1 containers: [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5]
I0127 12:15:16.474178 1104534 ssh_runner.go:195] Run: which crictl
I0127 12:15:16.478146 1104534 logs.go:123] Gathering logs for kube-proxy [b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7] ...
I0127 12:15:16.478174 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b4e9052c1fedabc674c2222dc1c4660a4446f3e68905594dd3752b87bfec8ab7"
I0127 12:15:16.519452 1104534 logs.go:123] Gathering logs for kindnet [b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9] ...
I0127 12:15:16.519482 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b01817eabb909bdf2fca7c99282d81be3e149b9e5e514379ae75fbbe909180a9"
I0127 12:15:16.590945 1104534 logs.go:123] Gathering logs for containerd ...
I0127 12:15:16.590973 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:16.664839 1104534 logs.go:123] Gathering logs for kube-apiserver [1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c] ...
I0127 12:15:16.664875 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1e2e3b1476e69079c1ee9e25f9bb356cc8e52fbb728ebaf0da89cee0bdfbf91c"
I0127 12:15:16.732116 1104534 logs.go:123] Gathering logs for etcd [483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f] ...
I0127 12:15:16.732151 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 483e568c5ee91d6fd7c9f00d6dc9c8fb56f0a60f9a677db9f74189916de6377f"
I0127 12:15:16.792124 1104534 logs.go:123] Gathering logs for kube-scheduler [d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6] ...
I0127 12:15:16.792156 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d3ca08813f7f28675608fdff26b822fa26ae938aa3184f12ed60a38df6a8e9b6"
I0127 12:15:16.857786 1104534 logs.go:123] Gathering logs for kube-proxy [cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388] ...
I0127 12:15:16.857816 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cb5eb19b75d1d15f926d57747591a2cfaa9e12237e4d321718f0a161969c4388"
I0127 12:15:16.904178 1104534 logs.go:123] Gathering logs for kube-controller-manager [32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191] ...
I0127 12:15:16.904205 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 32aa83148612a5a0d3262a4a0c01e05aafb6193b39935ef9b32e34b3731c4191"
I0127 12:15:16.992684 1104534 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:16.992719 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:17.052148 1104534 logs.go:138] Found kubelet problem: Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:15:17.088519 1104534 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:17.088558 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:17.105210 1104534 logs.go:123] Gathering logs for coredns [9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af] ...
I0127 12:15:17.105243 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9cd164c0fe739bf31cb613cada307e3ec8b9a582e74d7893fce085f9c45ce8af"
I0127 12:15:17.144493 1104534 logs.go:123] Gathering logs for storage-provisioner [f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333] ...
I0127 12:15:17.144522 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f6f404f21b109084490d079cfcc8b0371e82a1d33476cb90d3da0ee6ed14f333"
I0127 12:15:17.181902 1104534 logs.go:123] Gathering logs for storage-provisioner [14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37] ...
I0127 12:15:17.181928 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 14d55d7c85202cf7f3b7019c6c77fd30c98972e2a8dc201b5e49061c93dfed37"
I0127 12:15:17.227045 1104534 logs.go:123] Gathering logs for kube-controller-manager [e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487] ...
I0127 12:15:17.227078 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e451ed26c1c38fb8a0a7cc5ea845d7583691c679eea5deb9dbc55465102cf487"
I0127 12:15:17.284499 1104534 logs.go:123] Gathering logs for kindnet [fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6] ...
I0127 12:15:17.284529 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fd7a077525a0fd6934b044f8aa8e1a14646463a0d4a2018526c7fc86f3a518e6"
I0127 12:15:17.329309 1104534 logs.go:123] Gathering logs for kubernetes-dashboard [cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5] ...
I0127 12:15:17.329337 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cbf26bb26b863bdc58d25df8d595193c8e7929be38fe3a0662daa76fe04144d5"
I0127 12:15:17.380574 1104534 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:17.380647 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.32.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:17.497130 1104534 logs.go:123] Gathering logs for kube-apiserver [1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e] ...
I0127 12:15:17.497159 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1f35da231fa433c1460f1cd99f1f2b6dd0431b163618dc1e87a2cf3a793ebd2e"
I0127 12:15:17.560606 1104534 logs.go:123] Gathering logs for etcd [e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38] ...
I0127 12:15:17.560879 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e6b517158b9240c94112d076fb03688bf738388758d1146c807b822e1d06fc38"
I0127 12:15:17.611796 1104534 logs.go:123] Gathering logs for coredns [509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25] ...
I0127 12:15:17.611825 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 509d340d4dacfde08b300a919f3787c9c59261bf94f6ff99cbdfe93341133b25"
I0127 12:15:17.657382 1104534 logs.go:123] Gathering logs for kube-scheduler [9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863] ...
I0127 12:15:17.657409 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9a31e4cb737d1c87d5361ee380073dcb21626c4ac496edc8a2635f77f41c4863"
I0127 12:15:17.694504 1104534 logs.go:123] Gathering logs for container status ...
I0127 12:15:17.694531 1104534 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:17.738587 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:17.738669 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:17.738754 1104534 out.go:270] X Problems detected in kubelet:
W0127 12:15:17.738768 1104534 out.go:270] Jan 27 12:10:57 no-preload-835765 kubelet[659]: I0127 12:10:57.568961 659 status_manager.go:890] "Failed to get status for pod" podUID="d66094c8-5c1a-4aaa-a14c-27954c4c5434" pod="kubernetes-dashboard/kubernetes-dashboard-7779f9b69b-qwgv5" err="pods \"kubernetes-dashboard-7779f9b69b-qwgv5\" is forbidden: User \"system:node:no-preload-835765\" cannot get resource \"pods\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-835765' and this object"
I0127 12:15:17.738794 1104534 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:17.738803 1104534 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:27.747342 1104534 system_pods.go:59] 9 kube-system pods found
I0127 12:15:27.747465 1104534 system_pods.go:61] "coredns-668d6bf9bc-8wfgw" [42249467-e564-459a-ac3b-7c4a7fbf09ed] Running
I0127 12:15:27.747488 1104534 system_pods.go:61] "etcd-no-preload-835765" [475893ea-62c8-4098-b525-7ad0f8c530cb] Running
I0127 12:15:27.747532 1104534 system_pods.go:61] "kindnet-tmbh5" [ffbfdceb-9d13-493a-9644-7f1606fcd89e] Running
I0127 12:15:27.747550 1104534 system_pods.go:61] "kube-apiserver-no-preload-835765" [182ab7ba-7d94-4590-89d4-b0ca3764d7c6] Running
I0127 12:15:27.747570 1104534 system_pods.go:61] "kube-controller-manager-no-preload-835765" [3b430709-a5f7-498f-9d2f-f7ad5f00052f] Running
I0127 12:15:27.747608 1104534 system_pods.go:61] "kube-proxy-6j77q" [0bd6d8c3-2a9d-4367-8763-62b461359b69] Running
I0127 12:15:27.747631 1104534 system_pods.go:61] "kube-scheduler-no-preload-835765" [f608cbd8-acbb-4a70-9aa0-d252403e88c2] Running
I0127 12:15:27.747653 1104534 system_pods.go:61] "metrics-server-f79f97bbb-kzh9d" [1a8a6212-6eaa-4ec1-9b1e-78d01a400f56] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0127 12:15:27.747689 1104534 system_pods.go:61] "storage-provisioner" [6fc8bfd2-49a1-4a6b-a8fe-32a472ce1753] Running
I0127 12:15:27.747738 1104534 system_pods.go:74] duration metric: took 11.745114633s to wait for pod list to return data ...
I0127 12:15:27.747771 1104534 default_sa.go:34] waiting for default service account to be created ...
I0127 12:15:27.750820 1104534 default_sa.go:45] found service account: "default"
I0127 12:15:27.750847 1104534 default_sa.go:55] duration metric: took 3.053497ms for default service account to be created ...
I0127 12:15:27.750856 1104534 system_pods.go:137] waiting for k8s-apps to be running ...
I0127 12:15:27.756921 1104534 system_pods.go:87] 9 kube-system pods found
I0127 12:15:27.759968 1104534 system_pods.go:105] "coredns-668d6bf9bc-8wfgw" [42249467-e564-459a-ac3b-7c4a7fbf09ed] Running
I0127 12:15:27.759993 1104534 system_pods.go:105] "etcd-no-preload-835765" [475893ea-62c8-4098-b525-7ad0f8c530cb] Running
I0127 12:15:27.760000 1104534 system_pods.go:105] "kindnet-tmbh5" [ffbfdceb-9d13-493a-9644-7f1606fcd89e] Running
I0127 12:15:27.760005 1104534 system_pods.go:105] "kube-apiserver-no-preload-835765" [182ab7ba-7d94-4590-89d4-b0ca3764d7c6] Running
I0127 12:15:27.760013 1104534 system_pods.go:105] "kube-controller-manager-no-preload-835765" [3b430709-a5f7-498f-9d2f-f7ad5f00052f] Running
I0127 12:15:27.760018 1104534 system_pods.go:105] "kube-proxy-6j77q" [0bd6d8c3-2a9d-4367-8763-62b461359b69] Running
I0127 12:15:27.760022 1104534 system_pods.go:105] "kube-scheduler-no-preload-835765" [f608cbd8-acbb-4a70-9aa0-d252403e88c2] Running
I0127 12:15:27.760031 1104534 system_pods.go:105] "metrics-server-f79f97bbb-kzh9d" [1a8a6212-6eaa-4ec1-9b1e-78d01a400f56] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0127 12:15:27.760040 1104534 system_pods.go:105] "storage-provisioner" [6fc8bfd2-49a1-4a6b-a8fe-32a472ce1753] Running
I0127 12:15:27.760049 1104534 system_pods.go:147] duration metric: took 9.186131ms to wait for k8s-apps to be running ...
I0127 12:15:27.760059 1104534 system_svc.go:44] waiting for kubelet service to be running ....
I0127 12:15:27.760117 1104534 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0127 12:15:27.771714 1104534 system_svc.go:56] duration metric: took 11.643712ms WaitForService to wait for kubelet
I0127 12:15:27.771741 1104534 kubeadm.go:582] duration metric: took 4m41.455192192s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0127 12:15:27.771762 1104534 node_conditions.go:102] verifying NodePressure condition ...
I0127 12:15:27.775024 1104534 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0127 12:15:27.775061 1104534 node_conditions.go:123] node cpu capacity is 2
I0127 12:15:27.775073 1104534 node_conditions.go:105] duration metric: took 3.305528ms to run NodePressure ...
I0127 12:15:27.775086 1104534 start.go:241] waiting for startup goroutines ...
I0127 12:15:27.775095 1104534 start.go:246] waiting for cluster config update ...
I0127 12:15:27.775106 1104534 start.go:255] writing updated cluster config ...
I0127 12:15:27.775396 1104534 ssh_runner.go:195] Run: rm -f paused
I0127 12:15:27.836313 1104534 start.go:600] kubectl: 1.32.1, cluster: 1.32.1 (minor skew: 0)
I0127 12:15:27.839415 1104534 out.go:177] * Done! kubectl is now configured to use "no-preload-835765" cluster and "default" namespace by default
I0127 12:15:25.440470 1099122 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0127 12:15:25.452605 1099122 api_server.go:72] duration metric: took 5m53.082233775s to wait for apiserver process to appear ...
I0127 12:15:25.452629 1099122 api_server.go:88] waiting for apiserver healthz status ...
I0127 12:15:25.452667 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0127 12:15:25.452724 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0127 12:15:25.494090 1099122 cri.go:89] found id: "709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:25.494115 1099122 cri.go:89] found id: "f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:25.494121 1099122 cri.go:89] found id: ""
I0127 12:15:25.494128 1099122 logs.go:282] 2 containers: [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e]
I0127 12:15:25.494189 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.497645 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.500895 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0127 12:15:25.500968 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0127 12:15:25.542357 1099122 cri.go:89] found id: "8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:25.542442 1099122 cri.go:89] found id: "2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:25.542451 1099122 cri.go:89] found id: ""
I0127 12:15:25.542460 1099122 logs.go:282] 2 containers: [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1]
I0127 12:15:25.542525 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.548254 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.552119 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0127 12:15:25.552193 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0127 12:15:25.627449 1099122 cri.go:89] found id: "66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:25.627471 1099122 cri.go:89] found id: "006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:25.627476 1099122 cri.go:89] found id: ""
I0127 12:15:25.627484 1099122 logs.go:282] 2 containers: [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863]
I0127 12:15:25.627539 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.631955 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.635615 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0127 12:15:25.635695 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0127 12:15:25.686029 1099122 cri.go:89] found id: "15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:25.686052 1099122 cri.go:89] found id: "8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:25.686057 1099122 cri.go:89] found id: ""
I0127 12:15:25.686063 1099122 logs.go:282] 2 containers: [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b]
I0127 12:15:25.686121 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.691005 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.696361 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0127 12:15:25.696439 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0127 12:15:25.735204 1099122 cri.go:89] found id: "244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:25.735228 1099122 cri.go:89] found id: "69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:25.735233 1099122 cri.go:89] found id: ""
I0127 12:15:25.735246 1099122 logs.go:282] 2 containers: [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d]
I0127 12:15:25.735318 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.738739 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.742012 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0127 12:15:25.742080 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0127 12:15:25.783705 1099122 cri.go:89] found id: "35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:25.783728 1099122 cri.go:89] found id: "24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:25.783733 1099122 cri.go:89] found id: ""
I0127 12:15:25.783740 1099122 logs.go:282] 2 containers: [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84]
I0127 12:15:25.783798 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.787402 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.790806 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0127 12:15:25.790883 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0127 12:15:25.830016 1099122 cri.go:89] found id: "8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:25.830038 1099122 cri.go:89] found id: "92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:25.830043 1099122 cri.go:89] found id: ""
I0127 12:15:25.830050 1099122 logs.go:282] 2 containers: [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f]
I0127 12:15:25.830108 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.834070 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.837688 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0127 12:15:25.837767 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0127 12:15:25.880360 1099122 cri.go:89] found id: "1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:25.880381 1099122 cri.go:89] found id: "072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:25.880386 1099122 cri.go:89] found id: ""
I0127 12:15:25.880394 1099122 logs.go:282] 2 containers: [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4]
I0127 12:15:25.880459 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.884176 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.888084 1099122 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0127 12:15:25.888159 1099122 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0127 12:15:25.929160 1099122 cri.go:89] found id: "d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:25.929185 1099122 cri.go:89] found id: ""
I0127 12:15:25.929193 1099122 logs.go:282] 1 containers: [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6]
I0127 12:15:25.929257 1099122 ssh_runner.go:195] Run: which crictl
I0127 12:15:25.934269 1099122 logs.go:123] Gathering logs for kindnet [92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f] ...
I0127 12:15:25.934297 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f"
I0127 12:15:25.978515 1099122 logs.go:123] Gathering logs for kube-scheduler [8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b] ...
I0127 12:15:25.978546 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b"
I0127 12:15:26.023365 1099122 logs.go:123] Gathering logs for kube-proxy [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56] ...
I0127 12:15:26.023397 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56"
I0127 12:15:26.074845 1099122 logs.go:123] Gathering logs for kube-proxy [69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d] ...
I0127 12:15:26.074873 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d"
I0127 12:15:26.126098 1099122 logs.go:123] Gathering logs for container status ...
I0127 12:15:26.126177 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0127 12:15:26.170530 1099122 logs.go:123] Gathering logs for dmesg ...
I0127 12:15:26.170571 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0127 12:15:26.190451 1099122 logs.go:123] Gathering logs for coredns [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab] ...
I0127 12:15:26.190479 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab"
I0127 12:15:26.240411 1099122 logs.go:123] Gathering logs for storage-provisioner [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609] ...
I0127 12:15:26.240439 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609"
I0127 12:15:26.285457 1099122 logs.go:123] Gathering logs for kindnet [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026] ...
I0127 12:15:26.285491 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026"
I0127 12:15:26.332919 1099122 logs.go:123] Gathering logs for containerd ...
I0127 12:15:26.332949 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0127 12:15:26.397666 1099122 logs.go:123] Gathering logs for kube-apiserver [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f] ...
I0127 12:15:26.397707 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f"
I0127 12:15:26.459604 1099122 logs.go:123] Gathering logs for etcd [2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1] ...
I0127 12:15:26.459638 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1"
I0127 12:15:26.511966 1099122 logs.go:123] Gathering logs for kube-scheduler [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47] ...
I0127 12:15:26.512123 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47"
I0127 12:15:26.553498 1099122 logs.go:123] Gathering logs for etcd [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9] ...
I0127 12:15:26.553568 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9"
I0127 12:15:26.619235 1099122 logs.go:123] Gathering logs for coredns [006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863] ...
I0127 12:15:26.619267 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863"
I0127 12:15:26.662492 1099122 logs.go:123] Gathering logs for kube-controller-manager [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d] ...
I0127 12:15:26.662523 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d"
I0127 12:15:26.717270 1099122 logs.go:123] Gathering logs for kube-controller-manager [24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84] ...
I0127 12:15:26.717303 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84"
I0127 12:15:26.789783 1099122 logs.go:123] Gathering logs for storage-provisioner [072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4] ...
I0127 12:15:26.789827 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4"
I0127 12:15:26.841274 1099122 logs.go:123] Gathering logs for kubelet ...
I0127 12:15:26.841302 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0127 12:15:26.899918 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.065326 663 reflector.go:138] object-"kube-system"/"metrics-server-token-827qh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-827qh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900177 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.068245 663 reflector.go:138] object-"kube-system"/"kindnet-token-jxc27": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-jxc27" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900398 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.072508 663 reflector.go:138] object-"kube-system"/"kube-proxy-token-pzcvk": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-pzcvk" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900685 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.092394 663 reflector.go:138] object-"kube-system"/"coredns-token-m2lsh": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-m2lsh" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.900893 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.102861 663 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901112 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.106409 663 reflector.go:138] object-"default"/"default-token-pmbfm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-pmbfm" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901342 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135445 663 reflector.go:138] object-"kube-system"/"storage-provisioner-token-bbnlz": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-bbnlz" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.901547 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:53 old-k8s-version-999803 kubelet[663]: E0127 12:09:53.135737 663 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-999803" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-999803' and this object
W0127 12:15:26.910804 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:55 old-k8s-version-999803 kubelet[663]: E0127 12:09:55.695476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.910996 1099122 logs.go:138] Found kubelet problem: Jan 27 12:09:56 old-k8s-version-999803 kubelet[663]: E0127 12:09:56.587407 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.913762 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:08 old-k8s-version-999803 kubelet[663]: E0127 12:10:08.310539 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.915693 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:18 old-k8s-version-999803 kubelet[663]: E0127 12:10:18.685285 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.916152 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:19 old-k8s-version-999803 kubelet[663]: E0127 12:10:19.697364 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.916671 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:23 old-k8s-version-999803 kubelet[663]: E0127 12:10:23.289410 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.917136 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:27 old-k8s-version-999803 kubelet[663]: E0127 12:10:27.725508 663 pod_workers.go:191] Error syncing pod f73574be-9aec-4a33-ac88-97d900488a22 ("storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(f73574be-9aec-4a33-ac88-97d900488a22)"
W0127 12:15:26.917721 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:29 old-k8s-version-999803 kubelet[663]: E0127 12:10:29.743840 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.920519 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:35 old-k8s-version-999803 kubelet[663]: E0127 12:10:35.298365 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.920848 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:38 old-k8s-version-999803 kubelet[663]: E0127 12:10:38.666640 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.921170 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:46 old-k8s-version-999803 kubelet[663]: E0127 12:10:46.289557 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.921758 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:53 old-k8s-version-999803 kubelet[663]: E0127 12:10:53.813459 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.921942 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:57 old-k8s-version-999803 kubelet[663]: E0127 12:10:57.289244 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.922272 1099122 logs.go:138] Found kubelet problem: Jan 27 12:10:58 old-k8s-version-999803 kubelet[663]: E0127 12:10:58.662563 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.922455 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.289633 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.922880 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:10 old-k8s-version-999803 kubelet[663]: E0127 12:11:10.290664 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.925368 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:21 old-k8s-version-999803 kubelet[663]: E0127 12:11:21.295133 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.925709 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:25 old-k8s-version-999803 kubelet[663]: E0127 12:11:25.289132 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.925894 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:36 old-k8s-version-999803 kubelet[663]: E0127 12:11:36.291476 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.926506 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:40 old-k8s-version-999803 kubelet[663]: E0127 12:11:40.940519 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.926833 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:48 old-k8s-version-999803 kubelet[663]: E0127 12:11:48.662783 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.927017 1099122 logs.go:138] Found kubelet problem: Jan 27 12:11:50 old-k8s-version-999803 kubelet[663]: E0127 12:11:50.289726 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.927348 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.288851 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.927543 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:03 old-k8s-version-999803 kubelet[663]: E0127 12:12:03.289979 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.927729 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:14 old-k8s-version-999803 kubelet[663]: E0127 12:12:14.295027 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.928059 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:18 old-k8s-version-999803 kubelet[663]: E0127 12:12:18.288843 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.928244 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:26 old-k8s-version-999803 kubelet[663]: E0127 12:12:26.289650 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.928570 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:30 old-k8s-version-999803 kubelet[663]: E0127 12:12:30.289329 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.928753 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:37 old-k8s-version-999803 kubelet[663]: E0127 12:12:37.289235 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.929138 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:44 old-k8s-version-999803 kubelet[663]: E0127 12:12:44.288931 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.931588 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:50 old-k8s-version-999803 kubelet[663]: E0127 12:12:50.299810 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0127 12:15:26.931921 1099122 logs.go:138] Found kubelet problem: Jan 27 12:12:57 old-k8s-version-999803 kubelet[663]: E0127 12:12:57.288813 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.932106 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:03 old-k8s-version-999803 kubelet[663]: E0127 12:13:03.289391 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.932704 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:11 old-k8s-version-999803 kubelet[663]: E0127 12:13:11.199586 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.932890 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:14 old-k8s-version-999803 kubelet[663]: E0127 12:13:14.289503 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.933222 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:18 old-k8s-version-999803 kubelet[663]: E0127 12:13:18.662572 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.933406 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:29 old-k8s-version-999803 kubelet[663]: E0127 12:13:29.289301 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.933735 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:31 old-k8s-version-999803 kubelet[663]: E0127 12:13:31.288795 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.933921 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.289460 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.934248 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:42 old-k8s-version-999803 kubelet[663]: E0127 12:13:42.290963 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.934432 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:53 old-k8s-version-999803 kubelet[663]: E0127 12:13:53.289152 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.934759 1099122 logs.go:138] Found kubelet problem: Jan 27 12:13:57 old-k8s-version-999803 kubelet[663]: E0127 12:13:57.288829 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.934942 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:07 old-k8s-version-999803 kubelet[663]: E0127 12:14:07.289177 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.935267 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: E0127 12:14:12.289332 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.935452 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:21 old-k8s-version-999803 kubelet[663]: E0127 12:14:21.289169 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.935784 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: E0127 12:14:26.288817 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.935967 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:32 old-k8s-version-999803 kubelet[663]: E0127 12:14:32.289382 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.936292 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: E0127 12:14:37.288718 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.936477 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.936802 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.936988 1099122 logs.go:138] Found kubelet problem: Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.937320 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.937506 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:26.937836 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:26.938019 1099122 logs.go:138] Found kubelet problem: Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:26.938029 1099122 logs.go:123] Gathering logs for describe nodes ...
I0127 12:15:26.938044 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0127 12:15:27.111830 1099122 logs.go:123] Gathering logs for kube-apiserver [f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e] ...
I0127 12:15:27.111864 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e"
I0127 12:15:27.165850 1099122 logs.go:123] Gathering logs for kubernetes-dashboard [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6] ...
I0127 12:15:27.165886 1099122 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6"
I0127 12:15:27.211532 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:27.211556 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0127 12:15:27.211643 1099122 out.go:270] X Problems detected in kubelet:
W0127 12:15:27.211657 1099122 out.go:270] Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:27.211679 1099122 out.go:270] Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:27.211711 1099122 out.go:270] Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0127 12:15:27.211720 1099122 out.go:270] Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
W0127 12:15:27.211727 1099122 out.go:270] Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0127 12:15:27.211737 1099122 out.go:358] Setting ErrFile to fd 2...
I0127 12:15:27.211744 1099122 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0127 12:15:37.213669 1099122 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0127 12:15:37.226947 1099122 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0127 12:15:37.229972 1099122 out.go:201]
W0127 12:15:37.232459 1099122 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0127 12:15:37.232495 1099122 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0127 12:15:37.232520 1099122 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0127 12:15:37.232529 1099122 out.go:270] *
W0127 12:15:37.233525 1099122 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0127 12:15:37.237108 1099122 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
40a6145bb133e 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 2b138d8b8e124 dashboard-metrics-scraper-8d5bb5db8-j2wzf
1ff19c9b7a633 ba04bb24b9575 4 minutes ago Running storage-provisioner 2 5e923d1d0e532 storage-provisioner
d60a3e1bf1944 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 91c673f7cae7e kubernetes-dashboard-cd95d586-rjt7d
244500be1f9c8 25a5233254979 5 minutes ago Running kube-proxy 1 eb5661c2b8b94 kube-proxy-nt2l9
e089ef56a1bbf 1611cd07b61d5 5 minutes ago Running busybox 1 76fb1a81ce155 busybox
072df961772d8 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 5e923d1d0e532 storage-provisioner
66bd9052021d0 db91994f4ee8f 5 minutes ago Running coredns 1 e0dcf3e01f0f4 coredns-74ff55c5b-8pc5m
8eb127c0b3e7c 2be0bcf609c65 5 minutes ago Running kindnet-cni 1 265153f6608b1 kindnet-wxrcg
709fbfcd9d83c 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 2e9bac101deb8 kube-apiserver-old-k8s-version-999803
8503304f4f6bb 05b738aa1bc63 5 minutes ago Running etcd 1 0e313e6b8d512 etcd-old-k8s-version-999803
35981ab98de2c 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 bc94ad27d5a90 kube-controller-manager-old-k8s-version-999803
15e7801bd3deb e7605f88f17d6 5 minutes ago Running kube-scheduler 1 c9f1411109218 kube-scheduler-old-k8s-version-999803
a83e94f3f97f3 1611cd07b61d5 6 minutes ago Exited busybox 0 367fc2acf537a busybox
006c6bbf3f760 db91994f4ee8f 7 minutes ago Exited coredns 0 ea6e3f9f35fbc coredns-74ff55c5b-8pc5m
92c639516afeb 2be0bcf609c65 8 minutes ago Exited kindnet-cni 0 32ec40ccbb455 kindnet-wxrcg
69ff2212fa934 25a5233254979 8 minutes ago Exited kube-proxy 0 4954b0a0eaa72 kube-proxy-nt2l9
f80be7361c799 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 f03db3b4a9126 kube-apiserver-old-k8s-version-999803
24ee4e7a4be2f 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 7336995587cc5 kube-controller-manager-old-k8s-version-999803
8a26e44a90d82 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 34af66dcd10ec kube-scheduler-old-k8s-version-999803
2fce96f610992 05b738aa1bc63 8 minutes ago Exited etcd 0 856d0c31b309b etcd-old-k8s-version-999803
==> containerd <==
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.403957721Z" level=info msg="received exit event container_id:\"a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268\" id:\"a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268\" pid:2980 exit_status:255 exited_at:{seconds:1737979900 nanos:402001912}"
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.404209629Z" level=info msg="StartContainer for \"a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268\" returns successfully"
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.441723446Z" level=info msg="shim disconnected" id=a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268 namespace=k8s.io
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.441958329Z" level=warning msg="cleaning up after shim disconnected" id=a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268 namespace=k8s.io
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.442121943Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.955691546Z" level=info msg="RemoveContainer for \"926cfd6317dda80a16c8945f6d9903e72f1423f2c68d3f23fdf7212b1118eff8\""
Jan 27 12:11:40 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:11:40.959927325Z" level=info msg="RemoveContainer for \"926cfd6317dda80a16c8945f6d9903e72f1423f2c68d3f23fdf7212b1118eff8\" returns successfully"
Jan 27 12:12:50 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:12:50.292025127Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:12:50 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:12:50.297483850Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Jan 27 12:12:50 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:12:50.299379222Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Jan 27 12:12:50 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:12:50.299429731Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.291932350Z" level=info msg="CreateContainer within sandbox \"2b138d8b8e124838adc3a6301d0d96cc66ffe56272c7314a8159fcc6b934e24a\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.309948135Z" level=info msg="CreateContainer within sandbox \"2b138d8b8e124838adc3a6301d0d96cc66ffe56272c7314a8159fcc6b934e24a\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c\""
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.310742495Z" level=info msg="StartContainer for \"40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c\""
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.384624567Z" level=info msg="StartContainer for \"40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c\" returns successfully"
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.385981736Z" level=info msg="received exit event container_id:\"40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c\" id:\"40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c\" pid:3242 exit_status:255 exited_at:{seconds:1737979990 nanos:385740987}"
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.414485200Z" level=info msg="shim disconnected" id=40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c namespace=k8s.io
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.415427405Z" level=warning msg="cleaning up after shim disconnected" id=40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c namespace=k8s.io
Jan 27 12:13:10 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:10.415525076Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Jan 27 12:13:11 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:11.201533294Z" level=info msg="RemoveContainer for \"a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268\""
Jan 27 12:13:11 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:13:11.209160998Z" level=info msg="RemoveContainer for \"a0228c5a9654992ffae9e34e87744d06e6d850223bd15ca6cd7c2efee6edb268\" returns successfully"
Jan 27 12:15:37 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:15:37.294408088Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:37 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:15:37.307708982Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" host=fake.domain
Jan 27 12:15:37 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:15:37.309800722Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
Jan 27 12:15:37 old-k8s-version-999803 containerd[571]: time="2025-01-27T12:15:37.309822760Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [006c6bbf3f760aff2a8219083d9d31a73f2b60e843733fa6dcc1b9f4cd093863] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:37642 - 41631 "HINFO IN 5616843203759752398.2136082076748601635. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.03179285s
==> coredns [66bd9052021d00f07447d0209c88baaf69ca242ff38e150f4b27b7dc2fb0f8ab] <==
I0127 12:10:26.047581 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2025-01-27 12:09:56.046939567 +0000 UTC m=+0.037396583) (total time: 30.000515314s):
Trace[2019727887]: [30.000515314s] [30.000515314s] END
E0127 12:10:26.048190 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0127 12:10:26.055336 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2025-01-27 12:09:56.047498733 +0000 UTC m=+0.037955749) (total time: 30.007809178s):
Trace[939984059]: [30.007809178s] [30.007809178s] END
E0127 12:10:26.055594 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0127 12:10:26.055636 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2025-01-27 12:09:56.055250325 +0000 UTC m=+0.045707341) (total time: 30.000367642s):
Trace[911902081]: [30.000367642s] [30.000367642s] END
E0127 12:10:26.055771 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:35695 - 6719 "HINFO IN 4182568448720788649.3492565323918288497. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.023927941s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> describe nodes <==
Name: old-k8s-version-999803
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-999803
kubernetes.io/os=linux
minikube.k8s.io/commit=21d19df81a8d69cdaec1a8f1932c09dc00369650
minikube.k8s.io/name=old-k8s-version-999803
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2025_01_27T12_07_19_0700
minikube.k8s.io/version=v1.35.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 27 Jan 2025 12:07:15 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-999803
AcquireTime: <unset>
RenewTime: Mon, 27 Jan 2025 12:15:36 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 27 Jan 2025 12:10:43 +0000 Mon, 27 Jan 2025 12:07:10 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 27 Jan 2025 12:10:43 +0000 Mon, 27 Jan 2025 12:07:10 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 27 Jan 2025 12:10:43 +0000 Mon, 27 Jan 2025 12:07:10 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 27 Jan 2025 12:10:43 +0000 Mon, 27 Jan 2025 12:07:35 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-999803
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022300Ki
pods: 110
System Info:
Machine ID: fd860e1a1bd34c18b89313fc3e5f5650
System UUID: 885e0fc2-a16e-4a09-a4af-833dab2b5e11
Boot ID: 9a2b5a8b-82ce-43cf-92bd-6297263d30a0
Kernel Version: 5.15.0-1075-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.24
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m40s
kube-system coredns-74ff55c5b-8pc5m 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m3s
kube-system etcd-old-k8s-version-999803 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m11s
kube-system kindnet-wxrcg 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m3s
kube-system kube-apiserver-old-k8s-version-999803 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m11s
kube-system kube-controller-manager-old-k8s-version-999803 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m11s
kube-system kube-proxy-nt2l9 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m3s
kube-system kube-scheduler-old-k8s-version-999803 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m11s
kube-system metrics-server-9975d5f86-qzhdd 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m28s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m2s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-j2wzf 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m27s
kubernetes-dashboard kubernetes-dashboard-cd95d586-rjt7d 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m27s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m30s (x5 over 8m30s) kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m30s (x5 over 8m30s) kubelet Node old-k8s-version-999803 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m30s (x4 over 8m30s) kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientPID
Normal Starting 8m11s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m11s kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m11s kubelet Node old-k8s-version-999803 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m11s kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m11s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m3s kubelet Node old-k8s-version-999803 status is now: NodeReady
Normal Starting 8m2s kube-proxy Starting kube-proxy.
Normal Starting 5m58s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m58s (x8 over 5m58s) kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m58s (x8 over 5m58s) kubelet Node old-k8s-version-999803 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m58s (x7 over 5m58s) kubelet Node old-k8s-version-999803 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 5m58s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m41s kube-proxy Starting kube-proxy.
==> dmesg <==
==> etcd [2fce96f610992c05bada065ffdfd6882c70eac2e5f13eedecd13f482a579e4c1] <==
2025-01-27 12:07:09.455172 I | etcdserver/membership: added member ea7e25599daad906 [https://192.168.76.2:2380] to cluster 6f20f2c4b2fb5f8a
raft2025/01/27 12:07:09 INFO: ea7e25599daad906 is starting a new election at term 1
raft2025/01/27 12:07:09 INFO: ea7e25599daad906 became candidate at term 2
raft2025/01/27 12:07:09 INFO: ea7e25599daad906 received MsgVoteResp from ea7e25599daad906 at term 2
raft2025/01/27 12:07:09 INFO: ea7e25599daad906 became leader at term 2
raft2025/01/27 12:07:09 INFO: raft.node: ea7e25599daad906 elected leader ea7e25599daad906 at term 2
2025-01-27 12:07:09.941902 I | etcdserver: published {Name:old-k8s-version-999803 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2025-01-27 12:07:09.942232 I | etcdserver: setting up the initial cluster version to 3.4
2025-01-27 12:07:09.942415 I | embed: ready to serve client requests
2025-01-27 12:07:09.943098 N | etcdserver/membership: set the initial cluster version to 3.4
2025-01-27 12:07:09.943288 I | etcdserver/api: enabled capabilities for version 3.4
2025-01-27 12:07:09.943370 I | embed: ready to serve client requests
2025-01-27 12:07:09.944785 I | embed: serving client requests on 127.0.0.1:2379
2025-01-27 12:07:09.957919 I | embed: serving client requests on 192.168.76.2:2379
2025-01-27 12:07:30.494137 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:07:39.487283 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:07:49.487295 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:07:59.487221 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:09.487475 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:19.487241 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:29.487605 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:39.487329 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:49.487315 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:08:59.487460 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:09:09.488993 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [8503304f4f6bb440cdb9d56690c02a51bbf04721554ec4ea2e4f7beccf7609a9] <==
2025-01-27 12:11:36.778428 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:11:46.778419 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:11:56.778504 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:06.778397 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:16.778653 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:26.778412 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:36.778420 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:46.778609 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:12:56.778511 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:06.778377 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:16.778336 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:26.778531 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:36.778437 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:46.778518 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:13:56.778615 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:06.778481 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:16.778427 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:26.778409 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:36.778348 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:46.779381 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:14:56.778435 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:15:06.778388 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:15:16.778473 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:15:26.778691 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2025-01-27 12:15:36.778436 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
12:15:39 up 4:58, 0 users, load average: 0.50, 1.73, 2.48
Linux old-k8s-version-999803 5.15.0-1075-aws #82~20.04.1-Ubuntu SMP Thu Dec 19 05:23:06 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [8eb127c0b3e7ca2dbdea667c52e0c0902b2083ba316fae162ca871ba65874026] <==
I0127 12:13:36.625139 1 main.go:301] handling current node
I0127 12:13:46.626124 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:13:46.626313 1 main.go:301] handling current node
I0127 12:13:56.618193 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:13:56.618229 1 main.go:301] handling current node
I0127 12:14:06.625171 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:06.625208 1 main.go:301] handling current node
I0127 12:14:16.627196 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:16.627231 1 main.go:301] handling current node
I0127 12:14:26.626426 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:26.626461 1 main.go:301] handling current node
I0127 12:14:36.624088 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:36.624130 1 main.go:301] handling current node
I0127 12:14:46.626697 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:46.626731 1 main.go:301] handling current node
I0127 12:14:56.618178 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:14:56.618213 1 main.go:301] handling current node
I0127 12:15:06.623942 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:15:06.623979 1 main.go:301] handling current node
I0127 12:15:16.627006 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:15:16.627210 1 main.go:301] handling current node
I0127 12:15:26.626449 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:15:26.626482 1 main.go:301] handling current node
I0127 12:15:36.621468 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:15:36.621518 1 main.go:301] handling current node
==> kindnet [92c639516afeb9428e0222d9b5f8645599fd3d251f65494aa754e34cf98cb81f] <==
I0127 12:07:38.418601 1 main.go:182] noMask IPv4 subnets: [10.244.0.0/16]
I0127 12:07:38.730544 1 controller.go:361] Starting controller kube-network-policies
I0127 12:07:38.730567 1 controller.go:365] Waiting for informer caches to sync
I0127 12:07:38.730574 1 shared_informer.go:313] Waiting for caches to sync for kube-network-policies
I0127 12:07:38.931045 1 shared_informer.go:320] Caches are synced for kube-network-policies
I0127 12:07:38.931075 1 metrics.go:61] Registering metrics
I0127 12:07:38.931292 1 controller.go:401] Syncing nftables rules
I0127 12:07:48.737101 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:07:48.737213 1 main.go:301] handling current node
I0127 12:07:58.729887 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:07:58.729923 1 main.go:301] handling current node
I0127 12:08:08.737122 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:08.737180 1 main.go:301] handling current node
I0127 12:08:18.739364 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:18.739428 1 main.go:301] handling current node
I0127 12:08:28.730600 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:28.730642 1 main.go:301] handling current node
I0127 12:08:38.730021 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:38.730074 1 main.go:301] handling current node
I0127 12:08:48.729881 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:48.729913 1 main.go:301] handling current node
I0127 12:08:58.738778 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:08:58.738822 1 main.go:301] handling current node
I0127 12:09:08.736188 1 main.go:297] Handling node with IPs: map[192.168.76.2:{}]
I0127 12:09:08.736296 1 main.go:301] handling current node
==> kube-apiserver [709fbfcd9d83c342cc7508e2e43c597f833305e8b77752082803c329644a7a5f] <==
I0127 12:12:07.272778 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:12:07.272787 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0127 12:12:47.503780 1 client.go:360] parsed scheme: "passthrough"
I0127 12:12:47.503839 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:12:47.503849 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0127 12:12:56.837058 1 handler_proxy.go:102] no RequestInfo found in the context
E0127 12:12:56.837134 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0127 12:12:56.837143 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0127 12:13:30.899942 1 client.go:360] parsed scheme: "passthrough"
I0127 12:13:30.900003 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:13:30.900033 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0127 12:14:13.778646 1 client.go:360] parsed scheme: "passthrough"
I0127 12:14:13.778689 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:14:13.778698 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0127 12:14:52.745323 1 client.go:360] parsed scheme: "passthrough"
I0127 12:14:52.745372 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:14:52.745381 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0127 12:14:54.341557 1 handler_proxy.go:102] no RequestInfo found in the context
E0127 12:14:54.341629 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0127 12:14:54.341638 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0127 12:15:34.985146 1 client.go:360] parsed scheme: "passthrough"
I0127 12:15:34.985189 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:15:34.985198 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [f80be7361c799e67eb8078abf9c00c28743a0b13cb4d68bebcf185a9bbb0f91e] <==
I0127 12:07:16.736324 1 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0127 12:07:16.736635 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0127 12:07:16.748292 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0127 12:07:16.759723 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0127 12:07:16.759909 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0127 12:07:17.252265 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0127 12:07:17.301514 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0127 12:07:17.406392 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.76.2]
I0127 12:07:17.407703 1 controller.go:606] quota admission added evaluator for: endpoints
I0127 12:07:17.411454 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0127 12:07:18.515887 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0127 12:07:19.020712 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0127 12:07:19.076100 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0127 12:07:27.495259 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0127 12:07:35.142215 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0127 12:07:35.271513 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0127 12:07:50.304386 1 client.go:360] parsed scheme: "passthrough"
I0127 12:07:50.304432 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:07:50.304442 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0127 12:08:22.204668 1 client.go:360] parsed scheme: "passthrough"
I0127 12:08:22.204716 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:08:22.204726 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0127 12:09:00.136238 1 client.go:360] parsed scheme: "passthrough"
I0127 12:09:00.136300 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0127 12:09:00.136310 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [24ee4e7a4be2f4c68a515762ad5379c22343d7988e486be1f7e8598a1d0a3d84] <==
I0127 12:07:35.273740 1 range_allocator.go:172] Starting range CIDR allocator
I0127 12:07:35.273744 1 shared_informer.go:240] Waiting for caches to sync for cidrallocator
I0127 12:07:35.273748 1 shared_informer.go:247] Caches are synced for cidrallocator
I0127 12:07:35.276395 1 shared_informer.go:247] Caches are synced for GC
I0127 12:07:35.305138 1 shared_informer.go:247] Caches are synced for stateful set
I0127 12:07:35.305576 1 range_allocator.go:373] Set node old-k8s-version-999803 PodCIDR to [10.244.0.0/24]
I0127 12:07:35.305764 1 shared_informer.go:247] Caches are synced for resource quota
I0127 12:07:35.305938 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-nt2l9"
I0127 12:07:35.313777 1 shared_informer.go:247] Caches are synced for taint
I0127 12:07:35.314067 1 node_lifecycle_controller.go:1429] Initializing eviction metric for zone:
W0127 12:07:35.314622 1 node_lifecycle_controller.go:1044] Missing timestamp for Node old-k8s-version-999803. Assuming now as a timestamp.
I0127 12:07:35.314826 1 node_lifecycle_controller.go:1195] Controller detected that all Nodes are not-Ready. Entering master disruption mode.
I0127 12:07:35.313979 1 shared_informer.go:247] Caches are synced for attach detach
I0127 12:07:35.314235 1 taint_manager.go:187] Starting NoExecuteTaintManager
I0127 12:07:35.314546 1 event.go:291] "Event occurred" object="old-k8s-version-999803" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-999803 event: Registered Node old-k8s-version-999803 in Controller"
I0127 12:07:35.320839 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-wxrcg"
I0127 12:07:35.464162 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0127 12:07:35.723800 1 shared_informer.go:247] Caches are synced for garbage collector
I0127 12:07:35.723828 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0127 12:07:35.764352 1 shared_informer.go:247] Caches are synced for garbage collector
I0127 12:07:36.800036 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0127 12:07:36.846581 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-g2bxq"
I0127 12:07:40.315151 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0127 12:09:09.531270 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0127 12:09:09.874560 1 clusterroleaggregation_controller.go:181] view failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "view": the object has been modified; please apply your changes to the latest version and try again
==> kube-controller-manager [35981ab98de2c587686af3a8152486680cf438d67949177bf86e782a454cdc2d] <==
W0127 12:11:18.674025 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:11:43.422893 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:11:50.324680 1 request.go:655] Throttling request took 1.048372026s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0127 12:11:51.176278 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:12:13.924853 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:12:22.826631 1 request.go:655] Throttling request took 1.048370913s, request: GET:https://192.168.76.2:8443/apis/certificates.k8s.io/v1?timeout=32s
W0127 12:12:23.678086 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:12:44.426942 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:12:55.378486 1 request.go:655] Throttling request took 1.042132309s, request: GET:https://192.168.76.2:8443/apis/admissionregistration.k8s.io/v1?timeout=32s
W0127 12:12:56.179955 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:13:14.929217 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:13:27.830492 1 request.go:655] Throttling request took 1.048363919s, request: GET:https://192.168.76.2:8443/apis/storage.k8s.io/v1?timeout=32s
W0127 12:13:28.682060 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:13:45.431012 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:14:00.332469 1 request.go:655] Throttling request took 1.048414217s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0127 12:14:01.183977 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:14:15.932865 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:14:32.833122 1 request.go:655] Throttling request took 1.048339251s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0127 12:14:33.684592 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:14:46.434822 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:15:05.335320 1 request.go:655] Throttling request took 1.048652726s, request: GET:https://192.168.76.2:8443/apis/networking.k8s.io/v1?timeout=32s
W0127 12:15:06.186502 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0127 12:15:16.936897 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0127 12:15:37.837094 1 request.go:655] Throttling request took 1.048277989s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0127 12:15:38.688684 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [244500be1f9c883cb6f931dd2caa39adf2ce7dd681e6b395ffdd428b7cd73e56] <==
I0127 12:09:57.134580 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0127 12:09:57.134868 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0127 12:09:57.162834 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0127 12:09:57.163194 1 server_others.go:185] Using iptables Proxier.
I0127 12:09:57.164018 1 server.go:650] Version: v1.20.0
I0127 12:09:57.164914 1 config.go:315] Starting service config controller
I0127 12:09:57.164981 1 shared_informer.go:240] Waiting for caches to sync for service config
I0127 12:09:57.165257 1 config.go:224] Starting endpoint slice config controller
I0127 12:09:57.165301 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0127 12:09:57.265311 1 shared_informer.go:247] Caches are synced for service config
I0127 12:09:57.265412 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [69ff2212fa93444c9e96f052d22f3fcf7b914aeed28521b86cc27744fa7cb63d] <==
I0127 12:07:36.490369 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0127 12:07:36.490455 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0127 12:07:36.527640 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0127 12:07:36.527731 1 server_others.go:185] Using iptables Proxier.
I0127 12:07:36.527944 1 server.go:650] Version: v1.20.0
I0127 12:07:36.528448 1 config.go:315] Starting service config controller
I0127 12:07:36.528457 1 shared_informer.go:240] Waiting for caches to sync for service config
I0127 12:07:36.530615 1 config.go:224] Starting endpoint slice config controller
I0127 12:07:36.530627 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0127 12:07:36.628592 1 shared_informer.go:247] Caches are synced for service config
I0127 12:07:36.632473 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [15e7801bd3deb080be178ee5dca3e9054b157598660a5143e5f6027505f59e47] <==
I0127 12:09:46.791692 1 serving.go:331] Generated self-signed cert in-memory
W0127 12:09:52.864812 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0127 12:09:52.867584 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0127 12:09:52.867792 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0127 12:09:52.867888 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0127 12:09:53.330453 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0127 12:09:53.330532 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0127 12:09:53.330538 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0127 12:09:53.330551 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0127 12:09:53.471075 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [8a26e44a90d82cc1704ea53da393d1619a1aa80921953105454f66345703881b] <==
W0127 12:07:15.878573 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0127 12:07:15.878687 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0127 12:07:15.878716 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0127 12:07:15.878756 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0127 12:07:15.945998 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0127 12:07:15.955449 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0127 12:07:15.955732 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0127 12:07:15.955831 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
E0127 12:07:15.981813 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0127 12:07:15.989495 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0127 12:07:15.989850 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0127 12:07:15.990070 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0127 12:07:15.990281 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0127 12:07:15.990478 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0127 12:07:15.990672 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0127 12:07:15.990923 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0127 12:07:15.991099 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0127 12:07:15.991285 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0127 12:07:15.991455 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0127 12:07:15.991641 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0127 12:07:16.841992 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0127 12:07:16.971713 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0127 12:07:17.034727 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0127 12:07:17.086926 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
I0127 12:07:17.556037 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Jan 27 12:14:07 old-k8s-version-999803 kubelet[663]: E0127 12:14:07.289177 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: I0127 12:14:12.288480 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:14:12 old-k8s-version-999803 kubelet[663]: E0127 12:14:12.289332 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:14:21 old-k8s-version-999803 kubelet[663]: E0127 12:14:21.289169 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: I0127 12:14:26.288415 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:14:26 old-k8s-version-999803 kubelet[663]: E0127 12:14:26.288817 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:14:32 old-k8s-version-999803 kubelet[663]: E0127 12:14:32.289382 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: I0127 12:14:37.288373 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:14:37 old-k8s-version-999803 kubelet[663]: E0127 12:14:37.288718 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:14:47 old-k8s-version-999803 kubelet[663]: E0127 12:14:47.289211 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: I0127 12:14:50.288800 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:14:50 old-k8s-version-999803 kubelet[663]: E0127 12:14:50.289608 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:14:58 old-k8s-version-999803 kubelet[663]: E0127 12:14:58.289567 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: I0127 12:15:01.288522 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:15:01 old-k8s-version-999803 kubelet[663]: E0127 12:15:01.288887 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:12 old-k8s-version-999803 kubelet[663]: E0127 12:15:12.289984 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: I0127 12:15:16.289291 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:15:16 old-k8s-version-999803 kubelet[663]: E0127 12:15:16.289567 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:24 old-k8s-version-999803 kubelet[663]: E0127 12:15:24.289280 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Jan 27 12:15:31 old-k8s-version-999803 kubelet[663]: I0127 12:15:31.288441 663 scope.go:95] [topologymanager] RemoveContainer - Container ID: 40a6145bb133e4fd91dabd154367b0d0e4a5c11fbad40fb3fb220613822fb05c
Jan 27 12:15:31 old-k8s-version-999803 kubelet[663]: E0127 12:15:31.288809 663 pod_workers.go:191] Error syncing pod 899b8438-3079-49ad-86e7-4d860b77226a ("dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-j2wzf_kubernetes-dashboard(899b8438-3079-49ad-86e7-4d860b77226a)"
Jan 27 12:15:37 old-k8s-version-999803 kubelet[663]: E0127 12:15:37.310111 663 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Jan 27 12:15:37 old-k8s-version-999803 kubelet[663]: E0127 12:15:37.310172 663 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Jan 27 12:15:37 old-k8s-version-999803 kubelet[663]: E0127 12:15:37.310670 663 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-827qh,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2
f-4782-4243-88b6-16e0d2b7175f): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
Jan 27 12:15:37 old-k8s-version-999803 kubelet[663]: E0127 12:15:37.310716 663 pod_workers.go:191] Error syncing pod 01eb7c2f-4782-4243-88b6-16e0d2b7175f ("metrics-server-9975d5f86-qzhdd_kube-system(01eb7c2f-4782-4243-88b6-16e0d2b7175f)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
==> kubernetes-dashboard [d60a3e1bf1944552103fad77eec238f749ba5acad1a8659d1c64c14613b93cc6] <==
2025/01/27 12:10:21 Using namespace: kubernetes-dashboard
2025/01/27 12:10:21 Using in-cluster config to connect to apiserver
2025/01/27 12:10:21 Using secret token for csrf signing
2025/01/27 12:10:21 Initializing csrf token from kubernetes-dashboard-csrf secret
2025/01/27 12:10:21 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2025/01/27 12:10:21 Successful initial request to the apiserver, version: v1.20.0
2025/01/27 12:10:21 Generating JWE encryption key
2025/01/27 12:10:21 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2025/01/27 12:10:21 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2025/01/27 12:10:21 Initializing JWE encryption key from synchronized object
2025/01/27 12:10:21 Creating in-cluster Sidecar client
2025/01/27 12:10:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:10:21 Serving insecurely on HTTP port: 9090
2025/01/27 12:10:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:11:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:11:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:12:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:12:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:13:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:13:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:14:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:14:51 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:15:21 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2025/01/27 12:10:21 Starting overwatch
==> storage-provisioner [072df961772d8282ba54c52a0302c98c79f57d6e3f2dd679b1852199b2455bb4] <==
I0127 12:09:56.184951 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0127 12:10:26.188438 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [1ff19c9b7a63387c9efa0456868dae9c8e181163e1563d382675a4945e811609] <==
I0127 12:10:43.444818 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0127 12:10:43.470074 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0127 12:10:43.470359 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0127 12:11:00.944474 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0127 12:11:00.947513 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-999803_d448eea1-8bf9-4e1f-b48b-510603e45bd1!
I0127 12:11:00.947729 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"3144bf25-28ec-44c1-96c6-b5f616047066", APIVersion:"v1", ResourceVersion:"849", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-999803_d448eea1-8bf9-4e1f-b48b-510603e45bd1 became leader
I0127 12:11:01.048544 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-999803_d448eea1-8bf9-4e1f-b48b-510603e45bd1!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-999803 -n old-k8s-version-999803
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-999803 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-qzhdd
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-999803 describe pod metrics-server-9975d5f86-qzhdd
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-999803 describe pod metrics-server-9975d5f86-qzhdd: exit status 1 (117.280602ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-qzhdd" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-999803 describe pod metrics-server-9975d5f86-qzhdd: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (377.68s)