=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-605045 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-605045 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m15.041367248s)
-- stdout --
* [old-k8s-version-605045] minikube v1.34.0 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19688
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19688-2897765/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19688-2897765/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-605045" primary control-plane node in "old-k8s-version-605045" cluster
* Pulling base image v0.0.45-1726784731-19672 ...
* Restarting existing docker container for "old-k8s-version-605045" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-605045 addons enable metrics-server
* Enabled addons: default-storageclass, storage-provisioner, metrics-server, dashboard
-- /stdout --
** stderr **
I0923 12:44:01.087349 3102485 out.go:345] Setting OutFile to fd 1 ...
I0923 12:44:01.087699 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:44:01.087710 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:44:01.087716 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:44:01.087986 3102485 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19688-2897765/.minikube/bin
I0923 12:44:01.088377 3102485 out.go:352] Setting JSON to false
I0923 12:44:01.089351 3102485 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":159988,"bootTime":1726935453,"procs":200,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0923 12:44:01.089415 3102485 start.go:139] virtualization:
I0923 12:44:01.091829 3102485 out.go:177] * [old-k8s-version-605045] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0923 12:44:01.093926 3102485 out.go:177] - MINIKUBE_LOCATION=19688
I0923 12:44:01.093992 3102485 notify.go:220] Checking for updates...
I0923 12:44:01.100268 3102485 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0923 12:44:01.102146 3102485 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:44:01.103646 3102485 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19688-2897765/.minikube
I0923 12:44:01.105295 3102485 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0923 12:44:01.107150 3102485 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0923 12:44:01.109359 3102485 config.go:182] Loaded profile config "old-k8s-version-605045": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0923 12:44:01.112454 3102485 out.go:177] * Kubernetes 1.31.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.1
I0923 12:44:01.114237 3102485 driver.go:394] Setting default libvirt URI to qemu:///system
I0923 12:44:01.156878 3102485 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0923 12:44:01.157019 3102485 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0923 12:44:01.241163 3102485 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:58 OomKillDisable:true NGoroutines:67 SystemTime:2024-09-23 12:44:01.229465942 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0923 12:44:01.241281 3102485 docker.go:318] overlay module found
I0923 12:44:01.243354 3102485 out.go:177] * Using the docker driver based on existing profile
I0923 12:44:01.245240 3102485 start.go:297] selected driver: docker
I0923 12:44:01.245257 3102485 start.go:901] validating driver "docker" against &{Name:old-k8s-version-605045 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-605045 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:44:01.245384 3102485 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0923 12:44:01.246043 3102485 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0923 12:44:01.318843 3102485 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:58 OomKillDisable:true NGoroutines:67 SystemTime:2024-09-23 12:44:01.306975603 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0923 12:44:01.319270 3102485 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0923 12:44:01.319316 3102485 cni.go:84] Creating CNI manager for ""
I0923 12:44:01.319372 3102485 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0923 12:44:01.319431 3102485 start.go:340] cluster config:
{Name:old-k8s-version-605045 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-605045 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:44:01.321993 3102485 out.go:177] * Starting "old-k8s-version-605045" primary control-plane node in "old-k8s-version-605045" cluster
I0923 12:44:01.324405 3102485 cache.go:121] Beginning downloading kic base image for docker with containerd
I0923 12:44:01.326397 3102485 out.go:177] * Pulling base image v0.0.45-1726784731-19672 ...
I0923 12:44:01.328348 3102485 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0923 12:44:01.328422 3102485 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0923 12:44:01.328435 3102485 cache.go:56] Caching tarball of preloaded images
I0923 12:44:01.328539 3102485 preload.go:172] Found /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0923 12:44:01.328554 3102485 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0923 12:44:01.328678 3102485 profile.go:143] Saving config to /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/config.json ...
I0923 12:44:01.328933 3102485 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed in local docker daemon
I0923 12:44:01.352146 3102485 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed in local docker daemon, skipping pull
I0923 12:44:01.352170 3102485 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed exists in daemon, skipping load
I0923 12:44:01.352189 3102485 cache.go:194] Successfully downloaded all kic artifacts
I0923 12:44:01.352213 3102485 start.go:360] acquireMachinesLock for old-k8s-version-605045: {Name:mk225242fd48ee1df3a44019e0bef95cc5d4a595 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:44:01.352280 3102485 start.go:364] duration metric: took 45.112µs to acquireMachinesLock for "old-k8s-version-605045"
I0923 12:44:01.352307 3102485 start.go:96] Skipping create...Using existing machine configuration
I0923 12:44:01.352326 3102485 fix.go:54] fixHost starting:
I0923 12:44:01.352602 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:01.373924 3102485 fix.go:112] recreateIfNeeded on old-k8s-version-605045: state=Stopped err=<nil>
W0923 12:44:01.373960 3102485 fix.go:138] unexpected machine state, will restart: <nil>
I0923 12:44:01.379252 3102485 out.go:177] * Restarting existing docker container for "old-k8s-version-605045" ...
I0923 12:44:01.381295 3102485 cli_runner.go:164] Run: docker start old-k8s-version-605045
I0923 12:44:01.754970 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:01.784918 3102485 kic.go:430] container "old-k8s-version-605045" state is running.
I0923 12:44:01.785321 3102485 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-605045
I0923 12:44:01.811580 3102485 profile.go:143] Saving config to /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/config.json ...
I0923 12:44:01.811816 3102485 machine.go:93] provisionDockerMachine start ...
I0923 12:44:01.811880 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:01.839513 3102485 main.go:141] libmachine: Using SSH client type: native
I0923 12:44:01.839777 3102485 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42087 <nil> <nil>}
I0923 12:44:01.839788 3102485 main.go:141] libmachine: About to run SSH command:
hostname
I0923 12:44:01.840631 3102485 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:33906->127.0.0.1:42087: read: connection reset by peer
I0923 12:44:04.998832 3102485 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-605045
I0923 12:44:04.998895 3102485 ubuntu.go:169] provisioning hostname "old-k8s-version-605045"
I0923 12:44:04.998998 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:05.044787 3102485 main.go:141] libmachine: Using SSH client type: native
I0923 12:44:05.045045 3102485 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42087 <nil> <nil>}
I0923 12:44:05.045057 3102485 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-605045 && echo "old-k8s-version-605045" | sudo tee /etc/hostname
I0923 12:44:05.226868 3102485 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-605045
I0923 12:44:05.227009 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:05.269026 3102485 main.go:141] libmachine: Using SSH client type: native
I0923 12:44:05.269363 3102485 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42087 <nil> <nil>}
I0923 12:44:05.269391 3102485 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-605045' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-605045/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-605045' | sudo tee -a /etc/hosts;
fi
fi
I0923 12:44:05.432136 3102485 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0923 12:44:05.432209 3102485 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19688-2897765/.minikube CaCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19688-2897765/.minikube}
I0923 12:44:05.432257 3102485 ubuntu.go:177] setting up certificates
I0923 12:44:05.432298 3102485 provision.go:84] configureAuth start
I0923 12:44:05.432382 3102485 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-605045
I0923 12:44:05.464799 3102485 provision.go:143] copyHostCerts
I0923 12:44:05.464877 3102485 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem, removing ...
I0923 12:44:05.464896 3102485 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem
I0923 12:44:05.464976 3102485 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem (1675 bytes)
I0923 12:44:05.465088 3102485 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem, removing ...
I0923 12:44:05.465093 3102485 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem
I0923 12:44:05.465122 3102485 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem (1078 bytes)
I0923 12:44:05.465180 3102485 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem, removing ...
I0923 12:44:05.465190 3102485 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem
I0923 12:44:05.465215 3102485 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem (1123 bytes)
I0923 12:44:05.465269 3102485 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-605045 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-605045]
I0923 12:44:05.789801 3102485 provision.go:177] copyRemoteCerts
I0923 12:44:05.789883 3102485 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0923 12:44:05.789929 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:05.809183 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:05.913745 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0923 12:44:05.955970 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0923 12:44:05.984964 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0923 12:44:06.032578 3102485 provision.go:87] duration metric: took 600.236628ms to configureAuth
I0923 12:44:06.032611 3102485 ubuntu.go:193] setting minikube options for container-runtime
I0923 12:44:06.032992 3102485 config.go:182] Loaded profile config "old-k8s-version-605045": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0923 12:44:06.033028 3102485 machine.go:96] duration metric: took 4.221201798s to provisionDockerMachine
I0923 12:44:06.033045 3102485 start.go:293] postStartSetup for "old-k8s-version-605045" (driver="docker")
I0923 12:44:06.033060 3102485 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0923 12:44:06.033579 3102485 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0923 12:44:06.033672 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:06.075547 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:06.184883 3102485 ssh_runner.go:195] Run: cat /etc/os-release
I0923 12:44:06.189821 3102485 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0923 12:44:06.189877 3102485 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0923 12:44:06.189896 3102485 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0923 12:44:06.189904 3102485 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0923 12:44:06.189931 3102485 filesync.go:126] Scanning /home/jenkins/minikube-integration/19688-2897765/.minikube/addons for local assets ...
I0923 12:44:06.190021 3102485 filesync.go:126] Scanning /home/jenkins/minikube-integration/19688-2897765/.minikube/files for local assets ...
I0923 12:44:06.190131 3102485 filesync.go:149] local asset: /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem -> 29031582.pem in /etc/ssl/certs
I0923 12:44:06.190279 3102485 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0923 12:44:06.205735 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem --> /etc/ssl/certs/29031582.pem (1708 bytes)
I0923 12:44:06.247580 3102485 start.go:296] duration metric: took 214.514442ms for postStartSetup
I0923 12:44:06.247711 3102485 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0923 12:44:06.247763 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:06.274620 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:06.369740 3102485 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0923 12:44:06.375792 3102485 fix.go:56] duration metric: took 5.023467904s for fixHost
I0923 12:44:06.375815 3102485 start.go:83] releasing machines lock for "old-k8s-version-605045", held for 5.023521541s
I0923 12:44:06.375897 3102485 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-605045
I0923 12:44:06.402166 3102485 ssh_runner.go:195] Run: cat /version.json
I0923 12:44:06.402216 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:06.402463 3102485 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0923 12:44:06.402513 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:06.439473 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:06.440944 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:06.547188 3102485 ssh_runner.go:195] Run: systemctl --version
I0923 12:44:06.722140 3102485 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0923 12:44:06.727305 3102485 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0923 12:44:06.754486 3102485 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0923 12:44:06.754568 3102485 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0923 12:44:06.767198 3102485 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0923 12:44:06.767225 3102485 start.go:495] detecting cgroup driver to use...
I0923 12:44:06.767259 3102485 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0923 12:44:06.767379 3102485 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0923 12:44:06.790648 3102485 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0923 12:44:06.805855 3102485 docker.go:217] disabling cri-docker service (if available) ...
I0923 12:44:06.806064 3102485 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0923 12:44:06.828015 3102485 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0923 12:44:06.843604 3102485 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0923 12:44:06.994695 3102485 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0923 12:44:07.147856 3102485 docker.go:233] disabling docker service ...
I0923 12:44:07.148010 3102485 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0923 12:44:07.166247 3102485 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0923 12:44:07.183926 3102485 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0923 12:44:07.332055 3102485 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0923 12:44:07.498946 3102485 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0923 12:44:07.517892 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0923 12:44:07.537379 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0923 12:44:07.550213 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0923 12:44:07.562913 3102485 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0923 12:44:07.563037 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0923 12:44:07.575087 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0923 12:44:07.586291 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0923 12:44:07.598274 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0923 12:44:07.609446 3102485 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0923 12:44:07.620312 3102485 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0923 12:44:07.631016 3102485 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0923 12:44:07.641448 3102485 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0923 12:44:07.656821 3102485 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:44:07.807992 3102485 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0923 12:44:08.081398 3102485 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0923 12:44:08.081495 3102485 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0923 12:44:08.086003 3102485 start.go:563] Will wait 60s for crictl version
I0923 12:44:08.086083 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:44:08.090933 3102485 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0923 12:44:08.141930 3102485 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0923 12:44:08.142003 3102485 ssh_runner.go:195] Run: containerd --version
I0923 12:44:08.166053 3102485 ssh_runner.go:195] Run: containerd --version
I0923 12:44:08.192932 3102485 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.22 ...
I0923 12:44:08.195048 3102485 cli_runner.go:164] Run: docker network inspect old-k8s-version-605045 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0923 12:44:08.221268 3102485 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0923 12:44:08.239556 3102485 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0923 12:44:08.256709 3102485 kubeadm.go:883] updating cluster {Name:old-k8s-version-605045 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-605045 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0923 12:44:08.256846 3102485 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0923 12:44:08.256909 3102485 ssh_runner.go:195] Run: sudo crictl images --output json
I0923 12:44:08.333304 3102485 containerd.go:627] all images are preloaded for containerd runtime.
I0923 12:44:08.333382 3102485 containerd.go:534] Images already preloaded, skipping extraction
I0923 12:44:08.333477 3102485 ssh_runner.go:195] Run: sudo crictl images --output json
I0923 12:44:08.402317 3102485 containerd.go:627] all images are preloaded for containerd runtime.
I0923 12:44:08.402343 3102485 cache_images.go:84] Images are preloaded, skipping loading
I0923 12:44:08.402352 3102485 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.20.0 containerd true true} ...
I0923 12:44:08.402470 3102485 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-605045 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-605045 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0923 12:44:08.402565 3102485 ssh_runner.go:195] Run: sudo crictl info
I0923 12:44:08.483815 3102485 cni.go:84] Creating CNI manager for ""
I0923 12:44:08.483844 3102485 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0923 12:44:08.483854 3102485 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0923 12:44:08.483874 3102485 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-605045 NodeName:old-k8s-version-605045 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0923 12:44:08.484003 3102485 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-605045"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0923 12:44:08.484073 3102485 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0923 12:44:08.496746 3102485 binaries.go:44] Found k8s binaries, skipping transfer
I0923 12:44:08.496823 3102485 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0923 12:44:08.510528 3102485 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0923 12:44:08.532228 3102485 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0923 12:44:08.553987 3102485 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0923 12:44:08.580681 3102485 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0923 12:44:08.586052 3102485 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0923 12:44:08.607407 3102485 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:44:08.743270 3102485 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0923 12:44:08.762023 3102485 certs.go:68] Setting up /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045 for IP: 192.168.85.2
I0923 12:44:08.762047 3102485 certs.go:194] generating shared ca certs ...
I0923 12:44:08.762064 3102485 certs.go:226] acquiring lock for ca certs: {Name:mk3307686e47e832a4d12b60b03ff3c8ff918f47 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:44:08.762200 3102485 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.key
I0923 12:44:08.762263 3102485 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.key
I0923 12:44:08.762277 3102485 certs.go:256] generating profile certs ...
I0923 12:44:08.762362 3102485 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/client.key
I0923 12:44:08.762435 3102485 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/apiserver.key.d78aadc0
I0923 12:44:08.762480 3102485 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/proxy-client.key
I0923 12:44:08.762593 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158.pem (1338 bytes)
W0923 12:44:08.762626 3102485 certs.go:480] ignoring /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158_empty.pem, impossibly tiny 0 bytes
I0923 12:44:08.762639 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem (1679 bytes)
I0923 12:44:08.762663 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem (1078 bytes)
I0923 12:44:08.762689 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem (1123 bytes)
I0923 12:44:08.762715 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem (1675 bytes)
I0923 12:44:08.762765 3102485 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem (1708 bytes)
I0923 12:44:08.763409 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0923 12:44:08.822683 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0923 12:44:08.889472 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0923 12:44:08.938974 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0923 12:44:08.998882 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0923 12:44:09.080621 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0923 12:44:09.131867 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0923 12:44:09.183633 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/old-k8s-version-605045/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0923 12:44:09.232095 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem --> /usr/share/ca-certificates/29031582.pem (1708 bytes)
I0923 12:44:09.278813 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0923 12:44:09.319821 3102485 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158.pem --> /usr/share/ca-certificates/2903158.pem (1338 bytes)
I0923 12:44:09.360407 3102485 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0923 12:44:09.389676 3102485 ssh_runner.go:195] Run: openssl version
I0923 12:44:09.404107 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0923 12:44:09.420854 3102485 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0923 12:44:09.425074 3102485 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 23 11:36 /usr/share/ca-certificates/minikubeCA.pem
I0923 12:44:09.425144 3102485 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0923 12:44:09.434803 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0923 12:44:09.448634 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2903158.pem && ln -fs /usr/share/ca-certificates/2903158.pem /etc/ssl/certs/2903158.pem"
I0923 12:44:09.468461 3102485 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2903158.pem
I0923 12:44:09.472753 3102485 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 23 12:06 /usr/share/ca-certificates/2903158.pem
I0923 12:44:09.472823 3102485 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2903158.pem
I0923 12:44:09.484169 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2903158.pem /etc/ssl/certs/51391683.0"
I0923 12:44:09.495584 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/29031582.pem && ln -fs /usr/share/ca-certificates/29031582.pem /etc/ssl/certs/29031582.pem"
I0923 12:44:09.510509 3102485 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/29031582.pem
I0923 12:44:09.514376 3102485 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 23 12:06 /usr/share/ca-certificates/29031582.pem
I0923 12:44:09.514453 3102485 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/29031582.pem
I0923 12:44:09.526029 3102485 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/29031582.pem /etc/ssl/certs/3ec20f2e.0"
I0923 12:44:09.538170 3102485 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0923 12:44:09.543948 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0923 12:44:09.553748 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0923 12:44:09.561511 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0923 12:44:09.575584 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0923 12:44:09.588054 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0923 12:44:09.600075 3102485 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0923 12:44:09.612036 3102485 kubeadm.go:392] StartCluster: {Name:old-k8s-version-605045 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-605045 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:44:09.612135 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0923 12:44:09.612197 3102485 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0923 12:44:09.684946 3102485 cri.go:89] found id: "8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:44:09.684970 3102485 cri.go:89] found id: "7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:44:09.684976 3102485 cri.go:89] found id: "5b397ef35967b921c43f5c7afe37dbd498e745c39e94adb4564df51ed332ea5d"
I0923 12:44:09.684980 3102485 cri.go:89] found id: "c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:44:09.684984 3102485 cri.go:89] found id: "7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:44:09.684987 3102485 cri.go:89] found id: "1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:44:09.684992 3102485 cri.go:89] found id: "2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:44:09.684996 3102485 cri.go:89] found id: "2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:44:09.684999 3102485 cri.go:89] found id: ""
I0923 12:44:09.685051 3102485 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0923 12:44:09.697747 3102485 cri.go:116] JSON = null
W0923 12:44:09.697801 3102485 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0923 12:44:09.697871 3102485 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0923 12:44:09.715348 3102485 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0923 12:44:09.715369 3102485 kubeadm.go:593] restartPrimaryControlPlane start ...
I0923 12:44:09.715422 3102485 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0923 12:44:09.730120 3102485 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0923 12:44:09.730583 3102485 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-605045" does not appear in /home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:44:09.730695 3102485 kubeconfig.go:62] /home/jenkins/minikube-integration/19688-2897765/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-605045" cluster setting kubeconfig missing "old-k8s-version-605045" context setting]
I0923 12:44:09.730959 3102485 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19688-2897765/kubeconfig: {Name:mkc814324ebd7e6787446f1c0db099ab6daa7ff3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:44:09.732282 3102485 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0923 12:44:09.741914 3102485 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0923 12:44:09.741949 3102485 kubeadm.go:597] duration metric: took 26.572933ms to restartPrimaryControlPlane
I0923 12:44:09.741960 3102485 kubeadm.go:394] duration metric: took 129.935628ms to StartCluster
I0923 12:44:09.741975 3102485 settings.go:142] acquiring lock: {Name:mk4415211fc0f47c243959f36c7d2f9eeca37653 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:44:09.742040 3102485 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:44:09.742651 3102485 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19688-2897765/kubeconfig: {Name:mkc814324ebd7e6787446f1c0db099ab6daa7ff3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:44:09.742853 3102485 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0923 12:44:09.743201 3102485 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0923 12:44:09.743271 3102485 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-605045"
I0923 12:44:09.743408 3102485 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-605045"
W0923 12:44:09.743414 3102485 addons.go:243] addon storage-provisioner should already be in state true
I0923 12:44:09.743452 3102485 host.go:66] Checking if "old-k8s-version-605045" exists ...
I0923 12:44:09.743937 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:09.744400 3102485 config.go:182] Loaded profile config "old-k8s-version-605045": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0923 12:44:09.744484 3102485 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-605045"
I0923 12:44:09.744520 3102485 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-605045"
I0923 12:44:09.744824 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:09.745346 3102485 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-605045"
I0923 12:44:09.745371 3102485 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-605045"
W0923 12:44:09.745380 3102485 addons.go:243] addon metrics-server should already be in state true
I0923 12:44:09.745405 3102485 host.go:66] Checking if "old-k8s-version-605045" exists ...
I0923 12:44:09.745831 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:09.748403 3102485 addons.go:69] Setting dashboard=true in profile "old-k8s-version-605045"
I0923 12:44:09.748528 3102485 addons.go:234] Setting addon dashboard=true in "old-k8s-version-605045"
W0923 12:44:09.748545 3102485 addons.go:243] addon dashboard should already be in state true
I0923 12:44:09.748576 3102485 host.go:66] Checking if "old-k8s-version-605045" exists ...
I0923 12:44:09.748758 3102485 out.go:177] * Verifying Kubernetes components...
I0923 12:44:09.749991 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:09.751176 3102485 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:44:09.815582 3102485 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0923 12:44:09.817801 3102485 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:44:09.817823 3102485 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0923 12:44:09.817900 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:09.831468 3102485 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0923 12:44:09.833471 3102485 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0923 12:44:09.834866 3102485 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-605045"
W0923 12:44:09.834894 3102485 addons.go:243] addon default-storageclass should already be in state true
I0923 12:44:09.834918 3102485 host.go:66] Checking if "old-k8s-version-605045" exists ...
I0923 12:44:09.835373 3102485 cli_runner.go:164] Run: docker container inspect old-k8s-version-605045 --format={{.State.Status}}
I0923 12:44:09.835598 3102485 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0923 12:44:09.836404 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0923 12:44:09.836424 3102485 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0923 12:44:09.836501 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:09.837654 3102485 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0923 12:44:09.837684 3102485 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0923 12:44:09.837744 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:09.879677 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:09.916761 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:09.918728 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:09.923592 3102485 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0923 12:44:09.923617 3102485 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0923 12:44:09.923681 3102485 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-605045
I0923 12:44:09.954645 3102485 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42087 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/old-k8s-version-605045/id_rsa Username:docker}
I0923 12:44:10.014285 3102485 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0923 12:44:10.072019 3102485 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-605045" to be "Ready" ...
I0923 12:44:10.158398 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:44:10.170945 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0923 12:44:10.171019 3102485 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0923 12:44:10.239681 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0923 12:44:10.239755 3102485 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0923 12:44:10.253695 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0923 12:44:10.255390 3102485 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0923 12:44:10.255451 3102485 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0923 12:44:10.329073 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0923 12:44:10.329226 3102485 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0923 12:44:10.355159 3102485 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0923 12:44:10.355223 3102485 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0923 12:44:10.386702 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0923 12:44:10.386726 3102485 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0923 12:44:10.430294 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0923 12:44:10.430321 3102485 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0923 12:44:10.487292 3102485 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0923 12:44:10.487365 3102485 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0923 12:44:10.544152 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0923 12:44:10.544224 3102485 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0923 12:44:10.560276 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.560310 3102485 retry.go:31] will retry after 191.983356ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.577535 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0923 12:44:10.617852 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0923 12:44:10.617928 3102485 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
W0923 12:44:10.636171 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.636254 3102485 retry.go:31] will retry after 339.596744ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.667813 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0923 12:44:10.667888 3102485 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0923 12:44:10.718979 3102485 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:10.719053 3102485 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0923 12:44:10.750414 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:10.752414 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0923 12:44:10.788539 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.788573 3102485 retry.go:31] will retry after 154.8038ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.944405 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:10.946450 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.946483 3102485 retry.go:31] will retry after 226.876954ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:10.946538 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.946551 3102485 retry.go:31] will retry after 497.821315ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:10.976860 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:11.114586 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.114622 3102485 retry.go:31] will retry after 490.507338ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:11.149821 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.149856 3102485 retry.go:31] will retry after 318.133375ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.174142 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0923 12:44:11.286975 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.287072 3102485 retry.go:31] will retry after 192.96777ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.445418 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:44:11.468751 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0923 12:44:11.480241 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:11.606002 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:11.692898 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.692975 3102485 retry.go:31] will retry after 589.236623ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:11.773393 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.773472 3102485 retry.go:31] will retry after 651.919996ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:11.807590 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.807689 3102485 retry.go:31] will retry after 737.003205ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:11.814918 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:11.814956 3102485 retry.go:31] will retry after 783.761031ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.072632 3102485 node_ready.go:53] error getting node "old-k8s-version-605045": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-605045": dial tcp 192.168.85.2:8443: connect: connection refused
I0923 12:44:12.283049 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0923 12:44:12.389934 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.389974 3102485 retry.go:31] will retry after 1.165294876s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.426189 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:12.499040 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.499070 3102485 retry.go:31] will retry after 441.708268ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.545231 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:12.599083 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:12.632027 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.632058 3102485 retry.go:31] will retry after 761.80126ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:12.703563 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.703597 3102485 retry.go:31] will retry after 576.888728ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:12.940986 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:13.065918 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.065966 3102485 retry.go:31] will retry after 1.372537837s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.280839 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:13.373138 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.373184 3102485 retry.go:31] will retry after 778.918432ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.394475 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0923 12:44:13.483130 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.483174 3102485 retry.go:31] will retry after 941.531266ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.556317 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0923 12:44:13.664130 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:13.664168 3102485 retry.go:31] will retry after 1.530430265s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:14.072801 3102485 node_ready.go:53] error getting node "old-k8s-version-605045": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-605045": dial tcp 192.168.85.2:8443: connect: connection refused
I0923 12:44:14.153155 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:14.270606 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:14.270640 3102485 retry.go:31] will retry after 1.366500149s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:14.425000 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:14.439266 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:14.516901 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:14.516936 3102485 retry.go:31] will retry after 2.020433238s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:14.545482 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:14.545532 3102485 retry.go:31] will retry after 2.067837068s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:15.195357 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0923 12:44:15.332174 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:15.332208 3102485 retry.go:31] will retry after 2.534677864s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:15.638185 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:15.729287 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:15.729330 3102485 retry.go:31] will retry after 1.506332209s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:16.073279 3102485 node_ready.go:53] error getting node "old-k8s-version-605045": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-605045": dial tcp 192.168.85.2:8443: connect: connection refused
I0923 12:44:16.538054 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0923 12:44:16.608324 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:16.608360 3102485 retry.go:31] will retry after 1.576341399s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:16.613566 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:16.689599 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:16.689638 3102485 retry.go:31] will retry after 1.665316473s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:17.236650 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0923 12:44:17.365271 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:17.365363 3102485 retry.go:31] will retry after 5.101349325s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:17.867031 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0923 12:44:17.982248 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:17.982283 3102485 retry.go:31] will retry after 4.028563817s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:18.073378 3102485 node_ready.go:53] error getting node "old-k8s-version-605045": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-605045": dial tcp 192.168.85.2:8443: connect: connection refused
I0923 12:44:18.185820 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:18.356036 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0923 12:44:18.626369 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:18.626406 3102485 retry.go:31] will retry after 5.61188231s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0923 12:44:18.930992 3102485 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:18.931030 3102485 retry.go:31] will retry after 2.624806836s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0923 12:44:21.556470 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0923 12:44:22.011065 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:44:22.467566 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0923 12:44:24.239379 3102485 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:44:29.242123 3102485 node_ready.go:49] node "old-k8s-version-605045" has status "Ready":"True"
I0923 12:44:29.242164 3102485 node_ready.go:38] duration metric: took 19.170103894s for node "old-k8s-version-605045" to be "Ready" ...
I0923 12:44:29.242182 3102485 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0923 12:44:29.381435 3102485 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-pb95z" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.442082 3102485 pod_ready.go:93] pod "coredns-74ff55c5b-pb95z" in "kube-system" namespace has status "Ready":"True"
I0923 12:44:29.442160 3102485 pod_ready.go:82] duration metric: took 60.676392ms for pod "coredns-74ff55c5b-pb95z" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.442186 3102485 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.459384 3102485 pod_ready.go:93] pod "etcd-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:44:29.459457 3102485 pod_ready.go:82] duration metric: took 17.248479ms for pod "etcd-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.459487 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.486874 3102485 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:44:29.486946 3102485 pod_ready.go:82] duration metric: took 27.436322ms for pod "kube-apiserver-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:44:29.486973 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:44:30.322691 3102485 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (8.76610799s)
I0923 12:44:30.387879 3102485 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.376765342s)
I0923 12:44:30.536576 3102485 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (8.068972613s)
I0923 12:44:30.536655 3102485 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-605045"
I0923 12:44:30.595156 3102485 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (6.355702867s)
I0923 12:44:30.599206 3102485 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-605045 addons enable metrics-server
I0923 12:44:30.602777 3102485 out.go:177] * Enabled addons: default-storageclass, storage-provisioner, metrics-server, dashboard
I0923 12:44:30.605613 3102485 addons.go:510] duration metric: took 20.862409042s for enable addons: enabled=[default-storageclass storage-provisioner metrics-server dashboard]
I0923 12:44:31.493314 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:33.992763 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:35.994902 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:38.493457 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:40.493789 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:42.493878 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:44.993659 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:46.994117 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:48.996838 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:50.998691 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:53.022233 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:55.494758 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:57.994077 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:44:59.994505 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:01.994596 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:04.494404 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:06.993708 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:09.072448 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:11.493700 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:13.493962 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:15.494158 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:17.496501 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:19.993060 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:22.015003 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:24.539397 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:26.994612 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:29.497184 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:31.501408 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:33.994220 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:36.493434 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:38.494618 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:40.993697 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:42.993796 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:45.494023 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:47.993809 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:49.493816 3102485 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:49.493848 3102485 pod_ready.go:82] duration metric: took 1m20.006846553s for pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.493861 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-9m2fb" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.504701 3102485 pod_ready.go:93] pod "kube-proxy-9m2fb" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:49.504776 3102485 pod_ready.go:82] duration metric: took 10.905739ms for pod "kube-proxy-9m2fb" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.504803 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:51.512244 3102485 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:52.510949 3102485 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:52.510976 3102485 pod_ready.go:82] duration metric: took 3.006151928s for pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:52.510987 3102485 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace to be "Ready" ...
I0923 12:45:54.517408 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:56.517938 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:59.018038 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:01.521178 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:04.017169 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:06.018012 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:08.020342 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:10.050466 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:12.519575 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:15.028066 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:17.516764 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:19.517985 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:21.527598 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:24.018262 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:26.516738 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:28.517429 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:30.517488 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:32.517691 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:34.518155 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:37.019755 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:39.517509 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:41.517906 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:44.018200 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:46.518620 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:49.017532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:51.517271 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:53.518286 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:55.518511 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:58.018393 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:00.040850 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:02.519588 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:05.017851 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:07.517609 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:10.120171 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:12.516830 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:14.518809 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:17.017796 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:19.517358 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:21.521109 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:24.017997 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:26.517391 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:28.518031 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:31.017891 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:33.022935 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:35.025532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:37.517416 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:39.518031 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:41.527952 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:44.017386 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:46.017790 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:48.018509 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:50.019200 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:52.517046 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:54.519238 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:57.017644 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:59.018465 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:01.517860 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:04.020306 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:06.518007 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:08.518217 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:10.518374 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:13.017127 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:15.025052 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:17.518912 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:19.519575 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:22.017866 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:24.519753 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:27.018096 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:29.517084 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:31.518247 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:34.017560 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:36.018135 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:38.018563 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:40.033260 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:42.516878 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:44.518028 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:47.017558 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:49.017887 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:51.104384 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:53.518375 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:56.017073 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:58.018046 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:00.051557 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:02.518428 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:05.018309 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:07.517421 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:09.517540 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:12.017948 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:14.518324 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:16.518725 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:19.017071 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:21.018952 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:23.517806 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:26.017683 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:28.018157 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:30.030456 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:32.517329 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:34.517424 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:36.517606 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:39.017532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:41.516456 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:43.522639 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:46.019113 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:48.022948 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:50.517786 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:52.520527 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:52.520556 3102485 pod_ready.go:82] duration metric: took 4m0.009561565s for pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace to be "Ready" ...
E0923 12:49:52.520568 3102485 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0923 12:49:52.520575 3102485 pod_ready.go:39] duration metric: took 5m23.27838172s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0923 12:49:52.520590 3102485 api_server.go:52] waiting for apiserver process to appear ...
I0923 12:49:52.520620 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:49:52.520680 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:49:52.582116 3102485 cri.go:89] found id: "b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:49:52.582139 3102485 cri.go:89] found id: "2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:49:52.582143 3102485 cri.go:89] found id: ""
I0923 12:49:52.582150 3102485 logs.go:276] 2 containers: [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85]
I0923 12:49:52.582208 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.586211 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.590182 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:49:52.590251 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:49:52.643577 3102485 cri.go:89] found id: "ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:49:52.643596 3102485 cri.go:89] found id: "1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:49:52.643601 3102485 cri.go:89] found id: ""
I0923 12:49:52.643608 3102485 logs.go:276] 2 containers: [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2]
I0923 12:49:52.643669 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.649826 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.653847 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:49:52.653956 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:49:52.723423 3102485 cri.go:89] found id: "0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:49:52.723449 3102485 cri.go:89] found id: "8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:49:52.723465 3102485 cri.go:89] found id: ""
I0923 12:49:52.723473 3102485 logs.go:276] 2 containers: [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993]
I0923 12:49:52.723539 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.728406 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.739893 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:49:52.739971 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:49:52.794862 3102485 cri.go:89] found id: "df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:49:52.794881 3102485 cri.go:89] found id: "2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:49:52.794886 3102485 cri.go:89] found id: ""
I0923 12:49:52.794893 3102485 logs.go:276] 2 containers: [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef]
I0923 12:49:52.794947 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.798877 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.802564 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:49:52.802683 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:49:52.882169 3102485 cri.go:89] found id: "3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:49:52.882242 3102485 cri.go:89] found id: "c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:49:52.882262 3102485 cri.go:89] found id: ""
I0923 12:49:52.882286 3102485 logs.go:276] 2 containers: [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8]
I0923 12:49:52.882372 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.886656 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.890389 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:49:52.890507 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:49:52.943880 3102485 cri.go:89] found id: "37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:49:52.943913 3102485 cri.go:89] found id: "7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:49:52.943918 3102485 cri.go:89] found id: ""
I0923 12:49:52.943926 3102485 logs.go:276] 2 containers: [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af]
I0923 12:49:52.943988 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.947650 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.951076 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:49:52.951161 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:49:52.988808 3102485 cri.go:89] found id: "a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:49:52.988830 3102485 cri.go:89] found id: "7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:49:52.988835 3102485 cri.go:89] found id: ""
I0923 12:49:52.988848 3102485 logs.go:276] 2 containers: [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f]
I0923 12:49:52.988909 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.992697 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.996136 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:49:52.996206 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:49:53.042311 3102485 cri.go:89] found id: "7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:49:53.042336 3102485 cri.go:89] found id: ""
I0923 12:49:53.042345 3102485 logs.go:276] 1 containers: [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a]
I0923 12:49:53.042421 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.046105 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:49:53.046208 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:49:53.088283 3102485 cri.go:89] found id: "036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:49:53.088309 3102485 cri.go:89] found id: "0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:49:53.088314 3102485 cri.go:89] found id: ""
I0923 12:49:53.088321 3102485 logs.go:276] 2 containers: [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649]
I0923 12:49:53.088398 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.092061 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.095472 3102485 logs.go:123] Gathering logs for kube-controller-manager [7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af] ...
I0923 12:49:53.095536 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:49:53.152558 3102485 logs.go:123] Gathering logs for kube-apiserver [2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85] ...
I0923 12:49:53.152592 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:49:53.214318 3102485 logs.go:123] Gathering logs for kube-scheduler [2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef] ...
I0923 12:49:53.214352 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:49:53.256471 3102485 logs.go:123] Gathering logs for etcd [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a] ...
I0923 12:49:53.256504 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:49:53.299642 3102485 logs.go:123] Gathering logs for coredns [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb] ...
I0923 12:49:53.299671 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:49:53.341299 3102485 logs.go:123] Gathering logs for kubernetes-dashboard [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a] ...
I0923 12:49:53.341331 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:49:53.382087 3102485 logs.go:123] Gathering logs for kubelet ...
I0923 12:49:53.382166 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:49:53.438281 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989651 658 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438539 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989920 658 reflector.go:138] object-"kube-system"/"coredns-token-ljzll": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-ljzll" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438760 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990004 658 reflector.go:138] object-"kube-system"/"kube-proxy-token-267ss": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-267ss" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438973 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990077 658 reflector.go:138] object-"kube-system"/"kindnet-token-np6vw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-np6vw" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439173 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990159 658 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439407 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990239 658 reflector.go:138] object-"kube-system"/"storage-provisioner-token-xjwkr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-xjwkr" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439616 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990319 658 reflector.go:138] object-"default"/"default-token-cl5l9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-cl5l9" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.441346 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:29 old-k8s-version-605045 kubelet[658]: E0923 12:44:29.145826 658 reflector.go:138] object-"kube-system"/"metrics-server-token-7ztnm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-7ztnm" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.449611 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.630570 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.450439 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.843250 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.453223 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:45 old-k8s-version-605045 kubelet[658]: E0923 12:44:45.501139 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.455318 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:53 old-k8s-version-605045 kubelet[658]: E0923 12:44:53.971600 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.455647 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:54 old-k8s-version-605045 kubelet[658]: E0923 12:44:54.975959 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.455973 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.411779 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.456158 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.466238 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.456946 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:04 old-k8s-version-605045 kubelet[658]: E0923 12:45:04.013052 658 pod_workers.go:191] Error syncing pod bc5f03f4-beaa-4a9b-a052-24d558f95beb ("storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"
W0923 12:49:53.459838 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:09 old-k8s-version-605045 kubelet[658]: E0923 12:45:09.476299 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.460496 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:11 old-k8s-version-605045 kubelet[658]: E0923 12:45:11.038230 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.460830 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:16 old-k8s-version-605045 kubelet[658]: E0923 12:45:16.410049 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.461150 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:24 old-k8s-version-605045 kubelet[658]: E0923 12:45:24.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.461739 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:32 old-k8s-version-605045 kubelet[658]: E0923 12:45:32.109824 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.462067 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:36 old-k8s-version-605045 kubelet[658]: E0923 12:45:36.410320 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.462252 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:39 old-k8s-version-605045 kubelet[658]: E0923 12:45:39.471779 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.462581 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:51 old-k8s-version-605045 kubelet[658]: E0923 12:45:51.466132 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.465099 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:53 old-k8s-version-605045 kubelet[658]: E0923 12:45:53.479122 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.465442 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:02 old-k8s-version-605045 kubelet[658]: E0923 12:46:02.465540 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.465628 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:07 old-k8s-version-605045 kubelet[658]: E0923 12:46:07.466508 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.466225 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:17 old-k8s-version-605045 kubelet[658]: E0923 12:46:17.299572 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.466410 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:22 old-k8s-version-605045 kubelet[658]: E0923 12:46:22.465904 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.466735 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:26 old-k8s-version-605045 kubelet[658]: E0923 12:46:26.410045 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.466919 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:35 old-k8s-version-605045 kubelet[658]: E0923 12:46:35.469207 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.467245 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:38 old-k8s-version-605045 kubelet[658]: E0923 12:46:38.465574 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.467441 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:47 old-k8s-version-605045 kubelet[658]: E0923 12:46:47.467336 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.467773 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:52 old-k8s-version-605045 kubelet[658]: E0923 12:46:52.465530 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.467957 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:02 old-k8s-version-605045 kubelet[658]: E0923 12:47:02.465921 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.468283 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:07 old-k8s-version-605045 kubelet[658]: E0923 12:47:07.466739 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.471301 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:14 old-k8s-version-605045 kubelet[658]: E0923 12:47:14.473947 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.471640 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:22 old-k8s-version-605045 kubelet[658]: E0923 12:47:22.465965 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.471829 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:26 old-k8s-version-605045 kubelet[658]: E0923 12:47:26.465940 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.472156 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:33 old-k8s-version-605045 kubelet[658]: E0923 12:47:33.467362 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.472346 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:38 old-k8s-version-605045 kubelet[658]: E0923 12:47:38.465964 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.472930 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:45 old-k8s-version-605045 kubelet[658]: E0923 12:47:45.536822 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473261 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:46 old-k8s-version-605045 kubelet[658]: E0923 12:47:46.541111 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473453 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:49 old-k8s-version-605045 kubelet[658]: E0923 12:47:49.465875 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.473780 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:00 old-k8s-version-605045 kubelet[658]: E0923 12:48:00.479190 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473963 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:02 old-k8s-version-605045 kubelet[658]: E0923 12:48:02.466076 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.474292 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:11 old-k8s-version-605045 kubelet[658]: E0923 12:48:11.465507 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.474477 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:16 old-k8s-version-605045 kubelet[658]: E0923 12:48:16.465938 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.474805 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:22 old-k8s-version-605045 kubelet[658]: E0923 12:48:22.465512 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.474992 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:30 old-k8s-version-605045 kubelet[658]: E0923 12:48:30.465840 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.475329 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:35 old-k8s-version-605045 kubelet[658]: E0923 12:48:35.474238 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.475519 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:44 old-k8s-version-605045 kubelet[658]: E0923 12:48:44.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.475848 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:49 old-k8s-version-605045 kubelet[658]: E0923 12:48:49.466027 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.476033 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:55 old-k8s-version-605045 kubelet[658]: E0923 12:48:55.466326 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.476361 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: E0923 12:49:04.465573 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.476571 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:07 old-k8s-version-605045 kubelet[658]: E0923 12:49:07.466660 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.476901 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: E0923 12:49:16.465577 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.477087 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:20 old-k8s-version-605045 kubelet[658]: E0923 12:49:20.466093 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.477419 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.477605 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.477931 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.478116 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.478441 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:49:53.478453 3102485 logs.go:123] Gathering logs for describe nodes ...
I0923 12:49:53.478468 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:49:53.626281 3102485 logs.go:123] Gathering logs for kube-scheduler [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89] ...
I0923 12:49:53.626312 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:49:53.671782 3102485 logs.go:123] Gathering logs for kube-proxy [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb] ...
I0923 12:49:53.671827 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:49:53.715920 3102485 logs.go:123] Gathering logs for kube-proxy [c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8] ...
I0923 12:49:53.715952 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:49:53.773316 3102485 logs.go:123] Gathering logs for kindnet [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6] ...
I0923 12:49:53.773355 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:49:53.824387 3102485 logs.go:123] Gathering logs for dmesg ...
I0923 12:49:53.824420 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:49:53.844575 3102485 logs.go:123] Gathering logs for coredns [8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993] ...
I0923 12:49:53.844609 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:49:53.896670 3102485 logs.go:123] Gathering logs for kube-controller-manager [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd] ...
I0923 12:49:53.896699 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:49:53.954435 3102485 logs.go:123] Gathering logs for kindnet [7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f] ...
I0923 12:49:53.954471 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:49:53.996651 3102485 logs.go:123] Gathering logs for storage-provisioner [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19] ...
I0923 12:49:53.996678 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:49:54.043169 3102485 logs.go:123] Gathering logs for storage-provisioner [0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649] ...
I0923 12:49:54.043207 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:49:54.091390 3102485 logs.go:123] Gathering logs for containerd ...
I0923 12:49:54.091423 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:49:54.154040 3102485 logs.go:123] Gathering logs for container status ...
I0923 12:49:54.154077 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:49:54.203612 3102485 logs.go:123] Gathering logs for kube-apiserver [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb] ...
I0923 12:49:54.203644 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:49:54.262283 3102485 logs.go:123] Gathering logs for etcd [1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2] ...
I0923 12:49:54.262317 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:49:54.308962 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:54.308990 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:49:54.309084 3102485 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0923 12:49:54.309098 3102485 out.go:270] Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:54.309128 3102485 out.go:270] Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:54.309153 3102485 out.go:270] Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:54.309161 3102485 out.go:270] Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:54.309172 3102485 out.go:270] Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:49:54.309177 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:54.309191 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:50:04.310257 3102485 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0923 12:50:04.324717 3102485 api_server.go:72] duration metric: took 5m54.581827s to wait for apiserver process to appear ...
I0923 12:50:04.324741 3102485 api_server.go:88] waiting for apiserver healthz status ...
I0923 12:50:04.324779 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:50:04.324837 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:50:04.371861 3102485 cri.go:89] found id: "b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:50:04.371888 3102485 cri.go:89] found id: "2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:50:04.371893 3102485 cri.go:89] found id: ""
I0923 12:50:04.371900 3102485 logs.go:276] 2 containers: [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85]
I0923 12:50:04.371957 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.376309 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.380595 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:50:04.380674 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:50:04.423923 3102485 cri.go:89] found id: "ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:50:04.423947 3102485 cri.go:89] found id: "1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:50:04.423952 3102485 cri.go:89] found id: ""
I0923 12:50:04.423960 3102485 logs.go:276] 2 containers: [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2]
I0923 12:50:04.424021 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.427792 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.431611 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:50:04.431688 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:50:04.472288 3102485 cri.go:89] found id: "0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:50:04.472310 3102485 cri.go:89] found id: "8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:50:04.472315 3102485 cri.go:89] found id: ""
I0923 12:50:04.472322 3102485 logs.go:276] 2 containers: [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993]
I0923 12:50:04.472380 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.476372 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.479861 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:50:04.479939 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:50:04.519484 3102485 cri.go:89] found id: "df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:50:04.519508 3102485 cri.go:89] found id: "2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:50:04.519513 3102485 cri.go:89] found id: ""
I0923 12:50:04.519521 3102485 logs.go:276] 2 containers: [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef]
I0923 12:50:04.519579 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.523443 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.527138 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:50:04.527233 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:50:04.571087 3102485 cri.go:89] found id: "3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:50:04.571107 3102485 cri.go:89] found id: "c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:50:04.571112 3102485 cri.go:89] found id: ""
I0923 12:50:04.571119 3102485 logs.go:276] 2 containers: [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8]
I0923 12:50:04.571176 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.575120 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.578668 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:50:04.578769 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:50:04.622151 3102485 cri.go:89] found id: "37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:50:04.622181 3102485 cri.go:89] found id: "7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:50:04.622186 3102485 cri.go:89] found id: ""
I0923 12:50:04.622194 3102485 logs.go:276] 2 containers: [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af]
I0923 12:50:04.622263 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.626117 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.629549 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:50:04.629624 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:50:04.669203 3102485 cri.go:89] found id: "a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:50:04.669277 3102485 cri.go:89] found id: "7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:50:04.669291 3102485 cri.go:89] found id: ""
I0923 12:50:04.669299 3102485 logs.go:276] 2 containers: [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f]
I0923 12:50:04.669417 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.673133 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.676629 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:50:04.676723 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:50:04.718915 3102485 cri.go:89] found id: "7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:50:04.718939 3102485 cri.go:89] found id: ""
I0923 12:50:04.718947 3102485 logs.go:276] 1 containers: [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a]
I0923 12:50:04.719031 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.723030 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:50:04.723104 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:50:04.773144 3102485 cri.go:89] found id: "036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:50:04.773225 3102485 cri.go:89] found id: "0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:50:04.773246 3102485 cri.go:89] found id: ""
I0923 12:50:04.773280 3102485 logs.go:276] 2 containers: [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649]
I0923 12:50:04.773393 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.777009 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.780384 3102485 logs.go:123] Gathering logs for describe nodes ...
I0923 12:50:04.780408 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:50:04.928411 3102485 logs.go:123] Gathering logs for etcd [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a] ...
I0923 12:50:04.928490 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:50:04.972999 3102485 logs.go:123] Gathering logs for kube-scheduler [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89] ...
I0923 12:50:04.973033 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:50:05.018662 3102485 logs.go:123] Gathering logs for kube-scheduler [2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef] ...
I0923 12:50:05.018696 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:50:05.073546 3102485 logs.go:123] Gathering logs for kube-controller-manager [7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af] ...
I0923 12:50:05.073652 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:50:05.135452 3102485 logs.go:123] Gathering logs for kindnet [7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f] ...
I0923 12:50:05.135491 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:50:05.180530 3102485 logs.go:123] Gathering logs for kubelet ...
I0923 12:50:05.180558 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:50:05.235647 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989651 658 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.235906 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989920 658 reflector.go:138] object-"kube-system"/"coredns-token-ljzll": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-ljzll" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236127 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990004 658 reflector.go:138] object-"kube-system"/"kube-proxy-token-267ss": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-267ss" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236340 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990077 658 reflector.go:138] object-"kube-system"/"kindnet-token-np6vw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-np6vw" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236541 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990159 658 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236769 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990239 658 reflector.go:138] object-"kube-system"/"storage-provisioner-token-xjwkr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-xjwkr" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236977 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990319 658 reflector.go:138] object-"default"/"default-token-cl5l9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-cl5l9" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.238682 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:29 old-k8s-version-605045 kubelet[658]: E0923 12:44:29.145826 658 reflector.go:138] object-"kube-system"/"metrics-server-token-7ztnm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-7ztnm" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.246929 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.630570 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.247743 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.843250 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.250499 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:45 old-k8s-version-605045 kubelet[658]: E0923 12:44:45.501139 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.252595 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:53 old-k8s-version-605045 kubelet[658]: E0923 12:44:53.971600 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.252923 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:54 old-k8s-version-605045 kubelet[658]: E0923 12:44:54.975959 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.253253 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.411779 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.253439 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.466238 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.254205 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:04 old-k8s-version-605045 kubelet[658]: E0923 12:45:04.013052 658 pod_workers.go:191] Error syncing pod bc5f03f4-beaa-4a9b-a052-24d558f95beb ("storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"
W0923 12:50:05.256971 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:09 old-k8s-version-605045 kubelet[658]: E0923 12:45:09.476299 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.257566 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:11 old-k8s-version-605045 kubelet[658]: E0923 12:45:11.038230 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.257891 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:16 old-k8s-version-605045 kubelet[658]: E0923 12:45:16.410049 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.258204 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:24 old-k8s-version-605045 kubelet[658]: E0923 12:45:24.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.258788 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:32 old-k8s-version-605045 kubelet[658]: E0923 12:45:32.109824 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.259112 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:36 old-k8s-version-605045 kubelet[658]: E0923 12:45:36.410320 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.259325 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:39 old-k8s-version-605045 kubelet[658]: E0923 12:45:39.471779 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.259652 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:51 old-k8s-version-605045 kubelet[658]: E0923 12:45:51.466132 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.262093 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:53 old-k8s-version-605045 kubelet[658]: E0923 12:45:53.479122 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.262418 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:02 old-k8s-version-605045 kubelet[658]: E0923 12:46:02.465540 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.262604 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:07 old-k8s-version-605045 kubelet[658]: E0923 12:46:07.466508 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.263188 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:17 old-k8s-version-605045 kubelet[658]: E0923 12:46:17.299572 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.263379 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:22 old-k8s-version-605045 kubelet[658]: E0923 12:46:22.465904 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.263705 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:26 old-k8s-version-605045 kubelet[658]: E0923 12:46:26.410045 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.263889 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:35 old-k8s-version-605045 kubelet[658]: E0923 12:46:35.469207 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.264212 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:38 old-k8s-version-605045 kubelet[658]: E0923 12:46:38.465574 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.264396 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:47 old-k8s-version-605045 kubelet[658]: E0923 12:46:47.467336 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.264721 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:52 old-k8s-version-605045 kubelet[658]: E0923 12:46:52.465530 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.264904 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:02 old-k8s-version-605045 kubelet[658]: E0923 12:47:02.465921 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.265242 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:07 old-k8s-version-605045 kubelet[658]: E0923 12:47:07.466739 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.267689 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:14 old-k8s-version-605045 kubelet[658]: E0923 12:47:14.473947 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.268018 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:22 old-k8s-version-605045 kubelet[658]: E0923 12:47:22.465965 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.268204 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:26 old-k8s-version-605045 kubelet[658]: E0923 12:47:26.465940 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.268528 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:33 old-k8s-version-605045 kubelet[658]: E0923 12:47:33.467362 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.268712 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:38 old-k8s-version-605045 kubelet[658]: E0923 12:47:38.465964 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.269300 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:45 old-k8s-version-605045 kubelet[658]: E0923 12:47:45.536822 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.269629 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:46 old-k8s-version-605045 kubelet[658]: E0923 12:47:46.541111 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.269813 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:49 old-k8s-version-605045 kubelet[658]: E0923 12:47:49.465875 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.270138 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:00 old-k8s-version-605045 kubelet[658]: E0923 12:48:00.479190 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.270323 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:02 old-k8s-version-605045 kubelet[658]: E0923 12:48:02.466076 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.270652 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:11 old-k8s-version-605045 kubelet[658]: E0923 12:48:11.465507 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.270835 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:16 old-k8s-version-605045 kubelet[658]: E0923 12:48:16.465938 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.271160 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:22 old-k8s-version-605045 kubelet[658]: E0923 12:48:22.465512 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.271360 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:30 old-k8s-version-605045 kubelet[658]: E0923 12:48:30.465840 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.271686 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:35 old-k8s-version-605045 kubelet[658]: E0923 12:48:35.474238 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.271870 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:44 old-k8s-version-605045 kubelet[658]: E0923 12:48:44.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.272196 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:49 old-k8s-version-605045 kubelet[658]: E0923 12:48:49.466027 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.272381 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:55 old-k8s-version-605045 kubelet[658]: E0923 12:48:55.466326 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.272706 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: E0923 12:49:04.465573 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.272892 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:07 old-k8s-version-605045 kubelet[658]: E0923 12:49:07.466660 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.273218 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: E0923 12:49:16.465577 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.273406 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:20 old-k8s-version-605045 kubelet[658]: E0923 12:49:20.466093 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.273731 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.273915 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.274240 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.274425 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.274751 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.277200 3102485 logs.go:138] Found kubelet problem: Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.277533 3102485 logs.go:138] Found kubelet problem: Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:50:05.277544 3102485 logs.go:123] Gathering logs for etcd [1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2] ...
I0923 12:50:05.277558 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:50:05.322579 3102485 logs.go:123] Gathering logs for coredns [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb] ...
I0923 12:50:05.322658 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:50:05.365537 3102485 logs.go:123] Gathering logs for coredns [8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993] ...
I0923 12:50:05.365569 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:50:05.407857 3102485 logs.go:123] Gathering logs for kube-proxy [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb] ...
I0923 12:50:05.407925 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:50:05.456230 3102485 logs.go:123] Gathering logs for kube-proxy [c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8] ...
I0923 12:50:05.456256 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:50:05.499217 3102485 logs.go:123] Gathering logs for kube-controller-manager [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd] ...
I0923 12:50:05.499248 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:50:05.563528 3102485 logs.go:123] Gathering logs for kubernetes-dashboard [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a] ...
I0923 12:50:05.563564 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:50:05.606773 3102485 logs.go:123] Gathering logs for kube-apiserver [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb] ...
I0923 12:50:05.606803 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:50:05.679239 3102485 logs.go:123] Gathering logs for containerd ...
I0923 12:50:05.679272 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:50:05.750829 3102485 logs.go:123] Gathering logs for storage-provisioner [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19] ...
I0923 12:50:05.750867 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:50:05.792354 3102485 logs.go:123] Gathering logs for storage-provisioner [0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649] ...
I0923 12:50:05.792381 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:50:05.836837 3102485 logs.go:123] Gathering logs for kindnet [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6] ...
I0923 12:50:05.836865 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:50:05.905615 3102485 logs.go:123] Gathering logs for kube-apiserver [2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85] ...
I0923 12:50:05.905647 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:50:05.963389 3102485 logs.go:123] Gathering logs for container status ...
I0923 12:50:05.963426 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:50:06.020318 3102485 logs.go:123] Gathering logs for dmesg ...
I0923 12:50:06.020356 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:50:06.038364 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:50:06.038396 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:50:06.038444 3102485 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0923 12:50:06.038457 3102485 out.go:270] Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:06.038464 3102485 out.go:270] Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:06.038477 3102485 out.go:270] Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:06.038485 3102485 out.go:270] Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:06.038496 3102485 out.go:270] Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:50:06.038518 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:50:06.038531 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:50:16.039609 3102485 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0923 12:50:16.049801 3102485 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0923 12:50:16.052452 3102485 out.go:201]
W0923 12:50:16.054574 3102485 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0923 12:50:16.054647 3102485 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0923 12:50:16.054671 3102485 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0923 12:50:16.054677 3102485 out.go:270] *
*
W0923 12:50:16.055931 3102485 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0923 12:50:16.058110 3102485 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-605045 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-605045
helpers_test.go:235: (dbg) docker inspect old-k8s-version-605045:
-- stdout --
[
{
"Id": "ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1",
"Created": "2024-09-23T12:41:07.572363964Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 3102679,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-09-23T12:44:01.556116217Z",
"FinishedAt": "2024-09-23T12:44:00.300083765Z"
},
"Image": "sha256:c94982da1293baee77c00993711af197ed62d6b1a4ee12c0caa4f57c70de4fdc",
"ResolvConfPath": "/var/lib/docker/containers/ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1/hostname",
"HostsPath": "/var/lib/docker/containers/ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1/hosts",
"LogPath": "/var/lib/docker/containers/ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1/ebb5aa29eb89bc383c06bc55a271705a08442d385b07e836b899df05c91d95c1-json.log",
"Name": "/old-k8s-version-605045",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-605045:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-605045",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/c61653332bf8039cb4cd414ec2430ed0d1647a22650a9129cc5a5c78eae05bfd-init/diff:/var/lib/docker/overlay2/e2b16ea68ee0680d6b3555ff1ad64b95e5f88f6159373a302ec4d54fa432d99a/diff",
"MergedDir": "/var/lib/docker/overlay2/c61653332bf8039cb4cd414ec2430ed0d1647a22650a9129cc5a5c78eae05bfd/merged",
"UpperDir": "/var/lib/docker/overlay2/c61653332bf8039cb4cd414ec2430ed0d1647a22650a9129cc5a5c78eae05bfd/diff",
"WorkDir": "/var/lib/docker/overlay2/c61653332bf8039cb4cd414ec2430ed0d1647a22650a9129cc5a5c78eae05bfd/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-605045",
"Source": "/var/lib/docker/volumes/old-k8s-version-605045/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-605045",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-605045",
"name.minikube.sigs.k8s.io": "old-k8s-version-605045",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "145254c45e4791100c02ebcc9b9140ac306fabe564865155c0dd2c63ee48c97b",
"SandboxKey": "/var/run/docker/netns/145254c45e47",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "42087"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "42088"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "42091"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "42089"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "42090"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-605045": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:55:02",
"DriverOpts": null,
"NetworkID": "54eadf17238d4d31d7e9b95d0cde7c8feef524ccd6ac4891d402b519e66c82ba",
"EndpointID": "5cdb0495cf4e11acb64709afab5ebf328fa6b550f2bfd9464b981f6b0cdb2e45",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-605045",
"ebb5aa29eb89"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-605045 -n old-k8s-version-605045
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-605045 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-605045 logs -n 25: (2.958413214s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| delete | -p cilium-492817 | cilium-492817 | jenkins | v1.34.0 | 23 Sep 24 12:39 UTC | 23 Sep 24 12:39 UTC |
| ssh | force-systemd-flag-410328 | force-systemd-flag-410328 | jenkins | v1.34.0 | 23 Sep 24 12:39 UTC | 23 Sep 24 12:39 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| start | -p force-systemd-env-461049 | force-systemd-env-461049 | jenkins | v1.34.0 | 23 Sep 24 12:39 UTC | 23 Sep 24 12:40 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p force-systemd-flag-410328 | force-systemd-flag-410328 | jenkins | v1.34.0 | 23 Sep 24 12:39 UTC | 23 Sep 24 12:39 UTC |
| start | -p cert-expiration-130595 | cert-expiration-130595 | jenkins | v1.34.0 | 23 Sep 24 12:39 UTC | 23 Sep 24 12:40 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-461049 | force-systemd-env-461049 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:40 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-461049 | force-systemd-env-461049 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:40 UTC |
| start | -p cert-options-007095 | cert-options-007095 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:40 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-007095 ssh | cert-options-007095 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:40 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-007095 -- sudo | cert-options-007095 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:40 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-007095 | cert-options-007095 | jenkins | v1.34.0 | 23 Sep 24 12:40 UTC | 23 Sep 24 12:41 UTC |
| start | -p old-k8s-version-605045 | old-k8s-version-605045 | jenkins | v1.34.0 | 23 Sep 24 12:41 UTC | 23 Sep 24 12:43 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-130595 | cert-expiration-130595 | jenkins | v1.34.0 | 23 Sep 24 12:43 UTC | 23 Sep 24 12:43 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-130595 | cert-expiration-130595 | jenkins | v1.34.0 | 23 Sep 24 12:43 UTC | 23 Sep 24 12:43 UTC |
| start | -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:43 UTC | 23 Sep 24 12:44 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-605045 | old-k8s-version-605045 | jenkins | v1.34.0 | 23 Sep 24 12:43 UTC | 23 Sep 24 12:43 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-605045 | old-k8s-version-605045 | jenkins | v1.34.0 | 23 Sep 24 12:43 UTC | 23 Sep 24 12:44 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-605045 | old-k8s-version-605045 | jenkins | v1.34.0 | 23 Sep 24 12:44 UTC | 23 Sep 24 12:44 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-605045 | old-k8s-version-605045 | jenkins | v1.34.0 | 23 Sep 24 12:44 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:45 UTC | 23 Sep 24 12:45 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:45 UTC | 23 Sep 24 12:45 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:45 UTC | 23 Sep 24 12:45 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:45 UTC | 23 Sep 24 12:50 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.1 | | | | | |
| image | no-preload-569169 image list | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:50 UTC | 23 Sep 24 12:50 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-569169 | no-preload-569169 | jenkins | v1.34.0 | 23 Sep 24 12:50 UTC | 23 Sep 24 12:50 UTC |
| | --alsologtostderr -v=1 | | | | | |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/09/23 12:45:14
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.23.0 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0923 12:45:14.279586 3107459 out.go:345] Setting OutFile to fd 1 ...
I0923 12:45:14.279730 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:45:14.279742 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:45:14.279747 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:45:14.279978 3107459 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19688-2897765/.minikube/bin
I0923 12:45:14.280327 3107459 out.go:352] Setting JSON to false
I0923 12:45:14.281467 3107459 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":160062,"bootTime":1726935453,"procs":244,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1070-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0923 12:45:14.281537 3107459 start.go:139] virtualization:
I0923 12:45:14.283922 3107459 out.go:177] * [no-preload-569169] minikube v1.34.0 on Ubuntu 20.04 (arm64)
I0923 12:45:14.286502 3107459 out.go:177] - MINIKUBE_LOCATION=19688
I0923 12:45:14.286583 3107459 notify.go:220] Checking for updates...
I0923 12:45:14.290067 3107459 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0923 12:45:14.291728 3107459 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:45:14.293160 3107459 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19688-2897765/.minikube
I0923 12:45:14.294611 3107459 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0923 12:45:14.296419 3107459 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0923 12:45:14.298822 3107459 config.go:182] Loaded profile config "no-preload-569169": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0923 12:45:14.299443 3107459 driver.go:394] Setting default libvirt URI to qemu:///system
I0923 12:45:14.326338 3107459 docker.go:123] docker version: linux-27.3.1:Docker Engine - Community
I0923 12:45:14.326470 3107459 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0923 12:45:14.379094 3107459 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-09-23 12:45:14.369137989 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0923 12:45:14.379215 3107459 docker.go:318] overlay module found
I0923 12:45:14.381182 3107459 out.go:177] * Using the docker driver based on existing profile
I0923 12:45:14.383173 3107459 start.go:297] selected driver: docker
I0923 12:45:14.383191 3107459 start.go:901] validating driver "docker" against &{Name:no-preload-569169 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-569169 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false Mount
String:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:45:14.383402 3107459 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0923 12:45:14.384035 3107459 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0923 12:45:14.447151 3107459 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:false BridgeNfIP6Tables:false Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-09-23 12:45:14.436437037 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1070-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:a
arch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214839296 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.3.1 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c Expected:7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c} RuncCommit:{ID:v1.1.14-0-g2c9f560 Expected:v1.1.14-0-g2c9f560} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:[WARNING: brid
ge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.17.1] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.7]] Warnings:<nil>}}
I0923 12:45:14.447605 3107459 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0923 12:45:14.447630 3107459 cni.go:84] Creating CNI manager for ""
I0923 12:45:14.447680 3107459 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0923 12:45:14.447722 3107459 start.go:340] cluster config:
{Name:no-preload-569169 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-569169 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker Moun
tIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:45:14.450403 3107459 out.go:177] * Starting "no-preload-569169" primary control-plane node in "no-preload-569169" cluster
I0923 12:45:14.452700 3107459 cache.go:121] Beginning downloading kic base image for docker with containerd
I0923 12:45:14.455464 3107459 out.go:177] * Pulling base image v0.0.45-1726784731-19672 ...
I0923 12:45:14.457939 3107459 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0923 12:45:14.458027 3107459 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed in local docker daemon
I0923 12:45:14.458099 3107459 profile.go:143] Saving config to /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/config.json ...
I0923 12:45:14.458403 3107459 cache.go:107] acquiring lock: {Name:mke8d8830849e51322239aa8cc0a3c463bcc6371 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458480 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0923 12:45:14.458498 3107459 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 100.397µs
I0923 12:45:14.458509 3107459 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0923 12:45:14.458535 3107459 cache.go:107] acquiring lock: {Name:mk380c7f0f446757cd4e5205bb6adf67fd3beaca Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458573 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 exists
I0923 12:45:14.458583 3107459 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.31.1" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1" took 58.314µs
I0923 12:45:14.458589 3107459 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.31.1 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.1 succeeded
I0923 12:45:14.458599 3107459 cache.go:107] acquiring lock: {Name:mk426dedfca3645ec0df888f1b65ce710f022963 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458632 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 exists
I0923 12:45:14.458703 3107459 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.31.1" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1" took 103.523µs
I0923 12:45:14.458720 3107459 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.31.1 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.1 succeeded
I0923 12:45:14.458744 3107459 cache.go:107] acquiring lock: {Name:mk1a6719ed2eed12e30fe3565a32489c6e7f380e Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458788 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 exists
I0923 12:45:14.458799 3107459 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.31.1" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1" took 68.037µs
I0923 12:45:14.458805 3107459 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.31.1 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.1 succeeded
I0923 12:45:14.458831 3107459 cache.go:107] acquiring lock: {Name:mk77a731cf4156fe8fa23e8d48b5d6e3eaebdeb9 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458881 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 exists
I0923 12:45:14.458893 3107459 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.31.1" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1" took 64.09µs
I0923 12:45:14.458900 3107459 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.31.1 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.1 succeeded
I0923 12:45:14.458917 3107459 cache.go:107] acquiring lock: {Name:mk8839f0ea7b3b217dec1b86ead05f417efd7de6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.458972 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0923 12:45:14.458991 3107459 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 69.883µs
I0923 12:45:14.459013 3107459 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0923 12:45:14.459024 3107459 cache.go:107] acquiring lock: {Name:mk1f7b1ae4ef6e0a2df8534fa736ff06aaa6deb5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.459059 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 exists
I0923 12:45:14.459068 3107459 cache.go:96] cache image "registry.k8s.io/etcd:3.5.15-0" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0" took 46.408µs
I0923 12:45:14.459090 3107459 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.15-0 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 succeeded
I0923 12:45:14.459099 3107459 cache.go:107] acquiring lock: {Name:mka73553c2e8e662311c860daefb5fd00644fe51 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.459133 3107459 cache.go:115] /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 exists
I0923 12:45:14.459142 3107459 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.3" -> "/home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3" took 44.414µs
I0923 12:45:14.459164 3107459 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.3 -> /home/jenkins/minikube-integration/19688-2897765/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.3 succeeded
I0923 12:45:14.459172 3107459 cache.go:87] Successfully saved all images to host disk.
I0923 12:45:14.477893 3107459 image.go:98] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed in local docker daemon, skipping pull
I0923 12:45:14.477914 3107459 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed exists in daemon, skipping load
I0923 12:45:14.477927 3107459 cache.go:194] Successfully downloaded all kic artifacts
I0923 12:45:14.477951 3107459 start.go:360] acquireMachinesLock for no-preload-569169: {Name:mkcc5cff1c73f085ff71e70bfa5e7e882d7b4ad1 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0923 12:45:14.478003 3107459 start.go:364] duration metric: took 37.144µs to acquireMachinesLock for "no-preload-569169"
I0923 12:45:14.478024 3107459 start.go:96] Skipping create...Using existing machine configuration
I0923 12:45:14.478029 3107459 fix.go:54] fixHost starting:
I0923 12:45:14.478325 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:14.504303 3107459 fix.go:112] recreateIfNeeded on no-preload-569169: state=Stopped err=<nil>
W0923 12:45:14.504329 3107459 fix.go:138] unexpected machine state, will restart: <nil>
I0923 12:45:14.510080 3107459 out.go:177] * Restarting existing docker container for "no-preload-569169" ...
I0923 12:45:11.493700 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:13.493962 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:15.494158 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:14.512866 3107459 cli_runner.go:164] Run: docker start no-preload-569169
I0923 12:45:14.867922 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:14.886982 3107459 kic.go:430] container "no-preload-569169" state is running.
I0923 12:45:14.887541 3107459 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-569169
I0923 12:45:14.911517 3107459 profile.go:143] Saving config to /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/config.json ...
I0923 12:45:14.912256 3107459 machine.go:93] provisionDockerMachine start ...
I0923 12:45:14.912404 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:14.933454 3107459 main.go:141] libmachine: Using SSH client type: native
I0923 12:45:14.933722 3107459 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42092 <nil> <nil>}
I0923 12:45:14.933732 3107459 main.go:141] libmachine: About to run SSH command:
hostname
I0923 12:45:14.934395 3107459 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:60690->127.0.0.1:42092: read: connection reset by peer
I0923 12:45:18.083493 3107459 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-569169
I0923 12:45:18.083554 3107459 ubuntu.go:169] provisioning hostname "no-preload-569169"
I0923 12:45:18.083624 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:18.109233 3107459 main.go:141] libmachine: Using SSH client type: native
I0923 12:45:18.109539 3107459 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42092 <nil> <nil>}
I0923 12:45:18.109564 3107459 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-569169 && echo "no-preload-569169" | sudo tee /etc/hostname
I0923 12:45:18.263968 3107459 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-569169
I0923 12:45:18.264085 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:18.282253 3107459 main.go:141] libmachine: Using SSH client type: native
I0923 12:45:18.282503 3107459 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x413650] 0x415e90 <nil> [] 0s} 127.0.0.1 42092 <nil> <nil>}
I0923 12:45:18.282526 3107459 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-569169' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-569169/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-569169' | sudo tee -a /etc/hosts;
fi
fi
I0923 12:45:18.415868 3107459 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0923 12:45:18.415959 3107459 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19688-2897765/.minikube CaCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19688-2897765/.minikube}
I0923 12:45:18.416008 3107459 ubuntu.go:177] setting up certificates
I0923 12:45:18.416031 3107459 provision.go:84] configureAuth start
I0923 12:45:18.416110 3107459 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-569169
I0923 12:45:18.433785 3107459 provision.go:143] copyHostCerts
I0923 12:45:18.433862 3107459 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem, removing ...
I0923 12:45:18.433881 3107459 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem
I0923 12:45:18.433961 3107459 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.pem (1078 bytes)
I0923 12:45:18.434060 3107459 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem, removing ...
I0923 12:45:18.434065 3107459 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem
I0923 12:45:18.434093 3107459 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/cert.pem (1123 bytes)
I0923 12:45:18.434159 3107459 exec_runner.go:144] found /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem, removing ...
I0923 12:45:18.434164 3107459 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem
I0923 12:45:18.434200 3107459 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19688-2897765/.minikube/key.pem (1675 bytes)
I0923 12:45:18.434251 3107459 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem org=jenkins.no-preload-569169 san=[127.0.0.1 192.168.76.2 localhost minikube no-preload-569169]
I0923 12:45:19.135742 3107459 provision.go:177] copyRemoteCerts
I0923 12:45:19.135864 3107459 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0923 12:45:19.135925 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:19.153237 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:19.248464 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0923 12:45:19.276297 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0923 12:45:19.303809 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0923 12:45:19.329412 3107459 provision.go:87] duration metric: took 913.351722ms to configureAuth
I0923 12:45:19.329439 3107459 ubuntu.go:193] setting minikube options for container-runtime
I0923 12:45:19.329640 3107459 config.go:182] Loaded profile config "no-preload-569169": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0923 12:45:19.329652 3107459 machine.go:96] duration metric: took 4.417310366s to provisionDockerMachine
I0923 12:45:19.329659 3107459 start.go:293] postStartSetup for "no-preload-569169" (driver="docker")
I0923 12:45:19.329675 3107459 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0923 12:45:19.329730 3107459 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0923 12:45:19.329771 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:19.346346 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:19.441054 3107459 ssh_runner.go:195] Run: cat /etc/os-release
I0923 12:45:19.444575 3107459 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0923 12:45:19.444628 3107459 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0923 12:45:19.444639 3107459 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0923 12:45:19.444650 3107459 info.go:137] Remote host: Ubuntu 22.04.5 LTS
I0923 12:45:19.444661 3107459 filesync.go:126] Scanning /home/jenkins/minikube-integration/19688-2897765/.minikube/addons for local assets ...
I0923 12:45:19.444725 3107459 filesync.go:126] Scanning /home/jenkins/minikube-integration/19688-2897765/.minikube/files for local assets ...
I0923 12:45:19.444818 3107459 filesync.go:149] local asset: /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem -> 29031582.pem in /etc/ssl/certs
I0923 12:45:19.444931 3107459 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0923 12:45:19.453809 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem --> /etc/ssl/certs/29031582.pem (1708 bytes)
I0923 12:45:19.489984 3107459 start.go:296] duration metric: took 160.303636ms for postStartSetup
I0923 12:45:19.490079 3107459 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0923 12:45:19.490122 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:19.507184 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:19.604936 3107459 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0923 12:45:19.609751 3107459 fix.go:56] duration metric: took 5.131712204s for fixHost
I0923 12:45:19.609777 3107459 start.go:83] releasing machines lock for "no-preload-569169", held for 5.131765496s
I0923 12:45:19.609849 3107459 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-569169
I0923 12:45:19.627361 3107459 ssh_runner.go:195] Run: cat /version.json
I0923 12:45:19.627400 3107459 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0923 12:45:19.627414 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:19.627467 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:19.649101 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:19.656898 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:19.743341 3107459 ssh_runner.go:195] Run: systemctl --version
I0923 12:45:19.884818 3107459 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0923 12:45:19.889761 3107459 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0923 12:45:19.908494 3107459 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0923 12:45:19.908615 3107459 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0923 12:45:19.917562 3107459 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0923 12:45:19.917586 3107459 start.go:495] detecting cgroup driver to use...
I0923 12:45:19.917618 3107459 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0923 12:45:19.917667 3107459 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0923 12:45:19.933866 3107459 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0923 12:45:19.946857 3107459 docker.go:217] disabling cri-docker service (if available) ...
I0923 12:45:19.946930 3107459 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0923 12:45:19.960380 3107459 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0923 12:45:19.972393 3107459 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0923 12:45:20.079364 3107459 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0923 12:45:20.173106 3107459 docker.go:233] disabling docker service ...
I0923 12:45:20.173224 3107459 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0923 12:45:20.187194 3107459 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0923 12:45:20.201026 3107459 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0923 12:45:20.293491 3107459 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0923 12:45:20.395249 3107459 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0923 12:45:20.408100 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0923 12:45:20.425805 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0923 12:45:20.437494 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0923 12:45:20.449498 3107459 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0923 12:45:20.449575 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0923 12:45:20.460952 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0923 12:45:20.482169 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0923 12:45:20.497215 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0923 12:45:20.508225 3107459 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0923 12:45:20.519905 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0923 12:45:20.530342 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0923 12:45:20.541807 3107459 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0923 12:45:20.554060 3107459 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0923 12:45:20.566266 3107459 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0923 12:45:20.579855 3107459 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:45:20.663349 3107459 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0923 12:45:20.855043 3107459 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0923 12:45:20.855160 3107459 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0923 12:45:20.860111 3107459 start.go:563] Will wait 60s for crictl version
I0923 12:45:20.860220 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:45:20.864526 3107459 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0923 12:45:20.913080 3107459 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.22
RuntimeApiVersion: v1
I0923 12:45:20.913197 3107459 ssh_runner.go:195] Run: containerd --version
I0923 12:45:20.939693 3107459 ssh_runner.go:195] Run: containerd --version
I0923 12:45:20.966429 3107459 out.go:177] * Preparing Kubernetes v1.31.1 on containerd 1.7.22 ...
I0923 12:45:17.496501 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:19.993060 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:20.968118 3107459 cli_runner.go:164] Run: docker network inspect no-preload-569169 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0923 12:45:20.982821 3107459 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0923 12:45:20.986705 3107459 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0923 12:45:20.999822 3107459 kubeadm.go:883] updating cluster {Name:no-preload-569169 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-569169 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenk
ins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0923 12:45:20.999983 3107459 preload.go:131] Checking if preload exists for k8s version v1.31.1 and runtime containerd
I0923 12:45:21.000040 3107459 ssh_runner.go:195] Run: sudo crictl images --output json
I0923 12:45:21.054257 3107459 containerd.go:627] all images are preloaded for containerd runtime.
I0923 12:45:21.054286 3107459 cache_images.go:84] Images are preloaded, skipping loading
I0923 12:45:21.054295 3107459 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.31.1 containerd true true} ...
I0923 12:45:21.054414 3107459 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=no-preload-569169 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.31.1 ClusterName:no-preload-569169 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0923 12:45:21.054490 3107459 ssh_runner.go:195] Run: sudo crictl info
I0923 12:45:21.107715 3107459 cni.go:84] Creating CNI manager for ""
I0923 12:45:21.107800 3107459 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0923 12:45:21.107825 3107459 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0923 12:45:21.107882 3107459 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.31.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-569169 NodeName:no-preload-569169 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP
ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0923 12:45:21.108064 3107459 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "no-preload-569169"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0923 12:45:21.108188 3107459 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.1
I0923 12:45:21.118183 3107459 binaries.go:44] Found k8s binaries, skipping transfer
I0923 12:45:21.118255 3107459 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0923 12:45:21.127573 3107459 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (321 bytes)
I0923 12:45:21.146854 3107459 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0923 12:45:21.173164 3107459 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2171 bytes)
I0923 12:45:21.191833 3107459 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0923 12:45:21.195499 3107459 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0923 12:45:21.206471 3107459 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:45:21.299913 3107459 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0923 12:45:21.320773 3107459 certs.go:68] Setting up /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169 for IP: 192.168.76.2
I0923 12:45:21.320797 3107459 certs.go:194] generating shared ca certs ...
I0923 12:45:21.320813 3107459 certs.go:226] acquiring lock for ca certs: {Name:mk3307686e47e832a4d12b60b03ff3c8ff918f47 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:45:21.321016 3107459 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.key
I0923 12:45:21.321082 3107459 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.key
I0923 12:45:21.321096 3107459 certs.go:256] generating profile certs ...
I0923 12:45:21.321201 3107459 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/client.key
I0923 12:45:21.321314 3107459 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/apiserver.key.f20ff4c7
I0923 12:45:21.321383 3107459 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/proxy-client.key
I0923 12:45:21.321517 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158.pem (1338 bytes)
W0923 12:45:21.321570 3107459 certs.go:480] ignoring /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158_empty.pem, impossibly tiny 0 bytes
I0923 12:45:21.321583 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca-key.pem (1679 bytes)
I0923 12:45:21.321612 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/ca.pem (1078 bytes)
I0923 12:45:21.321666 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/cert.pem (1123 bytes)
I0923 12:45:21.321693 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/key.pem (1675 bytes)
I0923 12:45:21.321758 3107459 certs.go:484] found cert: /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem (1708 bytes)
I0923 12:45:21.322365 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0923 12:45:21.351987 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0923 12:45:21.380997 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0923 12:45:21.410261 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0923 12:45:21.446296 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0923 12:45:21.479764 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0923 12:45:21.535872 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0923 12:45:21.564884 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/profiles/no-preload-569169/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0923 12:45:21.595383 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/files/etc/ssl/certs/29031582.pem --> /usr/share/ca-certificates/29031582.pem (1708 bytes)
I0923 12:45:21.621833 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0923 12:45:21.650439 3107459 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19688-2897765/.minikube/certs/2903158.pem --> /usr/share/ca-certificates/2903158.pem (1338 bytes)
I0923 12:45:21.677384 3107459 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0923 12:45:21.696174 3107459 ssh_runner.go:195] Run: openssl version
I0923 12:45:21.703795 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/29031582.pem && ln -fs /usr/share/ca-certificates/29031582.pem /etc/ssl/certs/29031582.pem"
I0923 12:45:21.714292 3107459 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/29031582.pem
I0923 12:45:21.718508 3107459 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Sep 23 12:06 /usr/share/ca-certificates/29031582.pem
I0923 12:45:21.718616 3107459 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/29031582.pem
I0923 12:45:21.725809 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/29031582.pem /etc/ssl/certs/3ec20f2e.0"
I0923 12:45:21.735423 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0923 12:45:21.745372 3107459 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0923 12:45:21.748950 3107459 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Sep 23 11:36 /usr/share/ca-certificates/minikubeCA.pem
I0923 12:45:21.749018 3107459 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0923 12:45:21.756123 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0923 12:45:21.765318 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2903158.pem && ln -fs /usr/share/ca-certificates/2903158.pem /etc/ssl/certs/2903158.pem"
I0923 12:45:21.775529 3107459 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2903158.pem
I0923 12:45:21.779212 3107459 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Sep 23 12:06 /usr/share/ca-certificates/2903158.pem
I0923 12:45:21.779344 3107459 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2903158.pem
I0923 12:45:21.786846 3107459 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2903158.pem /etc/ssl/certs/51391683.0"
I0923 12:45:21.796253 3107459 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0923 12:45:21.799984 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0923 12:45:21.807162 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0923 12:45:21.814551 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0923 12:45:21.821875 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0923 12:45:21.829075 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0923 12:45:21.836389 3107459 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0923 12:45:21.843452 3107459 kubeadm.go:392] StartCluster: {Name:no-preload-569169 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.45-1726784731-19672@sha256:7f8c62ddb0100a5b958dd19c5b5478b8c7ef13da9a0a4d6c7d18f43544e0dbed Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.1 ClusterName:no-preload-569169 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0923 12:45:21.843550 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0923 12:45:21.843661 3107459 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0923 12:45:21.900356 3107459 cri.go:89] found id: "fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:45:21.900377 3107459 cri.go:89] found id: "03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:45:21.900381 3107459 cri.go:89] found id: "fdc6f11b4366cd9b3ba19119f22e95b4fd66c8790667d019db30fb972746b36a"
I0923 12:45:21.900414 3107459 cri.go:89] found id: "3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:45:21.900418 3107459 cri.go:89] found id: "5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:45:21.900423 3107459 cri.go:89] found id: "0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:45:21.900426 3107459 cri.go:89] found id: "96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:45:21.900430 3107459 cri.go:89] found id: "7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:45:21.900433 3107459 cri.go:89] found id: ""
I0923 12:45:21.900485 3107459 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0923 12:45:21.923727 3107459 cri.go:116] JSON = null
W0923 12:45:21.923777 3107459 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0923 12:45:21.923849 3107459 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0923 12:45:21.942299 3107459 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0923 12:45:21.942317 3107459 kubeadm.go:593] restartPrimaryControlPlane start ...
I0923 12:45:21.942371 3107459 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0923 12:45:21.954613 3107459 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0923 12:45:21.955208 3107459 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-569169" does not appear in /home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:45:21.955483 3107459 kubeconfig.go:62] /home/jenkins/minikube-integration/19688-2897765/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-569169" cluster setting kubeconfig missing "no-preload-569169" context setting]
I0923 12:45:21.955979 3107459 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19688-2897765/kubeconfig: {Name:mkc814324ebd7e6787446f1c0db099ab6daa7ff3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:45:21.957357 3107459 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0923 12:45:21.971747 3107459 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0923 12:45:21.971778 3107459 kubeadm.go:597] duration metric: took 29.45461ms to restartPrimaryControlPlane
I0923 12:45:21.971788 3107459 kubeadm.go:394] duration metric: took 128.354351ms to StartCluster
I0923 12:45:21.971803 3107459 settings.go:142] acquiring lock: {Name:mk4415211fc0f47c243959f36c7d2f9eeca37653 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:45:21.971858 3107459 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19688-2897765/kubeconfig
I0923 12:45:21.972786 3107459 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19688-2897765/kubeconfig: {Name:mkc814324ebd7e6787446f1c0db099ab6daa7ff3 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0923 12:45:21.973019 3107459 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.1 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0923 12:45:21.973337 3107459 config.go:182] Loaded profile config "no-preload-569169": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.1
I0923 12:45:21.973401 3107459 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0923 12:45:21.973520 3107459 addons.go:69] Setting storage-provisioner=true in profile "no-preload-569169"
I0923 12:45:21.973538 3107459 addons.go:234] Setting addon storage-provisioner=true in "no-preload-569169"
W0923 12:45:21.973549 3107459 addons.go:243] addon storage-provisioner should already be in state true
I0923 12:45:21.973544 3107459 addons.go:69] Setting default-storageclass=true in profile "no-preload-569169"
I0923 12:45:21.973614 3107459 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-569169"
I0923 12:45:21.973571 3107459 host.go:66] Checking if "no-preload-569169" exists ...
I0923 12:45:21.974057 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:21.974169 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:21.973576 3107459 addons.go:69] Setting metrics-server=true in profile "no-preload-569169"
I0923 12:45:21.974805 3107459 addons.go:234] Setting addon metrics-server=true in "no-preload-569169"
W0923 12:45:21.974822 3107459 addons.go:243] addon metrics-server should already be in state true
I0923 12:45:21.974846 3107459 host.go:66] Checking if "no-preload-569169" exists ...
I0923 12:45:21.975273 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:21.973580 3107459 addons.go:69] Setting dashboard=true in profile "no-preload-569169"
I0923 12:45:21.978113 3107459 addons.go:234] Setting addon dashboard=true in "no-preload-569169"
W0923 12:45:21.978126 3107459 addons.go:243] addon dashboard should already be in state true
I0923 12:45:21.978163 3107459 host.go:66] Checking if "no-preload-569169" exists ...
I0923 12:45:21.979132 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:21.982101 3107459 out.go:177] * Verifying Kubernetes components...
I0923 12:45:21.990325 3107459 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0923 12:45:22.039148 3107459 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0923 12:45:22.041094 3107459 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0923 12:45:22.041120 3107459 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0923 12:45:22.041194 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:22.050937 3107459 addons.go:234] Setting addon default-storageclass=true in "no-preload-569169"
W0923 12:45:22.050959 3107459 addons.go:243] addon default-storageclass should already be in state true
I0923 12:45:22.050985 3107459 host.go:66] Checking if "no-preload-569169" exists ...
I0923 12:45:22.052443 3107459 cli_runner.go:164] Run: docker container inspect no-preload-569169 --format={{.State.Status}}
I0923 12:45:22.057874 3107459 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0923 12:45:22.060826 3107459 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:45:22.060847 3107459 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0923 12:45:22.060917 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:22.071174 3107459 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0923 12:45:22.076481 3107459 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0923 12:45:22.081186 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0923 12:45:22.081225 3107459 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0923 12:45:22.081308 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:22.125948 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:22.136772 3107459 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0923 12:45:22.136794 3107459 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0923 12:45:22.136884 3107459 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-569169
I0923 12:45:22.137133 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:22.166748 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:22.175175 3107459 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:42092 SSHKeyPath:/home/jenkins/minikube-integration/19688-2897765/.minikube/machines/no-preload-569169/id_rsa Username:docker}
I0923 12:45:22.238144 3107459 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0923 12:45:22.338501 3107459 node_ready.go:35] waiting up to 6m0s for node "no-preload-569169" to be "Ready" ...
I0923 12:45:22.409787 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0923 12:45:22.409813 3107459 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0923 12:45:22.439002 3107459 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0923 12:45:22.457318 3107459 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0923 12:45:22.457343 3107459 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0923 12:45:22.486312 3107459 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0923 12:45:22.486338 3107459 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0923 12:45:22.522746 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0923 12:45:22.522770 3107459 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0923 12:45:22.537903 3107459 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0923 12:45:22.614247 3107459 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0923 12:45:22.614271 3107459 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0923 12:45:22.644539 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0923 12:45:22.644566 3107459 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0923 12:45:22.822272 3107459 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0923 12:45:22.950641 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0923 12:45:22.950668 3107459 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0923 12:45:23.050324 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0923 12:45:23.050353 3107459 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0923 12:45:23.093533 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0923 12:45:23.093560 3107459 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0923 12:45:23.142225 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0923 12:45:23.142251 3107459 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0923 12:45:23.183302 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0923 12:45:23.183328 3107459 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0923 12:45:23.227243 3107459 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:45:23.227270 3107459 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0923 12:45:23.294294 3107459 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0923 12:45:22.015003 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:24.539397 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:26.972744 3107459 node_ready.go:49] node "no-preload-569169" has status "Ready":"True"
I0923 12:45:26.972768 3107459 node_ready.go:38] duration metric: took 4.634234049s for node "no-preload-569169" to be "Ready" ...
I0923 12:45:26.972777 3107459 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0923 12:45:27.085464 3107459 pod_ready.go:79] waiting up to 6m0s for pod "coredns-7c65d6cfc9-r5klx" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.114218 3107459 pod_ready.go:93] pod "coredns-7c65d6cfc9-r5klx" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.114245 3107459 pod_ready.go:82] duration metric: took 27.731484ms for pod "coredns-7c65d6cfc9-r5klx" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.114256 3107459 pod_ready.go:79] waiting up to 6m0s for pod "etcd-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.127497 3107459 pod_ready.go:93] pod "etcd-no-preload-569169" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.127524 3107459 pod_ready.go:82] duration metric: took 13.259935ms for pod "etcd-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.127540 3107459 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.140415 3107459 pod_ready.go:93] pod "kube-apiserver-no-preload-569169" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.140492 3107459 pod_ready.go:82] duration metric: took 12.942857ms for pod "kube-apiserver-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.140529 3107459 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.169025 3107459 pod_ready.go:93] pod "kube-controller-manager-no-preload-569169" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.169051 3107459 pod_ready.go:82] duration metric: took 28.500768ms for pod "kube-controller-manager-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.169062 3107459 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-bcttj" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.193813 3107459 pod_ready.go:93] pod "kube-proxy-bcttj" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.193840 3107459 pod_ready.go:82] duration metric: took 24.770652ms for pod "kube-proxy-bcttj" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.193852 3107459 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.578006 3107459 pod_ready.go:93] pod "kube-scheduler-no-preload-569169" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:27.578098 3107459 pod_ready.go:82] duration metric: took 384.236462ms for pod "kube-scheduler-no-preload-569169" in "kube-system" namespace to be "Ready" ...
I0923 12:45:27.578125 3107459 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace to be "Ready" ...
I0923 12:45:29.585927 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:30.141315 3107459 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (7.702260218s)
I0923 12:45:30.141393 3107459 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (7.603467052s)
W0923 12:45:30.141417 3107459 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0923 12:45:30.141444 3107459 retry.go:31] will retry after 300.93999ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0923 12:45:30.141519 3107459 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.319217775s)
I0923 12:45:30.141537 3107459 addons.go:475] Verifying addon metrics-server=true in "no-preload-569169"
I0923 12:45:30.263219 3107459 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (6.968877355s)
I0923 12:45:30.265331 3107459 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-569169 addons enable metrics-server
I0923 12:45:30.443563 3107459 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0923 12:45:30.647981 3107459 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0923 12:45:26.994612 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:29.497184 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:30.649743 3107459 addons.go:510] duration metric: took 8.676330766s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0923 12:45:32.084527 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:34.100914 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:31.501408 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:33.994220 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:36.584318 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:38.585239 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:36.493434 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:38.494618 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:40.993697 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:41.085910 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:43.584914 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:42.993796 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:45.494023 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:46.084222 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:48.084874 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:47.993809 3102485 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:49.493816 3102485 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:49.493848 3102485 pod_ready.go:82] duration metric: took 1m20.006846553s for pod "kube-controller-manager-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.493861 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-9m2fb" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.504701 3102485 pod_ready.go:93] pod "kube-proxy-9m2fb" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:49.504776 3102485 pod_ready.go:82] duration metric: took 10.905739ms for pod "kube-proxy-9m2fb" in "kube-system" namespace to be "Ready" ...
I0923 12:45:49.504803 3102485 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:50.584540 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:52.585422 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:51.512244 3102485 pod_ready.go:103] pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:52.510949 3102485 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace has status "Ready":"True"
I0923 12:45:52.510976 3102485 pod_ready.go:82] duration metric: took 3.006151928s for pod "kube-scheduler-old-k8s-version-605045" in "kube-system" namespace to be "Ready" ...
I0923 12:45:52.510987 3102485 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace to be "Ready" ...
I0923 12:45:54.517408 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:55.084943 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:57.085244 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:56.517938 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:59.018038 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:45:59.584727 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:01.584765 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:03.585809 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:01.521178 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:04.017169 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:06.018012 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:06.084366 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:08.085103 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:08.020342 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:10.050466 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:10.085697 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:12.584597 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:12.519575 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:15.028066 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:14.585240 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:17.084818 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:19.084978 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:17.516764 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:19.517985 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:21.584916 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:23.585393 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:21.527598 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:24.018262 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:26.084069 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:28.084871 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:26.516738 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:28.517429 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:30.517488 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:30.085891 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:32.584266 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:32.517691 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:34.518155 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:35.084662 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:37.584826 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:37.019755 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:39.517509 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:39.584893 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:42.085654 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:41.517906 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:44.018200 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:44.583918 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:46.584188 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:48.585500 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:46.518620 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:49.017532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:51.085557 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:53.584515 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:51.517271 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:53.518286 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:55.518511 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:55.585029 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:57.590044 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:46:58.018393 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:00.040850 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:00.097324 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:02.584655 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:02.519588 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:05.017851 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:05.085283 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:07.584458 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:07.517609 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:10.120171 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:09.584697 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:11.585432 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:14.084057 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:12.516830 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:14.518809 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:16.084844 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:18.086244 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:17.017796 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:19.517358 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:20.584059 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:23.084440 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:21.521109 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:24.017997 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:25.085483 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:27.585336 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:26.517391 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:28.518031 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:31.017891 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:30.086596 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:32.584286 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:33.022935 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:35.025532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:34.584798 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:37.085034 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:37.517416 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:39.518031 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:39.584643 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:41.584767 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:43.584813 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:41.527952 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:44.017386 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:46.017790 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:45.584955 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:48.085594 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:48.018509 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:50.019200 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:50.085838 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:52.585736 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:52.517046 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:54.519238 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:55.084374 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:57.084529 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:59.084598 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:57.017644 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:47:59.018465 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:01.585010 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:04.084378 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:01.517860 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:04.020306 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:06.584197 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:08.584797 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:06.518007 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:08.518217 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:10.518374 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:11.084766 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:13.583962 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:13.017127 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:15.025052 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:16.084116 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:18.085614 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:17.518912 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:19.519575 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:20.584085 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:22.584160 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:22.017866 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:24.519753 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:25.084739 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:27.591949 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:27.018096 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:29.517084 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:30.095520 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:32.583821 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:31.518247 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:34.017560 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:36.018135 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:34.584868 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:37.085221 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:39.085432 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:38.018563 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:40.033260 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:41.584139 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:43.584493 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:42.516878 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:44.518028 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:46.084462 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:48.084648 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:47.017558 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:49.017887 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:50.084902 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:52.584596 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:51.104384 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:53.518375 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:56.017073 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:54.585350 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:57.084590 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:58.018046 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:00.051557 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:48:59.584184 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:01.584554 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:03.584783 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:02.518428 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:05.018309 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:06.086357 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:08.586028 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:07.517421 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:09.517540 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:11.084457 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:13.584204 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:12.017948 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:14.518324 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:15.584669 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:18.085144 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:16.518725 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:19.017071 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:21.018952 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:20.085290 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:22.584914 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:23.517806 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:26.017683 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:25.085050 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:27.585265 3107459 pod_ready.go:103] pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:27.585313 3107459 pod_ready.go:82] duration metric: took 4m0.007165977s for pod "metrics-server-6867b74b74-5twtp" in "kube-system" namespace to be "Ready" ...
E0923 12:49:27.585324 3107459 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0923 12:49:27.585333 3107459 pod_ready.go:39] duration metric: took 4m0.612545092s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0923 12:49:27.585360 3107459 api_server.go:52] waiting for apiserver process to appear ...
I0923 12:49:27.585395 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:49:27.585457 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:49:27.631650 3107459 cri.go:89] found id: "9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:27.631717 3107459 cri.go:89] found id: "7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:27.631729 3107459 cri.go:89] found id: ""
I0923 12:49:27.631737 3107459 logs.go:276] 2 containers: [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4]
I0923 12:49:27.631795 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.636334 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.640122 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:49:27.640250 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:49:27.680878 3107459 cri.go:89] found id: "177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:27.680905 3107459 cri.go:89] found id: "0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:27.680910 3107459 cri.go:89] found id: ""
I0923 12:49:27.680917 3107459 logs.go:276] 2 containers: [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a]
I0923 12:49:27.680997 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.684773 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.689238 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:49:27.689315 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:49:27.731159 3107459 cri.go:89] found id: "116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:27.731232 3107459 cri.go:89] found id: "fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:27.731251 3107459 cri.go:89] found id: ""
I0923 12:49:27.731274 3107459 logs.go:276] 2 containers: [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a]
I0923 12:49:27.731412 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.735411 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.738996 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:49:27.739078 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:49:27.784507 3107459 cri.go:89] found id: "8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:27.784603 3107459 cri.go:89] found id: "5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:27.784636 3107459 cri.go:89] found id: ""
I0923 12:49:27.784679 3107459 logs.go:276] 2 containers: [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c]
I0923 12:49:27.784787 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.789657 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.793738 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:49:27.793838 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:49:27.836366 3107459 cri.go:89] found id: "eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:27.836392 3107459 cri.go:89] found id: "3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:27.836409 3107459 cri.go:89] found id: ""
I0923 12:49:27.836417 3107459 logs.go:276] 2 containers: [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c]
I0923 12:49:27.836476 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.840524 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.848358 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:49:27.848461 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:49:27.893710 3107459 cri.go:89] found id: "267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:27.893736 3107459 cri.go:89] found id: "96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:27.893741 3107459 cri.go:89] found id: ""
I0923 12:49:27.893749 3107459 logs.go:276] 2 containers: [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481]
I0923 12:49:27.893863 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.897696 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.901525 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:49:27.901599 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:49:27.942029 3107459 cri.go:89] found id: "a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:27.942049 3107459 cri.go:89] found id: "03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:27.942054 3107459 cri.go:89] found id: ""
I0923 12:49:27.942061 3107459 logs.go:276] 2 containers: [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45]
I0923 12:49:27.942120 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.946050 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.949585 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:49:27.949666 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:49:27.991564 3107459 cri.go:89] found id: "64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:27.991588 3107459 cri.go:89] found id: "3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:27.991593 3107459 cri.go:89] found id: ""
I0923 12:49:27.991601 3107459 logs.go:276] 2 containers: [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef]
I0923 12:49:27.991658 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.995383 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:27.998764 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:49:27.998874 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:49:28.050024 3107459 cri.go:89] found id: "c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:28.050049 3107459 cri.go:89] found id: ""
I0923 12:49:28.050057 3107459 logs.go:276] 1 containers: [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776]
I0923 12:49:28.050115 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:28.054040 3107459 logs.go:123] Gathering logs for coredns [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6] ...
I0923 12:49:28.054079 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:28.110088 3107459 logs.go:123] Gathering logs for kube-scheduler [5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c] ...
I0923 12:49:28.110173 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:28.160124 3107459 logs.go:123] Gathering logs for storage-provisioner [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6] ...
I0923 12:49:28.160155 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:28.198211 3107459 logs.go:123] Gathering logs for container status ...
I0923 12:49:28.198244 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:49:28.257585 3107459 logs.go:123] Gathering logs for kube-scheduler [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1] ...
I0923 12:49:28.257614 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:28.296724 3107459 logs.go:123] Gathering logs for kube-proxy [3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c] ...
I0923 12:49:28.296752 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:28.342932 3107459 logs.go:123] Gathering logs for storage-provisioner [3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef] ...
I0923 12:49:28.342961 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:28.387395 3107459 logs.go:123] Gathering logs for kubernetes-dashboard [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776] ...
I0923 12:49:28.387425 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:28.434998 3107459 logs.go:123] Gathering logs for kubelet ...
I0923 12:49:28.435025 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:49:28.484876 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:28.485133 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:28.516944 3107459 logs.go:123] Gathering logs for describe nodes ...
I0923 12:49:28.516981 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:49:28.668444 3107459 logs.go:123] Gathering logs for kube-apiserver [7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4] ...
I0923 12:49:28.668475 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:28.725090 3107459 logs.go:123] Gathering logs for coredns [fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a] ...
I0923 12:49:28.725129 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:28.766347 3107459 logs.go:123] Gathering logs for dmesg ...
I0923 12:49:28.766417 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:49:28.784993 3107459 logs.go:123] Gathering logs for kube-apiserver [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437] ...
I0923 12:49:28.785070 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:28.837937 3107459 logs.go:123] Gathering logs for kube-controller-manager [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd] ...
I0923 12:49:28.837968 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:28.906996 3107459 logs.go:123] Gathering logs for containerd ...
I0923 12:49:28.907032 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:49:28.975043 3107459 logs.go:123] Gathering logs for kindnet [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2] ...
I0923 12:49:28.975083 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:29.028562 3107459 logs.go:123] Gathering logs for kindnet [03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45] ...
I0923 12:49:29.028598 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:29.070915 3107459 logs.go:123] Gathering logs for etcd [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982] ...
I0923 12:49:29.070947 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:29.120434 3107459 logs.go:123] Gathering logs for etcd [0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a] ...
I0923 12:49:29.120467 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:29.175630 3107459 logs.go:123] Gathering logs for kube-proxy [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1] ...
I0923 12:49:29.175662 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:29.222468 3107459 logs.go:123] Gathering logs for kube-controller-manager [96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481] ...
I0923 12:49:29.222500 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:28.018157 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:30.030456 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:29.282026 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:29.282058 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:49:29.282127 3107459 out.go:270] X Problems detected in kubelet:
W0923 12:49:29.282144 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:29.282155 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:29.282172 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:29.282179 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:49:32.517329 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:34.517424 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:36.517606 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:39.017532 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:39.284016 3107459 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0923 12:49:39.296398 3107459 api_server.go:72] duration metric: took 4m17.32334127s to wait for apiserver process to appear ...
I0923 12:49:39.296424 3107459 api_server.go:88] waiting for apiserver healthz status ...
I0923 12:49:39.296461 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:49:39.296520 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:49:39.348939 3107459 cri.go:89] found id: "9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:39.348960 3107459 cri.go:89] found id: "7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:39.348965 3107459 cri.go:89] found id: ""
I0923 12:49:39.348973 3107459 logs.go:276] 2 containers: [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4]
I0923 12:49:39.349034 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.352826 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.356518 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:49:39.356606 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:49:39.397435 3107459 cri.go:89] found id: "177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:39.397470 3107459 cri.go:89] found id: "0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:39.397476 3107459 cri.go:89] found id: ""
I0923 12:49:39.397483 3107459 logs.go:276] 2 containers: [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a]
I0923 12:49:39.397542 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.401464 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.405171 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:49:39.405245 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:49:39.443075 3107459 cri.go:89] found id: "116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:39.443095 3107459 cri.go:89] found id: "fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:39.443101 3107459 cri.go:89] found id: ""
I0923 12:49:39.443107 3107459 logs.go:276] 2 containers: [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a]
I0923 12:49:39.443181 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.447015 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.451533 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:49:39.451656 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:49:39.493693 3107459 cri.go:89] found id: "8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:39.493717 3107459 cri.go:89] found id: "5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:39.493723 3107459 cri.go:89] found id: ""
I0923 12:49:39.493730 3107459 logs.go:276] 2 containers: [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c]
I0923 12:49:39.493790 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.497518 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.501418 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:49:39.501495 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:49:39.541972 3107459 cri.go:89] found id: "eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:39.542033 3107459 cri.go:89] found id: "3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:39.542065 3107459 cri.go:89] found id: ""
I0923 12:49:39.542092 3107459 logs.go:276] 2 containers: [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c]
I0923 12:49:39.542179 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.546139 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.549599 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:49:39.549677 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:49:39.596247 3107459 cri.go:89] found id: "267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:39.596325 3107459 cri.go:89] found id: "96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:39.596358 3107459 cri.go:89] found id: ""
I0923 12:49:39.596379 3107459 logs.go:276] 2 containers: [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481]
I0923 12:49:39.596465 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.602259 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.606222 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:49:39.606301 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:49:39.646231 3107459 cri.go:89] found id: "a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:39.646255 3107459 cri.go:89] found id: "03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:39.646260 3107459 cri.go:89] found id: ""
I0923 12:49:39.646267 3107459 logs.go:276] 2 containers: [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45]
I0923 12:49:39.646325 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.650273 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.655071 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:49:39.655144 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:49:39.728569 3107459 cri.go:89] found id: "c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:39.728632 3107459 cri.go:89] found id: ""
I0923 12:49:39.728665 3107459 logs.go:276] 1 containers: [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776]
I0923 12:49:39.728759 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.734052 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:49:39.734134 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:49:39.775492 3107459 cri.go:89] found id: "64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:39.775517 3107459 cri.go:89] found id: "3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:39.775523 3107459 cri.go:89] found id: ""
I0923 12:49:39.775530 3107459 logs.go:276] 2 containers: [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef]
I0923 12:49:39.775587 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.779214 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:39.783033 3107459 logs.go:123] Gathering logs for kube-apiserver [7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4] ...
I0923 12:49:39.783057 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:39.842820 3107459 logs.go:123] Gathering logs for kindnet [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2] ...
I0923 12:49:39.843045 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:39.898383 3107459 logs.go:123] Gathering logs for storage-provisioner [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6] ...
I0923 12:49:39.898413 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:39.936886 3107459 logs.go:123] Gathering logs for coredns [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6] ...
I0923 12:49:39.936918 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:39.978765 3107459 logs.go:123] Gathering logs for kube-controller-manager [96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481] ...
I0923 12:49:39.978795 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:40.068541 3107459 logs.go:123] Gathering logs for kubelet ...
I0923 12:49:40.068581 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:49:40.125340 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:40.125637 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:40.157780 3107459 logs.go:123] Gathering logs for kube-apiserver [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437] ...
I0923 12:49:40.157821 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:40.231842 3107459 logs.go:123] Gathering logs for etcd [0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a] ...
I0923 12:49:40.231870 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:40.285582 3107459 logs.go:123] Gathering logs for kube-proxy [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1] ...
I0923 12:49:40.285622 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:40.326206 3107459 logs.go:123] Gathering logs for dmesg ...
I0923 12:49:40.326232 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:49:40.342867 3107459 logs.go:123] Gathering logs for describe nodes ...
I0923 12:49:40.342897 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:49:40.472567 3107459 logs.go:123] Gathering logs for coredns [fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a] ...
I0923 12:49:40.472600 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:40.520756 3107459 logs.go:123] Gathering logs for kube-proxy [3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c] ...
I0923 12:49:40.520786 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:40.569001 3107459 logs.go:123] Gathering logs for kube-controller-manager [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd] ...
I0923 12:49:40.569069 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:40.632798 3107459 logs.go:123] Gathering logs for kindnet [03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45] ...
I0923 12:49:40.632838 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:40.678473 3107459 logs.go:123] Gathering logs for kubernetes-dashboard [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776] ...
I0923 12:49:40.678503 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:40.726132 3107459 logs.go:123] Gathering logs for storage-provisioner [3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef] ...
I0923 12:49:40.726160 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:40.769230 3107459 logs.go:123] Gathering logs for etcd [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982] ...
I0923 12:49:40.769262 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:40.819328 3107459 logs.go:123] Gathering logs for kube-scheduler [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1] ...
I0923 12:49:40.819451 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:40.873664 3107459 logs.go:123] Gathering logs for kube-scheduler [5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c] ...
I0923 12:49:40.873695 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:40.922111 3107459 logs.go:123] Gathering logs for containerd ...
I0923 12:49:40.922151 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:49:40.985502 3107459 logs.go:123] Gathering logs for container status ...
I0923 12:49:40.985537 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:49:41.036178 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:41.036206 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:49:41.036289 3107459 out.go:270] X Problems detected in kubelet:
W0923 12:49:41.036306 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:41.036314 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:41.036350 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:41.036368 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:49:41.516456 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:43.522639 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:46.019113 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:48.022948 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:50.517786 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:51.037685 3107459 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0923 12:49:51.046420 3107459 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0923 12:49:51.047587 3107459 api_server.go:141] control plane version: v1.31.1
I0923 12:49:51.047617 3107459 api_server.go:131] duration metric: took 11.751185788s to wait for apiserver health ...
I0923 12:49:51.047627 3107459 system_pods.go:43] waiting for kube-system pods to appear ...
I0923 12:49:51.047649 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:49:51.047723 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:49:51.088127 3107459 cri.go:89] found id: "9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:51.088159 3107459 cri.go:89] found id: "7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:51.088169 3107459 cri.go:89] found id: ""
I0923 12:49:51.088180 3107459 logs.go:276] 2 containers: [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4]
I0923 12:49:51.088249 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.093661 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.097584 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:49:51.097691 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:49:51.137926 3107459 cri.go:89] found id: "177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:51.137948 3107459 cri.go:89] found id: "0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:51.137953 3107459 cri.go:89] found id: ""
I0923 12:49:51.137961 3107459 logs.go:276] 2 containers: [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a]
I0923 12:49:51.138044 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.142029 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.145999 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:49:51.146078 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:49:51.194598 3107459 cri.go:89] found id: "116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:51.194619 3107459 cri.go:89] found id: "fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:51.194624 3107459 cri.go:89] found id: ""
I0923 12:49:51.194631 3107459 logs.go:276] 2 containers: [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a]
I0923 12:49:51.194689 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.198489 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.202024 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:49:51.202095 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:49:51.241619 3107459 cri.go:89] found id: "8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:51.241640 3107459 cri.go:89] found id: "5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:51.241646 3107459 cri.go:89] found id: ""
I0923 12:49:51.241653 3107459 logs.go:276] 2 containers: [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c]
I0923 12:49:51.241709 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.245706 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.249324 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:49:51.249435 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:49:51.289211 3107459 cri.go:89] found id: "eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:51.289243 3107459 cri.go:89] found id: "3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:51.289248 3107459 cri.go:89] found id: ""
I0923 12:49:51.289256 3107459 logs.go:276] 2 containers: [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c]
I0923 12:49:51.289312 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.293195 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.296613 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:49:51.296685 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:49:51.333977 3107459 cri.go:89] found id: "267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:51.334000 3107459 cri.go:89] found id: "96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:51.334011 3107459 cri.go:89] found id: ""
I0923 12:49:51.334018 3107459 logs.go:276] 2 containers: [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481]
I0923 12:49:51.334103 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.338027 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.342155 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:49:51.342230 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:49:51.382904 3107459 cri.go:89] found id: "a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:51.382927 3107459 cri.go:89] found id: "03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:51.382932 3107459 cri.go:89] found id: ""
I0923 12:49:51.382939 3107459 logs.go:276] 2 containers: [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45]
I0923 12:49:51.382999 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.386593 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.390052 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:49:51.390120 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:49:51.427871 3107459 cri.go:89] found id: "c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:51.427894 3107459 cri.go:89] found id: ""
I0923 12:49:51.427901 3107459 logs.go:276] 1 containers: [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776]
I0923 12:49:51.427960 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.431775 3107459 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:49:51.431851 3107459 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:49:51.493065 3107459 cri.go:89] found id: "64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:51.493100 3107459 cri.go:89] found id: "3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:51.493106 3107459 cri.go:89] found id: ""
I0923 12:49:51.493114 3107459 logs.go:276] 2 containers: [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef]
I0923 12:49:51.493169 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.496969 3107459 ssh_runner.go:195] Run: which crictl
I0923 12:49:51.500546 3107459 logs.go:123] Gathering logs for coredns [fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a] ...
I0923 12:49:51.500572 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fc42fc8d62cf1e2651d8ec16f744c88d9cc4b1bc29d3b60b39cd11f7143eb57a"
I0923 12:49:51.542167 3107459 logs.go:123] Gathering logs for kube-proxy [3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c] ...
I0923 12:49:51.542196 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3f9473f71c752b78182e203393ef4108e71b02e2a60351b2a96b5466cabe3b7c"
I0923 12:49:51.590390 3107459 logs.go:123] Gathering logs for kube-controller-manager [267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd] ...
I0923 12:49:51.590425 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 267e29d3bc50392009bbb0fcb8347e0d17b44537150adb1804a38eb2164909cd"
I0923 12:49:51.681232 3107459 logs.go:123] Gathering logs for kube-controller-manager [96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481] ...
I0923 12:49:51.681267 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 96ed3a9bd428b879be50768c0168d22bfda361abc363afe11655ff1c39325481"
I0923 12:49:51.766915 3107459 logs.go:123] Gathering logs for kindnet [a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2] ...
I0923 12:49:51.766953 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a2ef30766d6e29d4113e8052b1409c20fd911dd9a60f8ffe5efea8b6e116fcc2"
I0923 12:49:51.821438 3107459 logs.go:123] Gathering logs for storage-provisioner [64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6] ...
I0923 12:49:51.821528 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 64d9255be7c627de8ed6da81ca76b6f29de90a0cadaf130c0b36763d33a668f6"
I0923 12:49:51.885329 3107459 logs.go:123] Gathering logs for kube-apiserver [7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4] ...
I0923 12:49:51.885362 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7e839ac9dd4b4097f7328079de3415ae795f8651729dc528e954bce551d9cfa4"
I0923 12:49:51.962908 3107459 logs.go:123] Gathering logs for etcd [0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a] ...
I0923 12:49:51.962997 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0334d040b928229782ff2ee39db6d6f51d609598bcb2d3f2605a9d7f8f48213a"
I0923 12:49:52.023016 3107459 logs.go:123] Gathering logs for kube-scheduler [8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1] ...
I0923 12:49:52.023048 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8a987e5e7c638c416fbb62da6f6ebcf4fc81c638227d26b48306b7121da05cb1"
I0923 12:49:52.069963 3107459 logs.go:123] Gathering logs for storage-provisioner [3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef] ...
I0923 12:49:52.069990 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3a75d4eacfb3590fb86f5174b32c74fa5562eccfafe32efb6738f0007cc221ef"
I0923 12:49:52.109128 3107459 logs.go:123] Gathering logs for container status ...
I0923 12:49:52.109157 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:49:52.158327 3107459 logs.go:123] Gathering logs for kube-apiserver [9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437] ...
I0923 12:49:52.158356 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9789f1a1e638a4fbff99a542969d04a8b18fabaa93ab79223416bd07e0bb5437"
I0923 12:49:52.233782 3107459 logs.go:123] Gathering logs for etcd [177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982] ...
I0923 12:49:52.233816 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 177fa3486596f97ba5a45b458967a375a7eb68ae063bffb298e224632974d982"
I0923 12:49:52.287244 3107459 logs.go:123] Gathering logs for kube-scheduler [5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c] ...
I0923 12:49:52.287273 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5cb2c00337000bb7cd8bdff89255363283366b9b3f8117d3f867c57d1f6b384c"
I0923 12:49:52.340846 3107459 logs.go:123] Gathering logs for kube-proxy [eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1] ...
I0923 12:49:52.340883 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 eb5d52b20c6a7e29036d1bc211061b8ffa59d484083ad11f61fd3fee248047f1"
I0923 12:49:52.383408 3107459 logs.go:123] Gathering logs for kubernetes-dashboard [c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776] ...
I0923 12:49:52.383433 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c08bad2a807ce5ec7fef9b7e5a64f982639a783e14ab37c238802b6506881776"
I0923 12:49:52.420433 3107459 logs.go:123] Gathering logs for kubelet ...
I0923 12:49:52.420464 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:49:52.468033 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:52.468287 3107459 logs.go:138] Found kubelet problem: Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:52.501087 3107459 logs.go:123] Gathering logs for dmesg ...
I0923 12:49:52.501131 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:49:52.524116 3107459 logs.go:123] Gathering logs for describe nodes ...
I0923 12:49:52.524147 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:49:52.678387 3107459 logs.go:123] Gathering logs for coredns [116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6] ...
I0923 12:49:52.678539 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 116e6961e023a6bf08dc653125f9d94f9a87b445631be992c49c815fd25409f6"
I0923 12:49:52.754890 3107459 logs.go:123] Gathering logs for kindnet [03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45] ...
I0923 12:49:52.754919 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 03be0861b3b41a85dc2bafc39df39e7351691a862ae53f8f2cf62c860801df45"
I0923 12:49:52.834155 3107459 logs.go:123] Gathering logs for containerd ...
I0923 12:49:52.834183 3107459 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:49:52.914569 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:52.914603 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:49:52.915416 3107459 out.go:270] X Problems detected in kubelet:
W0923 12:49:52.915478 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: W0923 12:45:31.106205 661 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-569169" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-569169' and this object
W0923 12:49:52.915531 3107459 out.go:270] Sep 23 12:45:31 no-preload-569169 kubelet[661]: E0923 12:45:31.106723 661 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-569169\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-569169' and this object" logger="UnhandledError"
I0923 12:49:52.915569 3107459 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:52.915603 3107459 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:49:52.520527 3102485 pod_ready.go:103] pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace has status "Ready":"False"
I0923 12:49:52.520556 3102485 pod_ready.go:82] duration metric: took 4m0.009561565s for pod "metrics-server-9975d5f86-kf958" in "kube-system" namespace to be "Ready" ...
E0923 12:49:52.520568 3102485 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0923 12:49:52.520575 3102485 pod_ready.go:39] duration metric: took 5m23.27838172s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0923 12:49:52.520590 3102485 api_server.go:52] waiting for apiserver process to appear ...
I0923 12:49:52.520620 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:49:52.520680 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:49:52.582116 3102485 cri.go:89] found id: "b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:49:52.582139 3102485 cri.go:89] found id: "2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:49:52.582143 3102485 cri.go:89] found id: ""
I0923 12:49:52.582150 3102485 logs.go:276] 2 containers: [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85]
I0923 12:49:52.582208 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.586211 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.590182 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:49:52.590251 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:49:52.643577 3102485 cri.go:89] found id: "ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:49:52.643596 3102485 cri.go:89] found id: "1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:49:52.643601 3102485 cri.go:89] found id: ""
I0923 12:49:52.643608 3102485 logs.go:276] 2 containers: [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2]
I0923 12:49:52.643669 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.649826 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.653847 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:49:52.653956 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:49:52.723423 3102485 cri.go:89] found id: "0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:49:52.723449 3102485 cri.go:89] found id: "8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:49:52.723465 3102485 cri.go:89] found id: ""
I0923 12:49:52.723473 3102485 logs.go:276] 2 containers: [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993]
I0923 12:49:52.723539 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.728406 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.739893 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:49:52.739971 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:49:52.794862 3102485 cri.go:89] found id: "df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:49:52.794881 3102485 cri.go:89] found id: "2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:49:52.794886 3102485 cri.go:89] found id: ""
I0923 12:49:52.794893 3102485 logs.go:276] 2 containers: [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef]
I0923 12:49:52.794947 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.798877 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.802564 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:49:52.802683 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:49:52.882169 3102485 cri.go:89] found id: "3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:49:52.882242 3102485 cri.go:89] found id: "c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:49:52.882262 3102485 cri.go:89] found id: ""
I0923 12:49:52.882286 3102485 logs.go:276] 2 containers: [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8]
I0923 12:49:52.882372 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.886656 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.890389 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:49:52.890507 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:49:52.943880 3102485 cri.go:89] found id: "37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:49:52.943913 3102485 cri.go:89] found id: "7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:49:52.943918 3102485 cri.go:89] found id: ""
I0923 12:49:52.943926 3102485 logs.go:276] 2 containers: [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af]
I0923 12:49:52.943988 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.947650 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.951076 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:49:52.951161 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:49:52.988808 3102485 cri.go:89] found id: "a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:49:52.988830 3102485 cri.go:89] found id: "7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:49:52.988835 3102485 cri.go:89] found id: ""
I0923 12:49:52.988848 3102485 logs.go:276] 2 containers: [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f]
I0923 12:49:52.988909 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.992697 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:52.996136 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:49:52.996206 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:49:53.042311 3102485 cri.go:89] found id: "7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:49:53.042336 3102485 cri.go:89] found id: ""
I0923 12:49:53.042345 3102485 logs.go:276] 1 containers: [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a]
I0923 12:49:53.042421 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.046105 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:49:53.046208 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:49:53.088283 3102485 cri.go:89] found id: "036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:49:53.088309 3102485 cri.go:89] found id: "0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:49:53.088314 3102485 cri.go:89] found id: ""
I0923 12:49:53.088321 3102485 logs.go:276] 2 containers: [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649]
I0923 12:49:53.088398 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.092061 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:49:53.095472 3102485 logs.go:123] Gathering logs for kube-controller-manager [7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af] ...
I0923 12:49:53.095536 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:49:53.152558 3102485 logs.go:123] Gathering logs for kube-apiserver [2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85] ...
I0923 12:49:53.152592 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:49:53.214318 3102485 logs.go:123] Gathering logs for kube-scheduler [2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef] ...
I0923 12:49:53.214352 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:49:53.256471 3102485 logs.go:123] Gathering logs for etcd [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a] ...
I0923 12:49:53.256504 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:49:53.299642 3102485 logs.go:123] Gathering logs for coredns [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb] ...
I0923 12:49:53.299671 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:49:53.341299 3102485 logs.go:123] Gathering logs for kubernetes-dashboard [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a] ...
I0923 12:49:53.341331 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:49:53.382087 3102485 logs.go:123] Gathering logs for kubelet ...
I0923 12:49:53.382166 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:49:53.438281 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989651 658 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438539 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989920 658 reflector.go:138] object-"kube-system"/"coredns-token-ljzll": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-ljzll" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438760 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990004 658 reflector.go:138] object-"kube-system"/"kube-proxy-token-267ss": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-267ss" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.438973 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990077 658 reflector.go:138] object-"kube-system"/"kindnet-token-np6vw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-np6vw" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439173 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990159 658 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439407 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990239 658 reflector.go:138] object-"kube-system"/"storage-provisioner-token-xjwkr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-xjwkr" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.439616 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990319 658 reflector.go:138] object-"default"/"default-token-cl5l9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-cl5l9" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.441346 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:29 old-k8s-version-605045 kubelet[658]: E0923 12:44:29.145826 658 reflector.go:138] object-"kube-system"/"metrics-server-token-7ztnm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-7ztnm" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:49:53.449611 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.630570 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.450439 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.843250 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.453223 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:45 old-k8s-version-605045 kubelet[658]: E0923 12:44:45.501139 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.455318 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:53 old-k8s-version-605045 kubelet[658]: E0923 12:44:53.971600 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.455647 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:54 old-k8s-version-605045 kubelet[658]: E0923 12:44:54.975959 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.455973 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.411779 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.456158 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.466238 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.456946 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:04 old-k8s-version-605045 kubelet[658]: E0923 12:45:04.013052 658 pod_workers.go:191] Error syncing pod bc5f03f4-beaa-4a9b-a052-24d558f95beb ("storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"
W0923 12:49:53.459838 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:09 old-k8s-version-605045 kubelet[658]: E0923 12:45:09.476299 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.460496 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:11 old-k8s-version-605045 kubelet[658]: E0923 12:45:11.038230 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.460830 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:16 old-k8s-version-605045 kubelet[658]: E0923 12:45:16.410049 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.461150 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:24 old-k8s-version-605045 kubelet[658]: E0923 12:45:24.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.461739 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:32 old-k8s-version-605045 kubelet[658]: E0923 12:45:32.109824 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.462067 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:36 old-k8s-version-605045 kubelet[658]: E0923 12:45:36.410320 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.462252 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:39 old-k8s-version-605045 kubelet[658]: E0923 12:45:39.471779 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.462581 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:51 old-k8s-version-605045 kubelet[658]: E0923 12:45:51.466132 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.465099 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:53 old-k8s-version-605045 kubelet[658]: E0923 12:45:53.479122 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.465442 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:02 old-k8s-version-605045 kubelet[658]: E0923 12:46:02.465540 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.465628 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:07 old-k8s-version-605045 kubelet[658]: E0923 12:46:07.466508 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.466225 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:17 old-k8s-version-605045 kubelet[658]: E0923 12:46:17.299572 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.466410 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:22 old-k8s-version-605045 kubelet[658]: E0923 12:46:22.465904 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.466735 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:26 old-k8s-version-605045 kubelet[658]: E0923 12:46:26.410045 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.466919 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:35 old-k8s-version-605045 kubelet[658]: E0923 12:46:35.469207 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.467245 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:38 old-k8s-version-605045 kubelet[658]: E0923 12:46:38.465574 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.467441 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:47 old-k8s-version-605045 kubelet[658]: E0923 12:46:47.467336 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.467773 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:52 old-k8s-version-605045 kubelet[658]: E0923 12:46:52.465530 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.467957 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:02 old-k8s-version-605045 kubelet[658]: E0923 12:47:02.465921 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.468283 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:07 old-k8s-version-605045 kubelet[658]: E0923 12:47:07.466739 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.471301 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:14 old-k8s-version-605045 kubelet[658]: E0923 12:47:14.473947 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:49:53.471640 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:22 old-k8s-version-605045 kubelet[658]: E0923 12:47:22.465965 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.471829 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:26 old-k8s-version-605045 kubelet[658]: E0923 12:47:26.465940 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.472156 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:33 old-k8s-version-605045 kubelet[658]: E0923 12:47:33.467362 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.472346 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:38 old-k8s-version-605045 kubelet[658]: E0923 12:47:38.465964 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.472930 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:45 old-k8s-version-605045 kubelet[658]: E0923 12:47:45.536822 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473261 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:46 old-k8s-version-605045 kubelet[658]: E0923 12:47:46.541111 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473453 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:49 old-k8s-version-605045 kubelet[658]: E0923 12:47:49.465875 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.473780 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:00 old-k8s-version-605045 kubelet[658]: E0923 12:48:00.479190 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.473963 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:02 old-k8s-version-605045 kubelet[658]: E0923 12:48:02.466076 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.474292 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:11 old-k8s-version-605045 kubelet[658]: E0923 12:48:11.465507 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.474477 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:16 old-k8s-version-605045 kubelet[658]: E0923 12:48:16.465938 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.474805 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:22 old-k8s-version-605045 kubelet[658]: E0923 12:48:22.465512 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.474992 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:30 old-k8s-version-605045 kubelet[658]: E0923 12:48:30.465840 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.475329 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:35 old-k8s-version-605045 kubelet[658]: E0923 12:48:35.474238 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.475519 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:44 old-k8s-version-605045 kubelet[658]: E0923 12:48:44.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.475848 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:49 old-k8s-version-605045 kubelet[658]: E0923 12:48:49.466027 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.476033 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:55 old-k8s-version-605045 kubelet[658]: E0923 12:48:55.466326 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.476361 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: E0923 12:49:04.465573 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.476571 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:07 old-k8s-version-605045 kubelet[658]: E0923 12:49:07.466660 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.476901 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: E0923 12:49:16.465577 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.477087 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:20 old-k8s-version-605045 kubelet[658]: E0923 12:49:20.466093 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.477419 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.477605 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.477931 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:53.478116 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:53.478441 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:49:53.478453 3102485 logs.go:123] Gathering logs for describe nodes ...
I0923 12:49:53.478468 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:49:53.626281 3102485 logs.go:123] Gathering logs for kube-scheduler [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89] ...
I0923 12:49:53.626312 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:49:53.671782 3102485 logs.go:123] Gathering logs for kube-proxy [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb] ...
I0923 12:49:53.671827 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:49:53.715920 3102485 logs.go:123] Gathering logs for kube-proxy [c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8] ...
I0923 12:49:53.715952 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:49:53.773316 3102485 logs.go:123] Gathering logs for kindnet [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6] ...
I0923 12:49:53.773355 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:49:53.824387 3102485 logs.go:123] Gathering logs for dmesg ...
I0923 12:49:53.824420 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:49:53.844575 3102485 logs.go:123] Gathering logs for coredns [8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993] ...
I0923 12:49:53.844609 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:49:53.896670 3102485 logs.go:123] Gathering logs for kube-controller-manager [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd] ...
I0923 12:49:53.896699 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:49:53.954435 3102485 logs.go:123] Gathering logs for kindnet [7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f] ...
I0923 12:49:53.954471 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:49:53.996651 3102485 logs.go:123] Gathering logs for storage-provisioner [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19] ...
I0923 12:49:53.996678 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:49:54.043169 3102485 logs.go:123] Gathering logs for storage-provisioner [0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649] ...
I0923 12:49:54.043207 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:49:54.091390 3102485 logs.go:123] Gathering logs for containerd ...
I0923 12:49:54.091423 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:49:54.154040 3102485 logs.go:123] Gathering logs for container status ...
I0923 12:49:54.154077 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:49:54.203612 3102485 logs.go:123] Gathering logs for kube-apiserver [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb] ...
I0923 12:49:54.203644 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:49:54.262283 3102485 logs.go:123] Gathering logs for etcd [1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2] ...
I0923 12:49:54.262317 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:49:54.308962 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:54.308990 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:49:54.309084 3102485 out.go:270] X Problems detected in kubelet:
W0923 12:49:54.309098 3102485 out.go:270] Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:54.309128 3102485 out.go:270] Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:54.309153 3102485 out.go:270] Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:49:54.309161 3102485 out.go:270] Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:49:54.309172 3102485 out.go:270] Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:49:54.309177 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:49:54.309191 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:50:02.922302 3107459 system_pods.go:59] 9 kube-system pods found
I0923 12:50:02.922338 3107459 system_pods.go:61] "coredns-7c65d6cfc9-r5klx" [442d02d5-c0b6-416c-bc87-b8a2337a1301] Running
I0923 12:50:02.922345 3107459 system_pods.go:61] "etcd-no-preload-569169" [75f544ac-42d7-489a-bc48-9bdffa18755c] Running
I0923 12:50:02.922350 3107459 system_pods.go:61] "kindnet-shb4d" [37d396f8-f16a-4ddf-b53f-c437d2ec1c49] Running
I0923 12:50:02.922354 3107459 system_pods.go:61] "kube-apiserver-no-preload-569169" [091070ef-ccd3-45ea-a976-899c5d833219] Running
I0923 12:50:02.922360 3107459 system_pods.go:61] "kube-controller-manager-no-preload-569169" [6be673d7-a74b-4dbb-a17b-94c750685abb] Running
I0923 12:50:02.922363 3107459 system_pods.go:61] "kube-proxy-bcttj" [7aaf66d9-9d3b-4a92-82e0-b3b5dc8e4673] Running
I0923 12:50:02.922367 3107459 system_pods.go:61] "kube-scheduler-no-preload-569169" [ca379951-c239-4243-97f9-9fc4c75bce3a] Running
I0923 12:50:02.922374 3107459 system_pods.go:61] "metrics-server-6867b74b74-5twtp" [a9a15ae3-d31c-43f6-be0b-8fb9aa4f196a] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0923 12:50:02.922386 3107459 system_pods.go:61] "storage-provisioner" [8704a02a-10e0-43cd-842a-4de10f70c535] Running
I0923 12:50:02.922401 3107459 system_pods.go:74] duration metric: took 11.874767747s to wait for pod list to return data ...
I0923 12:50:02.922409 3107459 default_sa.go:34] waiting for default service account to be created ...
I0923 12:50:02.925267 3107459 default_sa.go:45] found service account: "default"
I0923 12:50:02.925296 3107459 default_sa.go:55] duration metric: took 2.878215ms for default service account to be created ...
I0923 12:50:02.925306 3107459 system_pods.go:116] waiting for k8s-apps to be running ...
I0923 12:50:02.930935 3107459 system_pods.go:86] 9 kube-system pods found
I0923 12:50:02.930970 3107459 system_pods.go:89] "coredns-7c65d6cfc9-r5klx" [442d02d5-c0b6-416c-bc87-b8a2337a1301] Running
I0923 12:50:02.930977 3107459 system_pods.go:89] "etcd-no-preload-569169" [75f544ac-42d7-489a-bc48-9bdffa18755c] Running
I0923 12:50:02.930985 3107459 system_pods.go:89] "kindnet-shb4d" [37d396f8-f16a-4ddf-b53f-c437d2ec1c49] Running
I0923 12:50:02.930990 3107459 system_pods.go:89] "kube-apiserver-no-preload-569169" [091070ef-ccd3-45ea-a976-899c5d833219] Running
I0923 12:50:02.930996 3107459 system_pods.go:89] "kube-controller-manager-no-preload-569169" [6be673d7-a74b-4dbb-a17b-94c750685abb] Running
I0923 12:50:02.931000 3107459 system_pods.go:89] "kube-proxy-bcttj" [7aaf66d9-9d3b-4a92-82e0-b3b5dc8e4673] Running
I0923 12:50:02.931004 3107459 system_pods.go:89] "kube-scheduler-no-preload-569169" [ca379951-c239-4243-97f9-9fc4c75bce3a] Running
I0923 12:50:02.931011 3107459 system_pods.go:89] "metrics-server-6867b74b74-5twtp" [a9a15ae3-d31c-43f6-be0b-8fb9aa4f196a] Pending / Ready:ContainersNotReady (containers with unready status: [metrics-server]) / ContainersReady:ContainersNotReady (containers with unready status: [metrics-server])
I0923 12:50:02.931015 3107459 system_pods.go:89] "storage-provisioner" [8704a02a-10e0-43cd-842a-4de10f70c535] Running
I0923 12:50:02.931025 3107459 system_pods.go:126] duration metric: took 5.713459ms to wait for k8s-apps to be running ...
I0923 12:50:02.931037 3107459 system_svc.go:44] waiting for kubelet service to be running ....
I0923 12:50:02.931097 3107459 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service kubelet
I0923 12:50:02.951124 3107459 system_svc.go:56] duration metric: took 20.076929ms WaitForService to wait for kubelet
I0923 12:50:02.951220 3107459 kubeadm.go:582] duration metric: took 4m40.978155055s to wait for: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0923 12:50:02.951254 3107459 node_conditions.go:102] verifying NodePressure condition ...
I0923 12:50:02.954625 3107459 node_conditions.go:122] node storage ephemeral capacity is 203034800Ki
I0923 12:50:02.954705 3107459 node_conditions.go:123] node cpu capacity is 2
I0923 12:50:02.954736 3107459 node_conditions.go:105] duration metric: took 3.343443ms to run NodePressure ...
I0923 12:50:02.954778 3107459 start.go:241] waiting for startup goroutines ...
I0923 12:50:02.954804 3107459 start.go:246] waiting for cluster config update ...
I0923 12:50:02.954829 3107459 start.go:255] writing updated cluster config ...
I0923 12:50:02.955198 3107459 ssh_runner.go:195] Run: rm -f paused
I0923 12:50:03.019577 3107459 start.go:600] kubectl: 1.31.1, cluster: 1.31.1 (minor skew: 0)
I0923 12:50:03.021829 3107459 out.go:177] * Done! kubectl is now configured to use "no-preload-569169" cluster and "default" namespace by default
I0923 12:50:04.310257 3102485 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0923 12:50:04.324717 3102485 api_server.go:72] duration metric: took 5m54.581827s to wait for apiserver process to appear ...
I0923 12:50:04.324741 3102485 api_server.go:88] waiting for apiserver healthz status ...
I0923 12:50:04.324779 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0923 12:50:04.324837 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0923 12:50:04.371861 3102485 cri.go:89] found id: "b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:50:04.371888 3102485 cri.go:89] found id: "2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:50:04.371893 3102485 cri.go:89] found id: ""
I0923 12:50:04.371900 3102485 logs.go:276] 2 containers: [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85]
I0923 12:50:04.371957 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.376309 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.380595 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0923 12:50:04.380674 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0923 12:50:04.423923 3102485 cri.go:89] found id: "ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:50:04.423947 3102485 cri.go:89] found id: "1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:50:04.423952 3102485 cri.go:89] found id: ""
I0923 12:50:04.423960 3102485 logs.go:276] 2 containers: [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2]
I0923 12:50:04.424021 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.427792 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.431611 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0923 12:50:04.431688 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0923 12:50:04.472288 3102485 cri.go:89] found id: "0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:50:04.472310 3102485 cri.go:89] found id: "8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:50:04.472315 3102485 cri.go:89] found id: ""
I0923 12:50:04.472322 3102485 logs.go:276] 2 containers: [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993]
I0923 12:50:04.472380 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.476372 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.479861 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0923 12:50:04.479939 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0923 12:50:04.519484 3102485 cri.go:89] found id: "df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:50:04.519508 3102485 cri.go:89] found id: "2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:50:04.519513 3102485 cri.go:89] found id: ""
I0923 12:50:04.519521 3102485 logs.go:276] 2 containers: [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef]
I0923 12:50:04.519579 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.523443 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.527138 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0923 12:50:04.527233 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0923 12:50:04.571087 3102485 cri.go:89] found id: "3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:50:04.571107 3102485 cri.go:89] found id: "c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:50:04.571112 3102485 cri.go:89] found id: ""
I0923 12:50:04.571119 3102485 logs.go:276] 2 containers: [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8]
I0923 12:50:04.571176 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.575120 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.578668 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0923 12:50:04.578769 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0923 12:50:04.622151 3102485 cri.go:89] found id: "37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:50:04.622181 3102485 cri.go:89] found id: "7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:50:04.622186 3102485 cri.go:89] found id: ""
I0923 12:50:04.622194 3102485 logs.go:276] 2 containers: [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af]
I0923 12:50:04.622263 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.626117 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.629549 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0923 12:50:04.629624 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0923 12:50:04.669203 3102485 cri.go:89] found id: "a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:50:04.669277 3102485 cri.go:89] found id: "7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:50:04.669291 3102485 cri.go:89] found id: ""
I0923 12:50:04.669299 3102485 logs.go:276] 2 containers: [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f]
I0923 12:50:04.669417 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.673133 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.676629 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0923 12:50:04.676723 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0923 12:50:04.718915 3102485 cri.go:89] found id: "7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:50:04.718939 3102485 cri.go:89] found id: ""
I0923 12:50:04.718947 3102485 logs.go:276] 1 containers: [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a]
I0923 12:50:04.719031 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.723030 3102485 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0923 12:50:04.723104 3102485 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0923 12:50:04.773144 3102485 cri.go:89] found id: "036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:50:04.773225 3102485 cri.go:89] found id: "0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:50:04.773246 3102485 cri.go:89] found id: ""
I0923 12:50:04.773280 3102485 logs.go:276] 2 containers: [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649]
I0923 12:50:04.773393 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.777009 3102485 ssh_runner.go:195] Run: which crictl
I0923 12:50:04.780384 3102485 logs.go:123] Gathering logs for describe nodes ...
I0923 12:50:04.780408 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0923 12:50:04.928411 3102485 logs.go:123] Gathering logs for etcd [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a] ...
I0923 12:50:04.928490 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a"
I0923 12:50:04.972999 3102485 logs.go:123] Gathering logs for kube-scheduler [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89] ...
I0923 12:50:04.973033 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89"
I0923 12:50:05.018662 3102485 logs.go:123] Gathering logs for kube-scheduler [2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef] ...
I0923 12:50:05.018696 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef"
I0923 12:50:05.073546 3102485 logs.go:123] Gathering logs for kube-controller-manager [7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af] ...
I0923 12:50:05.073652 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af"
I0923 12:50:05.135452 3102485 logs.go:123] Gathering logs for kindnet [7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f] ...
I0923 12:50:05.135491 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f"
I0923 12:50:05.180530 3102485 logs.go:123] Gathering logs for kubelet ...
I0923 12:50:05.180558 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0923 12:50:05.235647 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989651 658 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.235906 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.989920 658 reflector.go:138] object-"kube-system"/"coredns-token-ljzll": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-ljzll" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236127 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990004 658 reflector.go:138] object-"kube-system"/"kube-proxy-token-267ss": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-267ss" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236340 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990077 658 reflector.go:138] object-"kube-system"/"kindnet-token-np6vw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-np6vw" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236541 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990159 658 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236769 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990239 658 reflector.go:138] object-"kube-system"/"storage-provisioner-token-xjwkr": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-xjwkr" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.236977 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:28 old-k8s-version-605045 kubelet[658]: E0923 12:44:28.990319 658 reflector.go:138] object-"default"/"default-token-cl5l9": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-cl5l9" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.238682 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:29 old-k8s-version-605045 kubelet[658]: E0923 12:44:29.145826 658 reflector.go:138] object-"kube-system"/"metrics-server-token-7ztnm": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-7ztnm" is forbidden: User "system:node:old-k8s-version-605045" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-605045' and this object
W0923 12:50:05.246929 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.630570 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.247743 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:31 old-k8s-version-605045 kubelet[658]: E0923 12:44:31.843250 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.250499 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:45 old-k8s-version-605045 kubelet[658]: E0923 12:44:45.501139 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.252595 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:53 old-k8s-version-605045 kubelet[658]: E0923 12:44:53.971600 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.252923 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:54 old-k8s-version-605045 kubelet[658]: E0923 12:44:54.975959 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.253253 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.411779 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.253439 3102485 logs.go:138] Found kubelet problem: Sep 23 12:44:56 old-k8s-version-605045 kubelet[658]: E0923 12:44:56.466238 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.254205 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:04 old-k8s-version-605045 kubelet[658]: E0923 12:45:04.013052 658 pod_workers.go:191] Error syncing pod bc5f03f4-beaa-4a9b-a052-24d558f95beb ("storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(bc5f03f4-beaa-4a9b-a052-24d558f95beb)"
W0923 12:50:05.256971 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:09 old-k8s-version-605045 kubelet[658]: E0923 12:45:09.476299 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.257566 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:11 old-k8s-version-605045 kubelet[658]: E0923 12:45:11.038230 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.257891 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:16 old-k8s-version-605045 kubelet[658]: E0923 12:45:16.410049 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.258204 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:24 old-k8s-version-605045 kubelet[658]: E0923 12:45:24.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.258788 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:32 old-k8s-version-605045 kubelet[658]: E0923 12:45:32.109824 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.259112 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:36 old-k8s-version-605045 kubelet[658]: E0923 12:45:36.410320 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.259325 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:39 old-k8s-version-605045 kubelet[658]: E0923 12:45:39.471779 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.259652 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:51 old-k8s-version-605045 kubelet[658]: E0923 12:45:51.466132 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.262093 3102485 logs.go:138] Found kubelet problem: Sep 23 12:45:53 old-k8s-version-605045 kubelet[658]: E0923 12:45:53.479122 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.262418 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:02 old-k8s-version-605045 kubelet[658]: E0923 12:46:02.465540 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.262604 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:07 old-k8s-version-605045 kubelet[658]: E0923 12:46:07.466508 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.263188 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:17 old-k8s-version-605045 kubelet[658]: E0923 12:46:17.299572 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.263379 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:22 old-k8s-version-605045 kubelet[658]: E0923 12:46:22.465904 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.263705 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:26 old-k8s-version-605045 kubelet[658]: E0923 12:46:26.410045 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.263889 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:35 old-k8s-version-605045 kubelet[658]: E0923 12:46:35.469207 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.264212 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:38 old-k8s-version-605045 kubelet[658]: E0923 12:46:38.465574 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.264396 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:47 old-k8s-version-605045 kubelet[658]: E0923 12:46:47.467336 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.264721 3102485 logs.go:138] Found kubelet problem: Sep 23 12:46:52 old-k8s-version-605045 kubelet[658]: E0923 12:46:52.465530 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.264904 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:02 old-k8s-version-605045 kubelet[658]: E0923 12:47:02.465921 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.265242 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:07 old-k8s-version-605045 kubelet[658]: E0923 12:47:07.466739 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.267689 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:14 old-k8s-version-605045 kubelet[658]: E0923 12:47:14.473947 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.268018 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:22 old-k8s-version-605045 kubelet[658]: E0923 12:47:22.465965 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.268204 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:26 old-k8s-version-605045 kubelet[658]: E0923 12:47:26.465940 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.268528 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:33 old-k8s-version-605045 kubelet[658]: E0923 12:47:33.467362 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.268712 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:38 old-k8s-version-605045 kubelet[658]: E0923 12:47:38.465964 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.269300 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:45 old-k8s-version-605045 kubelet[658]: E0923 12:47:45.536822 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.269629 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:46 old-k8s-version-605045 kubelet[658]: E0923 12:47:46.541111 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.269813 3102485 logs.go:138] Found kubelet problem: Sep 23 12:47:49 old-k8s-version-605045 kubelet[658]: E0923 12:47:49.465875 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.270138 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:00 old-k8s-version-605045 kubelet[658]: E0923 12:48:00.479190 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.270323 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:02 old-k8s-version-605045 kubelet[658]: E0923 12:48:02.466076 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.270652 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:11 old-k8s-version-605045 kubelet[658]: E0923 12:48:11.465507 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.270835 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:16 old-k8s-version-605045 kubelet[658]: E0923 12:48:16.465938 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.271160 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:22 old-k8s-version-605045 kubelet[658]: E0923 12:48:22.465512 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.271360 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:30 old-k8s-version-605045 kubelet[658]: E0923 12:48:30.465840 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.271686 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:35 old-k8s-version-605045 kubelet[658]: E0923 12:48:35.474238 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.271870 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:44 old-k8s-version-605045 kubelet[658]: E0923 12:48:44.465984 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.272196 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:49 old-k8s-version-605045 kubelet[658]: E0923 12:48:49.466027 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.272381 3102485 logs.go:138] Found kubelet problem: Sep 23 12:48:55 old-k8s-version-605045 kubelet[658]: E0923 12:48:55.466326 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.272706 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: E0923 12:49:04.465573 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.272892 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:07 old-k8s-version-605045 kubelet[658]: E0923 12:49:07.466660 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.273218 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: E0923 12:49:16.465577 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.273406 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:20 old-k8s-version-605045 kubelet[658]: E0923 12:49:20.466093 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.273731 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.273915 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.274240 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.274425 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:05.274751 3102485 logs.go:138] Found kubelet problem: Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:05.277200 3102485 logs.go:138] Found kubelet problem: Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:05.277533 3102485 logs.go:138] Found kubelet problem: Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:50:05.277544 3102485 logs.go:123] Gathering logs for etcd [1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2] ...
I0923 12:50:05.277558 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2"
I0923 12:50:05.322579 3102485 logs.go:123] Gathering logs for coredns [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb] ...
I0923 12:50:05.322658 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb"
I0923 12:50:05.365537 3102485 logs.go:123] Gathering logs for coredns [8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993] ...
I0923 12:50:05.365569 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993"
I0923 12:50:05.407857 3102485 logs.go:123] Gathering logs for kube-proxy [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb] ...
I0923 12:50:05.407925 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb"
I0923 12:50:05.456230 3102485 logs.go:123] Gathering logs for kube-proxy [c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8] ...
I0923 12:50:05.456256 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8"
I0923 12:50:05.499217 3102485 logs.go:123] Gathering logs for kube-controller-manager [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd] ...
I0923 12:50:05.499248 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd"
I0923 12:50:05.563528 3102485 logs.go:123] Gathering logs for kubernetes-dashboard [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a] ...
I0923 12:50:05.563564 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a"
I0923 12:50:05.606773 3102485 logs.go:123] Gathering logs for kube-apiserver [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb] ...
I0923 12:50:05.606803 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb"
I0923 12:50:05.679239 3102485 logs.go:123] Gathering logs for containerd ...
I0923 12:50:05.679272 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0923 12:50:05.750829 3102485 logs.go:123] Gathering logs for storage-provisioner [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19] ...
I0923 12:50:05.750867 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19"
I0923 12:50:05.792354 3102485 logs.go:123] Gathering logs for storage-provisioner [0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649] ...
I0923 12:50:05.792381 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649"
I0923 12:50:05.836837 3102485 logs.go:123] Gathering logs for kindnet [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6] ...
I0923 12:50:05.836865 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6"
I0923 12:50:05.905615 3102485 logs.go:123] Gathering logs for kube-apiserver [2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85] ...
I0923 12:50:05.905647 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85"
I0923 12:50:05.963389 3102485 logs.go:123] Gathering logs for container status ...
I0923 12:50:05.963426 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0923 12:50:06.020318 3102485 logs.go:123] Gathering logs for dmesg ...
I0923 12:50:06.020356 3102485 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0923 12:50:06.038364 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:50:06.038396 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0923 12:50:06.038444 3102485 out.go:270] X Problems detected in kubelet:
W0923 12:50:06.038457 3102485 out.go:270] Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:06.038464 3102485 out.go:270] Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0923 12:50:06.038477 3102485 out.go:270] Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
W0923 12:50:06.038485 3102485 out.go:270] Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0923 12:50:06.038496 3102485 out.go:270] Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
I0923 12:50:06.038518 3102485 out.go:358] Setting ErrFile to fd 2...
I0923 12:50:06.038531 3102485 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0923 12:50:16.039609 3102485 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0923 12:50:16.049801 3102485 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0923 12:50:16.052452 3102485 out.go:201]
W0923 12:50:16.054574 3102485 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0923 12:50:16.054647 3102485 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0923 12:50:16.054671 3102485 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0923 12:50:16.054677 3102485 out.go:270] *
W0923 12:50:16.055931 3102485 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0923 12:50:16.058110 3102485 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
4e7193d4b39bc 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 ea38f9c601a25 dashboard-metrics-scraper-8d5bb5db8-7btzn
036c3b8487f90 ba04bb24b9575 5 minutes ago Running storage-provisioner 2 94921e2d0f194 storage-provisioner
7f77123376f58 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 2cb18bad0e3bd kubernetes-dashboard-cd95d586-kbkhr
0c783388fb537 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 94921e2d0f194 storage-provisioner
3613dc571cef5 25a5233254979 5 minutes ago Running kube-proxy 1 f9c415e0b1fc4 kube-proxy-9m2fb
7acdc9496705b 1611cd07b61d5 5 minutes ago Running busybox 1 6256fd653a319 busybox
0b37a02f00d76 db91994f4ee8f 5 minutes ago Running coredns 1 ba18dbe60e613 coredns-74ff55c5b-pb95z
a640ded465247 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 a0befa23d3ef1 kindnet-xr6zl
ec7d4cbc7a416 05b738aa1bc63 5 minutes ago Running etcd 1 9f9c8588cd840 etcd-old-k8s-version-605045
37856af16ad99 1df8a2b116bd1 5 minutes ago Running kube-controller-manager 1 78398563e6151 kube-controller-manager-old-k8s-version-605045
b2282feb959a3 2c08bbbc02d3a 5 minutes ago Running kube-apiserver 1 3a88afb9bda02 kube-apiserver-old-k8s-version-605045
df9e1754aa717 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 a7d93b3641799 kube-scheduler-old-k8s-version-605045
a2e6c2b12c626 1611cd07b61d5 6 minutes ago Exited busybox 0 5141279d2a106 busybox
8ff2c93901230 db91994f4ee8f 8 minutes ago Exited coredns 0 a09ce8ae9de9f coredns-74ff55c5b-pb95z
7b7c86609a480 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 fc6662601f18a kindnet-xr6zl
c012604027179 25a5233254979 8 minutes ago Exited kube-proxy 0 1231436aaa084 kube-proxy-9m2fb
7c7e0e07a9e06 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 931f6d2318c3b kube-controller-manager-old-k8s-version-605045
1d4f50e137bba 05b738aa1bc63 8 minutes ago Exited etcd 0 9de746e03b4df etcd-old-k8s-version-605045
2e898a14edc8d 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 6426278f57a57 kube-apiserver-old-k8s-version-605045
2580add0e36cc e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 77bd7af4febc3 kube-scheduler-old-k8s-version-605045
==> containerd <==
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.491036676Z" level=info msg="CreateContainer within sandbox \"ea38f9c601a2569f42859c53e167f013d12cec9624665194571c7952642847cd\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f\""
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.492846038Z" level=info msg="StartContainer for \"d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f\""
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.587954469Z" level=info msg="StartContainer for \"d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f\" returns successfully"
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.619262715Z" level=info msg="shim disconnected" id=d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f namespace=k8s.io
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.619576757Z" level=warning msg="cleaning up after shim disconnected" id=d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f namespace=k8s.io
Sep 23 12:46:16 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:16.619601815Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 23 12:46:17 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:17.300789673Z" level=info msg="RemoveContainer for \"4e6c7546d12f897baf7ccb1f48215f38e92d6303a04dff1d47ff2015d594d0e7\""
Sep 23 12:46:17 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:46:17.307091133Z" level=info msg="RemoveContainer for \"4e6c7546d12f897baf7ccb1f48215f38e92d6303a04dff1d47ff2015d594d0e7\" returns successfully"
Sep 23 12:47:14 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:14.466251233Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:47:14 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:14.471720271Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Sep 23 12:47:14 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:14.473199123Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Sep 23 12:47:14 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:14.473287550Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.468534470Z" level=info msg="CreateContainer within sandbox \"ea38f9c601a2569f42859c53e167f013d12cec9624665194571c7952642847cd\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.482087983Z" level=info msg="CreateContainer within sandbox \"ea38f9c601a2569f42859c53e167f013d12cec9624665194571c7952642847cd\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe\""
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.482763533Z" level=info msg="StartContainer for \"4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe\""
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.560151798Z" level=info msg="StartContainer for \"4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe\" returns successfully"
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.585039577Z" level=info msg="shim disconnected" id=4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe namespace=k8s.io
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.585094945Z" level=warning msg="cleaning up after shim disconnected" id=4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe namespace=k8s.io
Sep 23 12:47:44 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:44.585106826Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Sep 23 12:47:45 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:45.538646917Z" level=info msg="RemoveContainer for \"d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f\""
Sep 23 12:47:45 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:47:45.551903406Z" level=info msg="RemoveContainer for \"d59945de148935dccd5b1b3247f440954d84fe57040abd386b8281702c814c6f\" returns successfully"
Sep 23 12:50:01 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:50:01.466180697Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:50:01 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:50:01.483658180Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Sep 23 12:50:01 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:50:01.485389930Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Sep 23 12:50:01 old-k8s-version-605045 containerd[565]: time="2024-09-23T12:50:01.485435821Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [0b37a02f00d76de0d1b1e3d0fc7c2ca82878e8c7fdb9ff77b22d675e8e75cdeb] <==
I0923 12:45:01.721321 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-23 12:44:31.720710929 +0000 UTC m=+0.034024264) (total time: 30.000480499s):
Trace[2019727887]: [30.000480499s] [30.000480499s] END
E0923 12:45:01.721358 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0923 12:45:01.721595 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-23 12:44:31.72128138 +0000 UTC m=+0.034594715) (total time: 30.000299691s):
Trace[939984059]: [30.000299691s] [30.000299691s] END
E0923 12:45:01.721602 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0923 12:45:01.721744 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-09-23 12:44:31.721551139 +0000 UTC m=+0.034864482) (total time: 30.000181972s):
Trace[911902081]: [30.000181972s] [30.000181972s] END
E0923 12:45:01.721749 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
[INFO] plugin/ready: Still waiting on: "kubernetes"
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:49265 - 19444 "HINFO IN 104182449932966012.4262663174497182054. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.00457716s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> coredns [8ff2c93901230ca94db7c0763287c381fd3c5413e52c3e45be28d60a3638b993] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:40072 - 44776 "HINFO IN 3327055561030228005.1736002694204985564. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.011181988s
==> describe nodes <==
Name: old-k8s-version-605045
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-605045
kubernetes.io/os=linux
minikube.k8s.io/commit=a36553b39c7bbbd910f6bfb97f7b698be94b4e6e
minikube.k8s.io/name=old-k8s-version-605045
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_09_23T12_41_44_0700
minikube.k8s.io/version=v1.34.0
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 23 Sep 2024 12:41:41 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-605045
AcquireTime: <unset>
RenewTime: Mon, 23 Sep 2024 12:50:11 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 23 Sep 2024 12:45:19 +0000 Mon, 23 Sep 2024 12:41:34 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 23 Sep 2024 12:45:19 +0000 Mon, 23 Sep 2024 12:41:34 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 23 Sep 2024 12:45:19 +0000 Mon, 23 Sep 2024 12:41:34 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 23 Sep 2024 12:45:19 +0000 Mon, 23 Sep 2024 12:41:59 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-605045
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022304Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022304Ki
pods: 110
System Info:
Machine ID: fc3193edb2ea462bb87210e1f4d90c12
System UUID: 21a527bb-7643-4c63-8458-02233d2ecd63
Boot ID: d8899273-2c3a-49f7-8c9a-66d2209373ba
Kernel Version: 5.15.0-1070-aws
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.22
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m44s
kube-system coredns-74ff55c5b-pb95z 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m19s
kube-system etcd-old-k8s-version-605045 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m25s
kube-system kindnet-xr6zl 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m19s
kube-system kube-apiserver-old-k8s-version-605045 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m25s
kube-system kube-controller-manager-old-k8s-version-605045 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m25s
kube-system kube-proxy-9m2fb 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m19s
kube-system kube-scheduler-old-k8s-version-605045 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m25s
kube-system metrics-server-9975d5f86-kf958 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m31s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m17s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-7btzn 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m31s
kubernetes-dashboard kubernetes-dashboard-cd95d586-kbkhr 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m31s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m45s (x4 over 8m45s) kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m45s (x4 over 8m45s) kubelet Node old-k8s-version-605045 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m45s (x4 over 8m45s) kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientPID
Normal Starting 8m26s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m26s kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m26s kubelet Node old-k8s-version-605045 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m26s kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m26s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m19s kubelet Node old-k8s-version-605045 status is now: NodeReady
Normal Starting 8m17s kube-proxy Starting kube-proxy.
Normal Starting 6m1s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m1s (x7 over 6m1s) kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m1s (x8 over 6m1s) kubelet Node old-k8s-version-605045 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m1s (x8 over 6m1s) kubelet Node old-k8s-version-605045 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m1s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m45s kube-proxy Starting kube-proxy.
==> dmesg <==
==> etcd [1d4f50e137bbacd490ef68aa6755f494bdbd13d94cbf4f6b96dfc5b2f75932d2] <==
raft2024/09/23 12:41:35 INFO: 9f0758e1c58a86ed is starting a new election at term 1
raft2024/09/23 12:41:35 INFO: 9f0758e1c58a86ed became candidate at term 2
raft2024/09/23 12:41:35 INFO: 9f0758e1c58a86ed received MsgVoteResp from 9f0758e1c58a86ed at term 2
raft2024/09/23 12:41:35 INFO: 9f0758e1c58a86ed became leader at term 2
raft2024/09/23 12:41:35 INFO: raft.node: 9f0758e1c58a86ed elected leader 9f0758e1c58a86ed at term 2
2024-09-23 12:41:35.008772 I | etcdserver: setting up the initial cluster version to 3.4
2024-09-23 12:41:35.011549 N | etcdserver/membership: set the initial cluster version to 3.4
2024-09-23 12:41:35.011758 I | etcdserver/api: enabled capabilities for version 3.4
2024-09-23 12:41:35.011902 I | etcdserver: published {Name:old-k8s-version-605045 ClientURLs:[https://192.168.85.2:2379]} to cluster 68eaea490fab4e05
2024-09-23 12:41:35.011999 I | embed: ready to serve client requests
2024-09-23 12:41:35.013602 I | embed: serving client requests on 192.168.85.2:2379
2024-09-23 12:41:35.014012 I | embed: ready to serve client requests
2024-09-23 12:41:35.015513 I | embed: serving client requests on 127.0.0.1:2379
2024-09-23 12:41:56.118860 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:06.111300 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:16.111243 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:26.111265 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:36.118926 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:46.111351 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:42:56.111337 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:43:06.111383 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:43:16.111244 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:43:26.111793 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:43:36.111271 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:43:46.112760 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [ec7d4cbc7a416ae0957456b63255d13609ace8374894c9331cf4df0ab18e3a7a] <==
2024-09-23 12:46:13.602637 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:46:23.602762 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:46:33.602524 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:46:43.602648 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:46:53.602564 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:03.602541 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:13.602653 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:23.602584 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:33.602616 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:43.602956 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:47:53.602609 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:03.604461 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:13.602578 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:23.602567 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:33.602679 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:43.602673 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:48:53.602605 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:03.602651 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:13.602594 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:23.602667 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:33.602507 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:43.602739 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:49:53.602609 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:50:03.602643 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-09-23 12:50:13.602740 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
12:50:18 up 1 day, 20:32, 0 users, load average: 0.71, 1.47, 2.19
Linux old-k8s-version-605045 5.15.0-1070-aws #76~20.04.1-Ubuntu SMP Mon Sep 2 12:20:48 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.5 LTS"
==> kindnet [7b7c86609a480c3071f1c1855f3ff241269d00803f81188257286e0cb5ab383f] <==
I0923 12:42:04.212255 1 controller.go:338] Waiting for informer caches to sync
I0923 12:42:04.212260 1 shared_informer.go:313] Waiting for caches to sync for kube-network-policies
I0923 12:42:04.412826 1 shared_informer.go:320] Caches are synced for kube-network-policies
I0923 12:42:04.412948 1 metrics.go:61] Registering metrics
I0923 12:42:04.413056 1 controller.go:374] Syncing nftables rules
I0923 12:42:14.212259 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:42:14.212300 1 main.go:299] handling current node
I0923 12:42:24.212783 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:42:24.213001 1 main.go:299] handling current node
I0923 12:42:34.221397 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:42:34.221614 1 main.go:299] handling current node
I0923 12:42:44.217322 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:42:44.217356 1 main.go:299] handling current node
I0923 12:42:54.212425 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:42:54.212542 1 main.go:299] handling current node
I0923 12:43:04.212194 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:43:04.212229 1 main.go:299] handling current node
I0923 12:43:14.220575 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:43:14.220609 1 main.go:299] handling current node
I0923 12:43:24.216527 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:43:24.216559 1 main.go:299] handling current node
I0923 12:43:34.219374 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:43:34.219406 1 main.go:299] handling current node
I0923 12:43:44.216915 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:43:44.216947 1 main.go:299] handling current node
==> kindnet [a640ded4652476a451a288fd103b4cfadff434e6e1a9131a07c1b076942d21e6] <==
I0923 12:48:11.720189 1 main.go:299] handling current node
I0923 12:48:21.720767 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:48:21.720808 1 main.go:299] handling current node
I0923 12:48:31.712517 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:48:31.712554 1 main.go:299] handling current node
I0923 12:48:41.719413 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:48:41.719453 1 main.go:299] handling current node
I0923 12:48:51.712505 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:48:51.712536 1 main.go:299] handling current node
I0923 12:49:01.719375 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:01.719412 1 main.go:299] handling current node
I0923 12:49:11.712425 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:11.712482 1 main.go:299] handling current node
I0923 12:49:21.720783 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:21.720817 1 main.go:299] handling current node
I0923 12:49:31.712288 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:31.712443 1 main.go:299] handling current node
I0923 12:49:41.719390 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:41.719425 1 main.go:299] handling current node
I0923 12:49:51.715343 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:49:51.715440 1 main.go:299] handling current node
I0923 12:50:01.723444 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:50:01.723478 1 main.go:299] handling current node
I0923 12:50:11.718861 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0923 12:50:11.718895 1 main.go:299] handling current node
==> kube-apiserver [2e898a14edc8d3e5634da2afd4a8564b3d2e7fcdf9bd291c3eb2c0523f0c7f85] <==
I0923 12:41:41.888619 1 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0923 12:41:41.888735 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0923 12:41:41.928699 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0923 12:41:41.932700 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0923 12:41:41.932970 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0923 12:41:42.568769 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0923 12:41:42.610961 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0923 12:41:42.726433 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.85.2]
I0923 12:41:42.727714 1 controller.go:606] quota admission added evaluator for: endpoints
I0923 12:41:42.737518 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0923 12:41:43.495742 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0923 12:41:44.079185 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0923 12:41:44.192940 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0923 12:41:52.610782 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0923 12:41:59.649440 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0923 12:41:59.697637 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0923 12:42:16.891843 1 client.go:360] parsed scheme: "passthrough"
I0923 12:42:16.891890 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:42:16.891898 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0923 12:42:49.105747 1 client.go:360] parsed scheme: "passthrough"
I0923 12:42:49.105797 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:42:49.105829 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0923 12:43:30.842599 1 client.go:360] parsed scheme: "passthrough"
I0923 12:43:30.842658 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:43:30.842667 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [b2282feb959a32bad1209dc9e1469d0a0066fba60bfc2fb4094ae3fc687dfdbb] <==
I0923 12:46:56.951027 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:46:56.951036 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0923 12:47:31.713905 1 handler_proxy.go:102] no RequestInfo found in the context
E0923 12:47:31.713979 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0923 12:47:31.713995 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0923 12:47:35.096184 1 client.go:360] parsed scheme: "passthrough"
I0923 12:47:35.096253 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:47:35.096273 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0923 12:48:19.575890 1 client.go:360] parsed scheme: "passthrough"
I0923 12:48:19.576016 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:48:19.576134 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0923 12:49:02.655797 1 client.go:360] parsed scheme: "passthrough"
I0923 12:49:02.655847 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:49:02.655856 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0923 12:49:30.125593 1 handler_proxy.go:102] no RequestInfo found in the context
E0923 12:49:30.125678 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0923 12:49:30.125688 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0923 12:49:36.571321 1 client.go:360] parsed scheme: "passthrough"
I0923 12:49:36.571364 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:49:36.571373 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0923 12:50:12.334864 1 client.go:360] parsed scheme: "passthrough"
I0923 12:50:12.334910 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0923 12:50:12.334918 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [37856af16ad99aeba94fde07e04c227e6c85915d00dafff4a60c1c20b0f562fd] <==
W0923 12:45:55.068724 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:46:18.714562 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:46:26.719233 1 request.go:655] Throttling request took 1.048030814s, request: GET:https://192.168.85.2:8443/apis/autoscaling/v2beta1?timeout=32s
W0923 12:46:27.570835 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:46:49.216435 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:46:59.221244 1 request.go:655] Throttling request took 1.048380127s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0923 12:47:00.073249 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:47:19.718539 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:47:31.724114 1 request.go:655] Throttling request took 1.047950703s, request: GET:https://192.168.85.2:8443/apis/certificates.k8s.io/v1?timeout=32s
W0923 12:47:32.575865 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:47:50.220346 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:48:04.226273 1 request.go:655] Throttling request took 1.0481854s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0923 12:48:05.077999 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:48:20.792981 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:48:36.728556 1 request.go:655] Throttling request took 1.048328963s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0923 12:48:37.581488 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:48:51.295374 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:49:09.231941 1 request.go:655] Throttling request took 1.048429891s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0923 12:49:10.085222 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:49:21.848753 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:49:41.735670 1 request.go:655] Throttling request took 1.048192074s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0923 12:49:42.587176 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0923 12:49:52.351704 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0923 12:50:14.237724 1 request.go:655] Throttling request took 1.048466475s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0923 12:50:15.089494 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [7c7e0e07a9e066b767ae758ebfecb8074648180f820d3e2d75ae49339a7553af] <==
I0923 12:41:59.654223 1 shared_informer.go:247] Caches are synced for resource quota
I0923 12:41:59.655894 1 shared_informer.go:247] Caches are synced for resource quota
I0923 12:41:59.661426 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0923 12:41:59.671491 1 shared_informer.go:247] Caches are synced for HPA
I0923 12:41:59.679636 1 shared_informer.go:247] Caches are synced for attach detach
I0923 12:41:59.691804 1 shared_informer.go:247] Caches are synced for daemon sets
I0923 12:41:59.704411 1 shared_informer.go:247] Caches are synced for disruption
I0923 12:41:59.704435 1 disruption.go:339] Sending events to api server.
I0923 12:41:59.706542 1 shared_informer.go:247] Caches are synced for endpoint
I0923 12:41:59.707586 1 shared_informer.go:247] Caches are synced for ReplicaSet
I0923 12:41:59.715881 1 shared_informer.go:247] Caches are synced for ReplicationController
I0923 12:41:59.717590 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-9m2fb"
I0923 12:41:59.717617 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-xr6zl"
I0923 12:41:59.737549 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-kz5zm"
E0923 12:41:59.813845 1 daemon_controller.go:320] kube-system/kindnet failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kindnet", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"eec9b228-a151-4529-98e4-04791392653d", ResourceVersion:"271", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63862692104, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"},\"name\":\"kindnet\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"k
indnet\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"HOST_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.hostIP\"}}},{\"name\":\"POD_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"POD_SUBNET\",\"value\":\"10.244.0.0/16\"}],\"image\":\"docker.io/kindest/kindnetd:v20240813-c6f155d6\",\"name\":\"kindnet-cni\",\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"}},\"securityContext\":{\"capabilities\":{\"add\":[\"NET_RAW\",\"NET_ADMIN\"]},\"privileged\":false},\"volumeMounts\":[{\"mountPath\":\"/etc/cni/net.d\",\"name\":\"cni-cfg\"},{\"mountPath\":\"/run/xtables.lock\",\"name\":\"xtables-lock\",\"readOnly\":false},{\"mountPath\":\"/lib/modules\",\"name\":\"lib-modules\",\"readOnly\":true}]}],\"hostNetwork\":true,\"serviceAccountName\":\"kindnet\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"operator\
":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/etc/cni/net.d\",\"type\":\"DirectoryOrCreate\"},\"name\":\"cni-cfg\"},{\"hostPath\":{\"path\":\"/run/xtables.lock\",\"type\":\"FileOrCreate\"},\"name\":\"xtables-lock\"},{\"hostPath\":{\"path\":\"/lib/modules\"},\"name\":\"lib-modules\"}]}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x4001d73b20), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x4001d73b40)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0x4001d73b60), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string
{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"cni-cfg", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001d73b80), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil),
FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001d73ba0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Glust
erfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x4001d73bc0), EmptyDi
r:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil),
PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kindnet-cni", Image:"docker.io/kindest/kindnetd:v20240813-c6f155d6", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"HOST_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001d73be0)}, v1.EnvVar{Name:"POD_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x4001d73c20)}, v1.EnvVar{Name:"POD_SUBNET", Value:"10.244.0.0/16", ValueFrom:(*v1.EnvVarSource)(nil)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:
0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"cni-cfg", ReadOnly:false, MountPath:"/etc/cni/net.d", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:
(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x4001ceb5c0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0x4001d8ab48), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"kindnet", DeprecatedServiceAccount:"kindnet", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x40001fb730), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}},
HostAliases:[]v1.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x40005b0190)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x4001d8ab90)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kindnet": the object has been modified; please apply your changes to the latest version and try again
I0923 12:41:59.814714 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-pb95z"
I0923 12:41:59.822501 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0923 12:42:00.123470 1 shared_informer.go:247] Caches are synced for garbage collector
I0923 12:42:00.141242 1 shared_informer.go:247] Caches are synced for garbage collector
I0923 12:42:00.141267 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0923 12:42:00.983979 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0923 12:42:00.998936 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-kz5zm"
I0923 12:43:46.153207 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0923 12:43:46.247420 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
E0923 12:43:46.278947 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [3613dc571cef58ead6749eb2383e0e79ed8c48eb15efb4bba28cf918e2dd3efb] <==
I0923 12:44:33.141066 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0923 12:44:33.141617 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0923 12:44:33.167328 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0923 12:44:33.167616 1 server_others.go:185] Using iptables Proxier.
I0923 12:44:33.168266 1 server.go:650] Version: v1.20.0
I0923 12:44:33.169949 1 config.go:224] Starting endpoint slice config controller
I0923 12:44:33.170110 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0923 12:44:33.171048 1 config.go:315] Starting service config controller
I0923 12:44:33.171206 1 shared_informer.go:240] Waiting for caches to sync for service config
I0923 12:44:33.270384 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0923 12:44:33.271538 1 shared_informer.go:247] Caches are synced for service config
==> kube-proxy [c01260402717934e0fda25d9ba74c52f11b9fe853dc5b4c12f80f8799ca4f2a8] <==
I0923 12:42:01.298737 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0923 12:42:01.299032 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0923 12:42:01.318244 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0923 12:42:01.318585 1 server_others.go:185] Using iptables Proxier.
I0923 12:42:01.319008 1 server.go:650] Version: v1.20.0
I0923 12:42:01.319734 1 config.go:315] Starting service config controller
I0923 12:42:01.319882 1 shared_informer.go:240] Waiting for caches to sync for service config
I0923 12:42:01.321011 1 config.go:224] Starting endpoint slice config controller
I0923 12:42:01.321166 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0923 12:42:01.422004 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0923 12:42:01.422047 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [2580add0e36cca9407c4890d5578a3ecf589374f987138526bd939a45122e8ef] <==
W0923 12:41:41.067431 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0923 12:41:41.067529 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0923 12:41:41.067558 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0923 12:41:41.068028 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0923 12:41:41.153548 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0923 12:41:41.153697 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0923 12:41:41.154501 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0923 12:41:41.153487 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0923 12:41:41.185116 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0923 12:41:41.185451 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0923 12:41:41.185685 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0923 12:41:41.186065 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0923 12:41:41.186332 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0923 12:41:41.186393 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0923 12:41:41.186443 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0923 12:41:41.186508 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0923 12:41:41.186562 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0923 12:41:41.186616 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0923 12:41:41.186664 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0923 12:41:41.212943 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0923 12:41:42.049998 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0923 12:41:42.074388 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0923 12:41:42.210271 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0923 12:41:42.256570 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
I0923 12:41:44.553946 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [df9e1754aa7176c9aa5049e2d7574a5dfd7dd21b02f14db6d9f5a24905f3cd89] <==
I0923 12:44:24.980691 1 serving.go:331] Generated self-signed cert in-memory
W0923 12:44:29.046070 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0923 12:44:29.046102 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0923 12:44:29.046120 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0923 12:44:29.046126 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0923 12:44:29.307207 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0923 12:44:29.307243 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0923 12:44:29.312676 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0923 12:44:29.312764 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0923 12:44:29.409861 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Sep 23 12:48:49 old-k8s-version-605045 kubelet[658]: E0923 12:48:49.466027 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:48:55 old-k8s-version-605045 kubelet[658]: E0923 12:48:55.466326 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: I0923 12:49:04.465204 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:49:04 old-k8s-version-605045 kubelet[658]: E0923 12:49:04.465573 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:07 old-k8s-version-605045 kubelet[658]: E0923 12:49:07.466660 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: I0923 12:49:16.465209 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:49:16 old-k8s-version-605045 kubelet[658]: E0923 12:49:16.465577 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:20 old-k8s-version-605045 kubelet[658]: E0923 12:49:20.466093 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: I0923 12:49:29.465178 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:49:29 old-k8s-version-605045 kubelet[658]: E0923 12:49:29.465520 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:32 old-k8s-version-605045 kubelet[658]: E0923 12:49:32.465885 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: I0923 12:49:41.465312 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:49:41 old-k8s-version-605045 kubelet[658]: E0923 12:49:41.466162 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:49:47 old-k8s-version-605045 kubelet[658]: E0923 12:49:47.469587 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: I0923 12:49:52.465139 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:49:52 old-k8s-version-605045 kubelet[658]: E0923 12:49:52.465952 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.485787 658 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.485859 658 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486001 658 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-7ztnm,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-kf958_kube-system(c86555f
f-2cba-4287-abf0-8093aab3c29a): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Sep 23 12:50:01 old-k8s-version-605045 kubelet[658]: E0923 12:50:01.486035 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: I0923 12:50:04.465207 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:50:04 old-k8s-version-605045 kubelet[658]: E0923 12:50:04.465610 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
Sep 23 12:50:13 old-k8s-version-605045 kubelet[658]: E0923 12:50:13.466331 658 pod_workers.go:191] Error syncing pod c86555ff-2cba-4287-abf0-8093aab3c29a ("metrics-server-9975d5f86-kf958_kube-system(c86555ff-2cba-4287-abf0-8093aab3c29a)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Sep 23 12:50:18 old-k8s-version-605045 kubelet[658]: I0923 12:50:18.465180 658 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4e7193d4b39bc95a722acd825f1ddd864fbef394ccf127e0f216622d16d908fe
Sep 23 12:50:18 old-k8s-version-605045 kubelet[658]: E0923 12:50:18.465597 658 pod_workers.go:191] Error syncing pod 084a43ab-5137-4d1a-a05b-0a7ed25eae52 ("dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-7btzn_kubernetes-dashboard(084a43ab-5137-4d1a-a05b-0a7ed25eae52)"
==> kubernetes-dashboard [7f77123376f589c236b87db48d11c0941e9837cae0912c199ba429f76e31629a] <==
2024/09/23 12:44:56 Using namespace: kubernetes-dashboard
2024/09/23 12:44:56 Using in-cluster config to connect to apiserver
2024/09/23 12:44:56 Using secret token for csrf signing
2024/09/23 12:44:56 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/09/23 12:44:56 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/09/23 12:44:56 Successful initial request to the apiserver, version: v1.20.0
2024/09/23 12:44:56 Generating JWE encryption key
2024/09/23 12:44:56 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/09/23 12:44:56 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/09/23 12:44:56 Initializing JWE encryption key from synchronized object
2024/09/23 12:44:56 Creating in-cluster Sidecar client
2024/09/23 12:44:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:44:57 Serving insecurely on HTTP port: 9090
2024/09/23 12:45:27 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:45:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:46:27 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:46:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:47:27 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:47:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:48:27 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:48:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:49:27 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:49:57 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/09/23 12:44:56 Starting overwatch
==> storage-provisioner [036c3b8487f9041f4dfadb532e6ec45639c4d42f55a3699697f42f3e22679e19] <==
I0923 12:45:17.656246 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0923 12:45:17.675896 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0923 12:45:17.675989 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0923 12:45:35.201884 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0923 12:45:35.205379 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-605045_2727ca76-e3d8-4a29-89d6-75da74e93141!
I0923 12:45:35.206648 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"ea175063-3427-4c47-b310-1c3ee60b6be2", APIVersion:"v1", ResourceVersion:"846", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-605045_2727ca76-e3d8-4a29-89d6-75da74e93141 became leader
I0923 12:45:35.319763 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-605045_2727ca76-e3d8-4a29-89d6-75da74e93141!
==> storage-provisioner [0c783388fb5375603ae87338a5bc59d00ab3b4a8029fe8ef41c7f1d31e3b1649] <==
I0923 12:44:33.205650 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0923 12:45:03.207873 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-605045 -n old-k8s-version-605045
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-605045 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-kf958
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-605045 describe pod metrics-server-9975d5f86-kf958
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-605045 describe pod metrics-server-9975d5f86-kf958: exit status 1 (139.899329ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-kf958" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-605045 describe pod metrics-server-9975d5f86-kf958: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (379.77s)