=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-530457 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-530457 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0: exit status 102 (6m15.881768033s)
-- stdout --
* [old-k8s-version-530457] minikube v1.33.1 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19529
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19529-440552/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19529-440552/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
* Using the docker driver based on existing profile
* Starting "old-k8s-version-530457" primary control-plane node in "old-k8s-version-530457" cluster
* Pulling base image v0.0.44-1724775115-19521 ...
* Restarting existing docker container for "old-k8s-version-530457" ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-530457 addons enable metrics-server
* Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
-- /stdout --
** stderr **
I0828 19:03:52.878763 635612 out.go:345] Setting OutFile to fd 1 ...
I0828 19:03:52.883015 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:03:52.883044 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:03:52.883063 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:03:52.883382 635612 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19529-440552/.minikube/bin
I0828 19:03:52.883841 635612 out.go:352] Setting JSON to false
I0828 19:03:52.884911 635612 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":9981,"bootTime":1724861852,"procs":212,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1068-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0828 19:03:52.885010 635612 start.go:139] virtualization:
I0828 19:03:52.890152 635612 out.go:177] * [old-k8s-version-530457] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0828 19:03:52.892794 635612 out.go:177] - MINIKUBE_LOCATION=19529
I0828 19:03:52.892855 635612 notify.go:220] Checking for updates...
I0828 19:03:52.898500 635612 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0828 19:03:52.900960 635612 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:03:52.903879 635612 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19529-440552/.minikube
I0828 19:03:52.913400 635612 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0828 19:03:52.917023 635612 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0828 19:03:52.939956 635612 config.go:182] Loaded profile config "old-k8s-version-530457": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0828 19:03:52.948149 635612 out.go:177] * Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
I0828 19:03:52.951199 635612 driver.go:392] Setting default libvirt URI to qemu:///system
I0828 19:03:53.005432 635612 docker.go:123] docker version: linux-27.2.0:Docker Engine - Community
I0828 19:03:53.005562 635612 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0828 19:03:53.119145 635612 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:34 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-28 19:03:53.105942752 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.2.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:472731909fa34bd7bc9c087e4c27943f9835f111 Expected:472731909fa34bd7bc9c087e4c27943f9835f111} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0828 19:03:53.119257 635612 docker.go:307] overlay module found
I0828 19:03:53.122252 635612 out.go:177] * Using the docker driver based on existing profile
I0828 19:03:53.124942 635612 start.go:297] selected driver: docker
I0828 19:03:53.124962 635612 start.go:901] validating driver "docker" against &{Name:old-k8s-version-530457 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-530457 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountStr
ing:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:03:53.125076 635612 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0828 19:03:53.125842 635612 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0828 19:03:53.233978 635612 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:34 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-28 19:03:53.223865302 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.2.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:472731909fa34bd7bc9c087e4c27943f9835f111 Expected:472731909fa34bd7bc9c087e4c27943f9835f111} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0828 19:03:53.234307 635612 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0828 19:03:53.234328 635612 cni.go:84] Creating CNI manager for ""
I0828 19:03:53.234343 635612 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0828 19:03:53.234381 635612 start.go:340] cluster config:
{Name:old-k8s-version-530457 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-530457 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP
: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:03:53.237576 635612 out.go:177] * Starting "old-k8s-version-530457" primary control-plane node in "old-k8s-version-530457" cluster
I0828 19:03:53.240400 635612 cache.go:121] Beginning downloading kic base image for docker with crio
I0828 19:03:53.243547 635612 out.go:177] * Pulling base image v0.0.44-1724775115-19521 ...
I0828 19:03:53.246436 635612 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0828 19:03:53.246496 635612 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19529-440552/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4
I0828 19:03:53.246506 635612 cache.go:56] Caching tarball of preloaded images
I0828 19:03:53.246588 635612 preload.go:172] Found /home/jenkins/minikube-integration/19529-440552/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4 in cache, skipping download
I0828 19:03:53.246596 635612 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0828 19:03:53.246724 635612 profile.go:143] Saving config to /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/config.json ...
I0828 19:03:53.246927 635612 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local docker daemon
W0828 19:03:53.271682 635612 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce is of wrong architecture
I0828 19:03:53.271699 635612 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce to local cache
I0828 19:03:53.271767 635612 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local cache directory
I0828 19:03:53.271783 635612 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local cache directory, skipping pull
I0828 19:03:53.271788 635612 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce exists in cache, skipping pull
I0828 19:03:53.271795 635612 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce as a tarball
I0828 19:03:53.271800 635612 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce from local cache
I0828 19:03:53.409280 635612 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce from cached tarball
I0828 19:03:53.409321 635612 cache.go:194] Successfully downloaded all kic artifacts
I0828 19:03:53.409362 635612 start.go:360] acquireMachinesLock for old-k8s-version-530457: {Name:mk98f808c4afb30d67a45ce32c13e9cadedca9d2 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:03:53.409427 635612 start.go:364] duration metric: took 38.703µs to acquireMachinesLock for "old-k8s-version-530457"
I0828 19:03:53.409446 635612 start.go:96] Skipping create...Using existing machine configuration
I0828 19:03:53.409452 635612 fix.go:54] fixHost starting:
I0828 19:03:53.409734 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:03:53.437414 635612 fix.go:112] recreateIfNeeded on old-k8s-version-530457: state=Stopped err=<nil>
W0828 19:03:53.437442 635612 fix.go:138] unexpected machine state, will restart: <nil>
I0828 19:03:53.440792 635612 out.go:177] * Restarting existing docker container for "old-k8s-version-530457" ...
I0828 19:03:53.443581 635612 cli_runner.go:164] Run: docker start old-k8s-version-530457
I0828 19:03:53.858823 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:03:53.888019 635612 kic.go:430] container "old-k8s-version-530457" state is running.
I0828 19:03:53.888389 635612 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-530457
I0828 19:03:53.915505 635612 profile.go:143] Saving config to /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/config.json ...
I0828 19:03:53.915720 635612 machine.go:93] provisionDockerMachine start ...
I0828 19:03:53.915785 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:53.950058 635612 main.go:141] libmachine: Using SSH client type: native
I0828 19:03:53.950325 635612 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33450 <nil> <nil>}
I0828 19:03:53.950337 635612 main.go:141] libmachine: About to run SSH command:
hostname
I0828 19:03:53.951030 635612 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:59054->127.0.0.1:33450: read: connection reset by peer
I0828 19:03:57.108923 635612 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-530457
I0828 19:03:57.108963 635612 ubuntu.go:169] provisioning hostname "old-k8s-version-530457"
I0828 19:03:57.109042 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:57.140902 635612 main.go:141] libmachine: Using SSH client type: native
I0828 19:03:57.141203 635612 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33450 <nil> <nil>}
I0828 19:03:57.141218 635612 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-530457 && echo "old-k8s-version-530457" | sudo tee /etc/hostname
I0828 19:03:57.314913 635612 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-530457
I0828 19:03:57.315068 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:57.349544 635612 main.go:141] libmachine: Using SSH client type: native
I0828 19:03:57.349783 635612 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33450 <nil> <nil>}
I0828 19:03:57.349800 635612 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-530457' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-530457/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-530457' | sudo tee -a /etc/hosts;
fi
fi
I0828 19:03:57.497233 635612 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0828 19:03:57.497260 635612 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19529-440552/.minikube CaCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19529-440552/.minikube}
I0828 19:03:57.497279 635612 ubuntu.go:177] setting up certificates
I0828 19:03:57.497289 635612 provision.go:84] configureAuth start
I0828 19:03:57.497362 635612 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-530457
I0828 19:03:57.522911 635612 provision.go:143] copyHostCerts
I0828 19:03:57.522992 635612 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem, removing ...
I0828 19:03:57.523016 635612 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem
I0828 19:03:57.523099 635612 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem (1679 bytes)
I0828 19:03:57.523200 635612 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem, removing ...
I0828 19:03:57.523209 635612 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem
I0828 19:03:57.523235 635612 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem (1082 bytes)
I0828 19:03:57.523299 635612 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem, removing ...
I0828 19:03:57.523310 635612 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem
I0828 19:03:57.523339 635612 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem (1123 bytes)
I0828 19:03:57.523400 635612 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-530457 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-530457]
I0828 19:03:57.907907 635612 provision.go:177] copyRemoteCerts
I0828 19:03:57.907976 635612 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0828 19:03:57.908022 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:57.925251 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:03:58.033552 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0828 19:03:58.075943 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0828 19:03:58.128185 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0828 19:03:58.175490 635612 provision.go:87] duration metric: took 678.187148ms to configureAuth
I0828 19:03:58.175529 635612 ubuntu.go:193] setting minikube options for container-runtime
I0828 19:03:58.175754 635612 config.go:182] Loaded profile config "old-k8s-version-530457": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0828 19:03:58.175871 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:58.209398 635612 main.go:141] libmachine: Using SSH client type: native
I0828 19:03:58.209646 635612 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33450 <nil> <nil>}
I0828 19:03:58.209662 635612 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0828 19:03:58.670874 635612 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0828 19:03:58.670962 635612 machine.go:96] duration metric: took 4.755232508s to provisionDockerMachine
I0828 19:03:58.670991 635612 start.go:293] postStartSetup for "old-k8s-version-530457" (driver="docker")
I0828 19:03:58.671033 635612 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0828 19:03:58.671118 635612 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0828 19:03:58.671186 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:58.692404 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:03:58.822903 635612 ssh_runner.go:195] Run: cat /etc/os-release
I0828 19:03:58.828821 635612 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0828 19:03:58.828860 635612 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0828 19:03:58.828872 635612 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0828 19:03:58.828879 635612 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0828 19:03:58.828890 635612 filesync.go:126] Scanning /home/jenkins/minikube-integration/19529-440552/.minikube/addons for local assets ...
I0828 19:03:58.828948 635612 filesync.go:126] Scanning /home/jenkins/minikube-integration/19529-440552/.minikube/files for local assets ...
I0828 19:03:58.829033 635612 filesync.go:149] local asset: /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem -> 4459402.pem in /etc/ssl/certs
I0828 19:03:58.829144 635612 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0828 19:03:58.838886 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem --> /etc/ssl/certs/4459402.pem (1708 bytes)
I0828 19:03:58.874162 635612 start.go:296] duration metric: took 203.127095ms for postStartSetup
I0828 19:03:58.874257 635612 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0828 19:03:58.874303 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:58.915088 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:03:59.014017 635612 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0828 19:03:59.019502 635612 fix.go:56] duration metric: took 5.610040969s for fixHost
I0828 19:03:59.019525 635612 start.go:83] releasing machines lock for "old-k8s-version-530457", held for 5.610090026s
I0828 19:03:59.019594 635612 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-530457
I0828 19:03:59.051920 635612 ssh_runner.go:195] Run: cat /version.json
I0828 19:03:59.051978 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:59.052235 635612 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0828 19:03:59.052284 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:03:59.089407 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:03:59.094182 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:03:59.200983 635612 ssh_runner.go:195] Run: systemctl --version
I0828 19:03:59.382437 635612 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0828 19:03:59.571042 635612 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0828 19:03:59.575847 635612 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0828 19:03:59.585373 635612 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0828 19:03:59.585457 635612 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0828 19:03:59.597189 635612 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0828 19:03:59.597216 635612 start.go:495] detecting cgroup driver to use...
I0828 19:03:59.597260 635612 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0828 19:03:59.597326 635612 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0828 19:03:59.613221 635612 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0828 19:03:59.628689 635612 docker.go:217] disabling cri-docker service (if available) ...
I0828 19:03:59.628766 635612 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0828 19:03:59.646248 635612 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0828 19:03:59.662724 635612 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0828 19:03:59.802359 635612 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0828 19:03:59.942951 635612 docker.go:233] disabling docker service ...
I0828 19:03:59.943047 635612 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0828 19:03:59.961262 635612 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0828 19:03:59.976021 635612 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0828 19:04:00.134254 635612 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0828 19:04:00.332284 635612 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0828 19:04:00.352329 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0828 19:04:00.377392 635612 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0828 19:04:00.377489 635612 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:04:00.392535 635612 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0828 19:04:00.392633 635612 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:04:00.408805 635612 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:04:00.431528 635612 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:04:00.453653 635612 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0828 19:04:00.473597 635612 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0828 19:04:00.484466 635612 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0828 19:04:00.497405 635612 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:04:00.665821 635612 ssh_runner.go:195] Run: sudo systemctl restart crio
I0828 19:04:00.875707 635612 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0828 19:04:00.875791 635612 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0828 19:04:00.880055 635612 start.go:563] Will wait 60s for crictl version
I0828 19:04:00.880148 635612 ssh_runner.go:195] Run: which crictl
I0828 19:04:00.885984 635612 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0828 19:04:00.959579 635612 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0828 19:04:00.959736 635612 ssh_runner.go:195] Run: crio --version
I0828 19:04:01.051396 635612 ssh_runner.go:195] Run: crio --version
I0828 19:04:01.120868 635612 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
I0828 19:04:01.123701 635612 cli_runner.go:164] Run: docker network inspect old-k8s-version-530457 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0828 19:04:01.146328 635612 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0828 19:04:01.151158 635612 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0828 19:04:01.169215 635612 kubeadm.go:883] updating cluster {Name:old-k8s-version-530457 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-530457 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0828 19:04:01.169439 635612 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0828 19:04:01.169514 635612 ssh_runner.go:195] Run: sudo crictl images --output json
I0828 19:04:01.258440 635612 crio.go:514] all images are preloaded for cri-o runtime.
I0828 19:04:01.258509 635612 crio.go:433] Images already preloaded, skipping extraction
I0828 19:04:01.258618 635612 ssh_runner.go:195] Run: sudo crictl images --output json
I0828 19:04:01.335417 635612 crio.go:514] all images are preloaded for cri-o runtime.
I0828 19:04:01.335438 635612 cache_images.go:84] Images are preloaded, skipping loading
I0828 19:04:01.335450 635612 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.20.0 crio true true} ...
I0828 19:04:01.335573 635612 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --enforce-node-allocatable= --hostname-override=old-k8s-version-530457 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-530457 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0828 19:04:01.335653 635612 ssh_runner.go:195] Run: crio config
I0828 19:04:01.450495 635612 cni.go:84] Creating CNI manager for ""
I0828 19:04:01.450622 635612 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0828 19:04:01.450677 635612 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0828 19:04:01.450726 635612 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-530457 NodeName:old-k8s-version-530457 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0828 19:04:01.450982 635612 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-530457"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0828 19:04:01.451073 635612 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0828 19:04:01.461446 635612 binaries.go:44] Found k8s binaries, skipping transfer
I0828 19:04:01.461596 635612 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0828 19:04:01.471377 635612 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (480 bytes)
I0828 19:04:01.506690 635612 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0828 19:04:01.532116 635612 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2117 bytes)
I0828 19:04:01.555190 635612 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0828 19:04:01.560154 635612 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0828 19:04:01.577879 635612 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:04:01.769232 635612 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0828 19:04:01.809546 635612 certs.go:68] Setting up /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457 for IP: 192.168.76.2
I0828 19:04:01.809597 635612 certs.go:194] generating shared ca certs ...
I0828 19:04:01.809615 635612 certs.go:226] acquiring lock for ca certs: {Name:mk3dbd0c4288cf1e03eeb5b77f9120790fc1f5e6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:04:01.809760 635612 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19529-440552/.minikube/ca.key
I0828 19:04:01.809802 635612 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.key
I0828 19:04:01.809809 635612 certs.go:256] generating profile certs ...
I0828 19:04:01.809894 635612 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/client.key
I0828 19:04:01.809971 635612 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/apiserver.key.a33dbf87
I0828 19:04:01.810017 635612 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/proxy-client.key
I0828 19:04:01.810133 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940.pem (1338 bytes)
W0828 19:04:01.810176 635612 certs.go:480] ignoring /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940_empty.pem, impossibly tiny 0 bytes
I0828 19:04:01.810191 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem (1675 bytes)
I0828 19:04:01.810217 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem (1082 bytes)
I0828 19:04:01.810240 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem (1123 bytes)
I0828 19:04:01.810265 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem (1679 bytes)
I0828 19:04:01.810306 635612 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem (1708 bytes)
I0828 19:04:01.811033 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0828 19:04:01.862240 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0828 19:04:01.942032 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0828 19:04:01.998365 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0828 19:04:02.052237 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0828 19:04:02.090925 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0828 19:04:02.125459 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0828 19:04:02.155399 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/old-k8s-version-530457/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0828 19:04:02.193164 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940.pem --> /usr/share/ca-certificates/445940.pem (1338 bytes)
I0828 19:04:02.250014 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem --> /usr/share/ca-certificates/4459402.pem (1708 bytes)
I0828 19:04:02.289286 635612 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0828 19:04:02.314400 635612 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0828 19:04:02.342456 635612 ssh_runner.go:195] Run: openssl version
I0828 19:04:02.352846 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/445940.pem && ln -fs /usr/share/ca-certificates/445940.pem /etc/ssl/certs/445940.pem"
I0828 19:04:02.365376 635612 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/445940.pem
I0828 19:04:02.369375 635612 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 28 18:21 /usr/share/ca-certificates/445940.pem
I0828 19:04:02.369440 635612 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/445940.pem
I0828 19:04:02.378500 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/445940.pem /etc/ssl/certs/51391683.0"
I0828 19:04:02.387605 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4459402.pem && ln -fs /usr/share/ca-certificates/4459402.pem /etc/ssl/certs/4459402.pem"
I0828 19:04:02.403098 635612 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4459402.pem
I0828 19:04:02.407417 635612 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 28 18:21 /usr/share/ca-certificates/4459402.pem
I0828 19:04:02.407480 635612 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4459402.pem
I0828 19:04:02.415251 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4459402.pem /etc/ssl/certs/3ec20f2e.0"
I0828 19:04:02.425677 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0828 19:04:02.438159 635612 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0828 19:04:02.442470 635612 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 28 18:03 /usr/share/ca-certificates/minikubeCA.pem
I0828 19:04:02.442542 635612 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0828 19:04:02.449472 635612 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0828 19:04:02.458635 635612 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0828 19:04:02.462150 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0828 19:04:02.469032 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0828 19:04:02.478842 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0828 19:04:02.485574 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0828 19:04:02.492307 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0828 19:04:02.503162 635612 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0828 19:04:02.511240 635612 kubeadm.go:392] StartCluster: {Name:old-k8s-version-530457 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-530457 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/m
inikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:04:02.511344 635612 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0828 19:04:02.511403 635612 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0828 19:04:02.575135 635612 cri.go:89] found id: ""
I0828 19:04:02.575208 635612 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0828 19:04:02.587604 635612 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0828 19:04:02.587624 635612 kubeadm.go:593] restartPrimaryControlPlane start ...
I0828 19:04:02.587676 635612 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0828 19:04:02.601258 635612 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0828 19:04:02.601922 635612 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-530457" does not appear in /home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:04:02.602182 635612 kubeconfig.go:62] /home/jenkins/minikube-integration/19529-440552/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-530457" cluster setting kubeconfig missing "old-k8s-version-530457" context setting]
I0828 19:04:02.602649 635612 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19529-440552/kubeconfig: {Name:mk2bbb990ac0e5b155d3e1ee34adea25cdbf73a4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:04:02.604020 635612 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0828 19:04:02.614058 635612 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0828 19:04:02.614093 635612 kubeadm.go:597] duration metric: took 26.463165ms to restartPrimaryControlPlane
I0828 19:04:02.614103 635612 kubeadm.go:394] duration metric: took 102.875432ms to StartCluster
I0828 19:04:02.614118 635612 settings.go:142] acquiring lock: {Name:mk3578bce439064a3b256aa0408671abd470922d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:04:02.614174 635612 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:04:02.615197 635612 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19529-440552/kubeconfig: {Name:mk2bbb990ac0e5b155d3e1ee34adea25cdbf73a4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:04:02.615411 635612 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0828 19:04:02.615752 635612 config.go:182] Loaded profile config "old-k8s-version-530457": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0828 19:04:02.615775 635612 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0828 19:04:02.615946 635612 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-530457"
I0828 19:04:02.616009 635612 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-530457"
W0828 19:04:02.616034 635612 addons.go:243] addon storage-provisioner should already be in state true
I0828 19:04:02.616094 635612 host.go:66] Checking if "old-k8s-version-530457" exists ...
I0828 19:04:02.615950 635612 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-530457"
I0828 19:04:02.616156 635612 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-530457"
I0828 19:04:02.616600 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:04:02.616699 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:04:02.615956 635612 addons.go:69] Setting dashboard=true in profile "old-k8s-version-530457"
I0828 19:04:02.617150 635612 addons.go:234] Setting addon dashboard=true in "old-k8s-version-530457"
W0828 19:04:02.617159 635612 addons.go:243] addon dashboard should already be in state true
I0828 19:04:02.617186 635612 host.go:66] Checking if "old-k8s-version-530457" exists ...
I0828 19:04:02.618004 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:04:02.623049 635612 out.go:177] * Verifying Kubernetes components...
I0828 19:04:02.615962 635612 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-530457"
I0828 19:04:02.625447 635612 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-530457"
W0828 19:04:02.625460 635612 addons.go:243] addon metrics-server should already be in state true
I0828 19:04:02.625493 635612 host.go:66] Checking if "old-k8s-version-530457" exists ...
I0828 19:04:02.626022 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:04:02.627450 635612 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:04:02.687188 635612 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-530457"
W0828 19:04:02.687210 635612 addons.go:243] addon default-storageclass should already be in state true
I0828 19:04:02.687236 635612 host.go:66] Checking if "old-k8s-version-530457" exists ...
I0828 19:04:02.687648 635612 cli_runner.go:164] Run: docker container inspect old-k8s-version-530457 --format={{.State.Status}}
I0828 19:04:02.704569 635612 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0828 19:04:02.706269 635612 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0828 19:04:02.707385 635612 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:04:02.707405 635612 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0828 19:04:02.707468 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:04:02.710802 635612 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0828 19:04:02.715304 635612 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0828 19:04:02.715329 635612 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0828 19:04:02.715397 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:04:02.715570 635612 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0828 19:04:02.721543 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0828 19:04:02.721570 635612 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0828 19:04:02.721635 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:04:02.754905 635612 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0828 19:04:02.754931 635612 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0828 19:04:02.754999 635612 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-530457
I0828 19:04:02.788761 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:04:02.789121 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:04:02.790873 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:04:02.813258 635612 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33450 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/old-k8s-version-530457/id_rsa Username:docker}
I0828 19:04:02.955344 635612 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0828 19:04:02.980342 635612 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-530457" to be "Ready" ...
I0828 19:04:02.983895 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:04:03.021135 635612 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0828 19:04:03.021206 635612 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0828 19:04:03.042414 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0828 19:04:03.042491 635612 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0828 19:04:03.066249 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:04:03.089705 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0828 19:04:03.089727 635612 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0828 19:04:03.102346 635612 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0828 19:04:03.102367 635612 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0828 19:04:03.162265 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0828 19:04:03.162288 635612 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0828 19:04:03.191332 635612 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:04:03.191405 635612 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0828 19:04:03.251509 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:04:03.273622 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0828 19:04:03.273695 635612 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0828 19:04:03.301661 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.301759 635612 retry.go:31] will retry after 159.719681ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:03.315178 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.315206 635612 retry.go:31] will retry after 139.330126ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.370827 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0828 19:04:03.370851 635612 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0828 19:04:03.398101 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0828 19:04:03.398122 635612 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0828 19:04:03.408981 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.409010 635612 retry.go:31] will retry after 151.35848ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.429001 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0828 19:04:03.429027 635612 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0828 19:04:03.455097 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:04:03.456646 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0828 19:04:03.456663 635612 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0828 19:04:03.463309 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:04:03.514476 635612 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:04:03.514499 635612 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0828 19:04:03.561330 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:04:03.598648 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0828 19:04:03.717192 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.717224 635612 retry.go:31] will retry after 362.99559ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:03.809045 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.809078 635612 retry.go:31] will retry after 329.552237ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:03.890975 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.891006 635612 retry.go:31] will retry after 554.305577ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:03.891051 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:03.891062 635612 retry.go:31] will retry after 300.432977ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.080931 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:04:04.140269 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:04.180032 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.180064 635612 retry.go:31] will retry after 511.767846ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.192379 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0828 19:04:04.318865 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.318901 635612 retry.go:31] will retry after 793.937957ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:04.379375 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.379409 635612 retry.go:31] will retry after 236.790595ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.445457 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0828 19:04:04.537700 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.537730 635612 retry.go:31] will retry after 442.55779ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.617127 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:04:04.692411 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0828 19:04:04.728179 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.728210 635612 retry.go:31] will retry after 352.200134ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:04.831089 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.831134 635612 retry.go:31] will retry after 555.971845ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:04.981332 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:04:04.981722 635612 node_ready.go:53] error getting node "old-k8s-version-530457": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-530457": dial tcp 192.168.76.2:8443: connect: connection refused
I0828 19:04:05.081482 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:04:05.113410 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:05.181468 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:05.181496 635612 retry.go:31] will retry after 971.663524ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:05.354820 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:05.354848 635612 retry.go:31] will retry after 1.136096454s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:05.354891 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:05.354898 635612 retry.go:31] will retry after 1.229234391s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:05.388075 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0828 19:04:05.517533 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:05.517562 635612 retry.go:31] will retry after 1.433921635s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.153455 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0828 19:04:06.330363 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.330393 635612 retry.go:31] will retry after 792.664477ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.491355 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:04:06.584779 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:06.709833 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.709863 635612 retry.go:31] will retry after 1.107513175s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0828 19:04:06.862484 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.862518 635612 retry.go:31] will retry after 1.098470017s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:06.951720 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0828 19:04:07.050891 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.050927 635612 retry.go:31] will retry after 1.311869745s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.124273 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0828 19:04:07.237419 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.237461 635612 retry.go:31] will retry after 2.811336854s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.481619 635612 node_ready.go:53] error getting node "old-k8s-version-530457": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-530457": dial tcp 192.168.76.2:8443: connect: connection refused
I0828 19:04:07.818247 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0828 19:04:07.893414 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.893445 635612 retry.go:31] will retry after 2.459002632s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:07.961642 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:08.042237 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:08.042270 635612 retry.go:31] will retry after 1.182137427s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:08.363757 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0828 19:04:08.446679 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:08.446715 635612 retry.go:31] will retry after 3.786441276s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:09.224617 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:09.291670 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:09.291702 635612 retry.go:31] will retry after 2.090165444s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:09.981149 635612 node_ready.go:53] error getting node "old-k8s-version-530457": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-530457": dial tcp 192.168.76.2:8443: connect: connection refused
I0828 19:04:10.049573 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0828 19:04:10.128990 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:10.129026 635612 retry.go:31] will retry after 2.708064572s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:10.353498 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0828 19:04:10.551325 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:10.551354 635612 retry.go:31] will retry after 4.1760764s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:11.382499 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0828 19:04:11.564475 635612 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:11.564505 635612 retry.go:31] will retry after 2.849837857s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0828 19:04:12.233547 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:04:12.837627 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:04:14.414532 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:04:14.727877 635612 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:04:21.075519 635612 node_ready.go:49] node "old-k8s-version-530457" has status "Ready":"True"
I0828 19:04:21.075547 635612 node_ready.go:38] duration metric: took 18.095123942s for node "old-k8s-version-530457" to be "Ready" ...
I0828 19:04:21.075558 635612 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0828 19:04:21.462961 635612 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-z6sb7" in "kube-system" namespace to be "Ready" ...
I0828 19:04:21.725595 635612 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (9.49200147s)
I0828 19:04:22.059324 635612 pod_ready.go:93] pod "coredns-74ff55c5b-z6sb7" in "kube-system" namespace has status "Ready":"True"
I0828 19:04:22.059398 635612 pod_ready.go:82] duration metric: took 596.360365ms for pod "coredns-74ff55c5b-z6sb7" in "kube-system" namespace to be "Ready" ...
I0828 19:04:22.059427 635612 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:04:22.300272 635612 pod_ready.go:93] pod "etcd-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:04:22.300348 635612 pod_ready.go:82] duration metric: took 240.899169ms for pod "etcd-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:04:22.300379 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:04:22.577530 635612 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (9.739858916s)
I0828 19:04:22.577567 635612 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-530457"
I0828 19:04:22.577620 635612 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.163067638s)
I0828 19:04:23.561804 635612 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (8.833877026s)
I0828 19:04:23.563540 635612 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-530457 addons enable metrics-server
I0828 19:04:23.565534 635612 out.go:177] * Enabled addons: default-storageclass, metrics-server, storage-provisioner, dashboard
I0828 19:04:23.567437 635612 addons.go:510] duration metric: took 20.951655982s for enable addons: enabled=[default-storageclass metrics-server storage-provisioner dashboard]
I0828 19:04:24.311170 635612 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:26.808887 635612 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:28.831658 635612 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:30.852138 635612 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:33.313247 635612 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:33.807930 635612 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:04:33.807958 635612 pod_ready.go:82] duration metric: took 11.507558138s for pod "kube-apiserver-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:04:33.807972 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:04:35.817389 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:37.833439 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:40.316295 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:42.317697 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:44.818108 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:46.822905 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:48.823652 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:50.846061 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:53.398075 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:55.818187 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:04:58.315811 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:00.331487 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:02.814455 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:05.315004 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:07.315619 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:09.815033 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:12.314749 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:14.813559 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:16.814930 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:19.315005 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:21.384470 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:23.826346 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:26.315132 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:28.318799 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:30.814334 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:32.815461 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:35.315325 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:37.316965 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:39.814131 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:41.814336 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:44.313663 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:45.814931 635612 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.814957 635612 pod_ready.go:82] duration metric: took 1m12.006976966s for pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.814970 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-mzrqc" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.820896 635612 pod_ready.go:93] pod "kube-proxy-mzrqc" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.820917 635612 pod_ready.go:82] duration metric: took 5.939148ms for pod "kube-proxy-mzrqc" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.820927 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.828415 635612 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.828506 635612 pod_ready.go:82] duration metric: took 7.56952ms for pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.828533 635612 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace to be "Ready" ...
I0828 19:05:47.835145 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:49.836051 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:51.836953 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:54.334874 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:56.335646 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:58.835270 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:00.835849 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:03.348129 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:05.834927 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:07.835375 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:10.335998 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:12.338041 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:14.841486 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:17.334506 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:19.834875 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:21.891228 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:24.335098 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:26.335545 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:28.840021 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:31.338658 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:33.836358 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:35.837036 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:37.846348 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:40.334278 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:42.336240 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:44.839981 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:46.842503 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:49.335849 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:51.841467 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:54.336015 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:56.836114 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:59.335150 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:01.339095 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:03.839487 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:06.335897 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:08.835302 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:10.847072 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:13.335296 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:15.335968 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:17.840577 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:20.334622 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:22.404395 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:24.841108 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:27.336643 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:29.844943 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:32.335817 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:34.838622 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:36.840754 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:39.334289 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:41.338500 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:43.838685 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:46.334910 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:48.335240 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:50.839720 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:52.845484 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:55.333808 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:57.335080 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:59.836333 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:02.336123 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:04.841753 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:07.337634 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:09.835428 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:11.837265 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:13.840664 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:16.334234 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:18.335444 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:20.840057 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:22.917136 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:25.337027 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:27.842024 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:29.843950 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:32.334667 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:34.334970 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:36.335232 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:38.840600 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:41.334802 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:43.338760 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:45.842787 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:48.335332 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:50.840584 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:52.841465 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:55.337126 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:57.840165 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:59.841202 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:01.844267 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:04.335030 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:06.335270 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:08.834703 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:10.835441 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:13.407484 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:15.846822 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:18.335367 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:20.840449 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:22.842273 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:25.335256 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:27.335969 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:29.839992 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:31.840538 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:34.335033 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:36.836165 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:38.840731 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:41.335746 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:43.839360 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:45.840744 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:45.840772 635612 pod_ready.go:82] duration metric: took 4m0.012205772s for pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace to be "Ready" ...
E0828 19:09:45.840793 635612 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0828 19:09:45.840804 635612 pod_ready.go:39] duration metric: took 5m24.765235278s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0828 19:09:45.840845 635612 api_server.go:52] waiting for apiserver process to appear ...
I0828 19:09:45.840891 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:09:45.840962 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:09:45.887113 635612 cri.go:89] found id: "5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:45.887194 635612 cri.go:89] found id: ""
I0828 19:09:45.887210 635612 logs.go:276] 1 containers: [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319]
I0828 19:09:45.887275 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.890895 635612 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:09:45.890968 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:09:45.930751 635612 cri.go:89] found id: "e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:45.930776 635612 cri.go:89] found id: ""
I0828 19:09:45.930785 635612 logs.go:276] 1 containers: [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3]
I0828 19:09:45.930843 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.934450 635612 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:09:45.934576 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:09:45.975797 635612 cri.go:89] found id: "a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:45.975822 635612 cri.go:89] found id: ""
I0828 19:09:45.975830 635612 logs.go:276] 1 containers: [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16]
I0828 19:09:45.975903 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.979458 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:09:45.979543 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:09:46.018424 635612 cri.go:89] found id: "f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:46.018450 635612 cri.go:89] found id: ""
I0828 19:09:46.018458 635612 logs.go:276] 1 containers: [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0]
I0828 19:09:46.018551 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.023097 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:09:46.023171 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:09:46.076288 635612 cri.go:89] found id: "ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:46.076311 635612 cri.go:89] found id: ""
I0828 19:09:46.076319 635612 logs.go:276] 1 containers: [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623]
I0828 19:09:46.076409 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.080293 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:09:46.080400 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:09:46.136404 635612 cri.go:89] found id: "e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:46.136428 635612 cri.go:89] found id: ""
I0828 19:09:46.136436 635612 logs.go:276] 1 containers: [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2]
I0828 19:09:46.136526 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.140231 635612 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:09:46.140350 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:09:46.182052 635612 cri.go:89] found id: "917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:46.182121 635612 cri.go:89] found id: ""
I0828 19:09:46.182143 635612 logs.go:276] 1 containers: [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11]
I0828 19:09:46.182207 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.185610 635612 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:09:46.185677 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:09:46.230085 635612 cri.go:89] found id: "6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:46.230111 635612 cri.go:89] found id: ""
I0828 19:09:46.230118 635612 logs.go:276] 1 containers: [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21]
I0828 19:09:46.230179 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.233921 635612 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:09:46.234029 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:09:46.272391 635612 cri.go:89] found id: "77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:46.272415 635612 cri.go:89] found id: "00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:46.272419 635612 cri.go:89] found id: ""
I0828 19:09:46.272426 635612 logs.go:276] 2 containers: [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30]
I0828 19:09:46.272512 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.275880 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.279309 635612 logs.go:123] Gathering logs for etcd [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3] ...
I0828 19:09:46.279335 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:46.332432 635612 logs.go:123] Gathering logs for kube-controller-manager [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2] ...
I0828 19:09:46.332463 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:46.414187 635612 logs.go:123] Gathering logs for kubernetes-dashboard [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21] ...
I0828 19:09:46.414229 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:46.454309 635612 logs.go:123] Gathering logs for CRI-O ...
I0828 19:09:46.454338 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0828 19:09:46.537491 635612 logs.go:123] Gathering logs for storage-provisioner [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb] ...
I0828 19:09:46.537526 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:46.587143 635612 logs.go:123] Gathering logs for kubelet ...
I0828 19:09:46.587174 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0828 19:09:46.639142 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875224 732 reflector.go:138] object-"kube-system"/"metrics-server-token-glpj8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-glpj8" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639377 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875341 732 reflector.go:138] object-"kube-system"/"coredns-token-24mr5": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-24mr5" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639592 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875410 732 reflector.go:138] object-"kube-system"/"kindnet-token-9j8wt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-9j8wt" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639794 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875469 732 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640010 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875528 732 reflector.go:138] object-"kube-system"/"kube-proxy-token-h26lx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-h26lx" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640217 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875586 732 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640445 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875638 732 reflector.go:138] object-"kube-system"/"storage-provisioner-token-6t6fb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-6t6fb" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640680 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875728 732 reflector.go:138] object-"default"/"default-token-986j2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-986j2" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.651199 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:22 old-k8s-version-530457 kubelet[732]: E0828 19:04:22.754021 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.652234 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:23 old-k8s-version-530457 kubelet[732]: E0828 19:04:23.617200 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.654357 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:37 old-k8s-version-530457 kubelet[732]: E0828 19:04:37.592436 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.656104 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:52 old-k8s-version-530457 kubelet[732]: E0828 19:04:52.576595 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.656693 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:57 old-k8s-version-530457 kubelet[732]: E0828 19:04:57.827984 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.657152 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:58 old-k8s-version-530457 kubelet[732]: E0828 19:04:58.830137 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.659245 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:03 old-k8s-version-530457 kubelet[732]: E0828 19:05:03.597500 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.659602 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:04 old-k8s-version-530457 kubelet[732]: E0828 19:05:04.983929 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.659789 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:15 old-k8s-version-530457 kubelet[732]: E0828 19:05:15.576664 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.660382 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:18 old-k8s-version-530457 kubelet[732]: E0828 19:05:18.866240 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.660709 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:24 old-k8s-version-530457 kubelet[732]: E0828 19:05:24.985883 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.660917 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:26 old-k8s-version-530457 kubelet[732]: E0828 19:05:26.577717 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.661245 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:37 old-k8s-version-530457 kubelet[732]: E0828 19:05:37.575455 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.661449 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:40 old-k8s-version-530457 kubelet[732]: E0828 19:05:40.577507 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.662040 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:48 old-k8s-version-530457 kubelet[732]: E0828 19:05:48.915152 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.664090 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:51 old-k8s-version-530457 kubelet[732]: E0828 19:05:51.588013 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.664430 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:54 old-k8s-version-530457 kubelet[732]: E0828 19:05:54.984049 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.664614 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:06 old-k8s-version-530457 kubelet[732]: E0828 19:06:06.575922 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.664941 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:08 old-k8s-version-530457 kubelet[732]: E0828 19:06:08.575436 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.665124 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:20 old-k8s-version-530457 kubelet[732]: E0828 19:06:20.576354 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.665502 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:23 old-k8s-version-530457 kubelet[732]: E0828 19:06:23.575443 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.665689 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:32 old-k8s-version-530457 kubelet[732]: E0828 19:06:32.577946 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.666344 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:36 old-k8s-version-530457 kubelet[732]: E0828 19:06:36.981996 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.666539 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:43 old-k8s-version-530457 kubelet[732]: E0828 19:06:43.575953 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.666866 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:44 old-k8s-version-530457 kubelet[732]: E0828 19:06:44.984324 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.667051 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:56 old-k8s-version-530457 kubelet[732]: E0828 19:06:56.575945 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.667383 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:58 old-k8s-version-530457 kubelet[732]: E0828 19:06:58.575915 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.667696 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576302 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.667893 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576985 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.668223 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:24 old-k8s-version-530457 kubelet[732]: E0828 19:07:24.575460 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.670290 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:25 old-k8s-version-530457 kubelet[732]: E0828 19:07:25.586438 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.670477 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:36 old-k8s-version-530457 kubelet[732]: E0828 19:07:36.577014 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.670808 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:38 old-k8s-version-530457 kubelet[732]: E0828 19:07:38.575534 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.670991 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:48 old-k8s-version-530457 kubelet[732]: E0828 19:07:48.576137 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.672571 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:49 old-k8s-version-530457 kubelet[732]: E0828 19:07:49.575420 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.672780 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:00 old-k8s-version-530457 kubelet[732]: E0828 19:08:00.576139 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.673388 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:06 old-k8s-version-530457 kubelet[732]: E0828 19:08:06.130079 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.673573 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.576054 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.673902 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.983952 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674234 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:25 old-k8s-version-530457 kubelet[732]: E0828 19:08:25.575482 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674417 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:29 old-k8s-version-530457 kubelet[732]: E0828 19:08:29.575996 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.674752 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:37 old-k8s-version-530457 kubelet[732]: E0828 19:08:37.575411 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674937 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:43 old-k8s-version-530457 kubelet[732]: E0828 19:08:43.575929 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.675265 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: E0828 19:08:50.577123 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.675449 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:56 old-k8s-version-530457 kubelet[732]: E0828 19:08:56.576060 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.675780 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: E0828 19:09:04.575429 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.675975 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:07 old-k8s-version-530457 kubelet[732]: E0828 19:09:07.576124 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.676552 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: E0828 19:09:17.575422 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.676737 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.677066 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.677250 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.677586 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.677772 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0828 19:09:46.677782 635612 logs.go:123] Gathering logs for describe nodes ...
I0828 19:09:46.677797 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0828 19:09:46.858651 635612 logs.go:123] Gathering logs for kube-apiserver [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319] ...
I0828 19:09:46.858682 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:46.933892 635612 logs.go:123] Gathering logs for kube-scheduler [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0] ...
I0828 19:09:46.933928 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:46.980032 635612 logs.go:123] Gathering logs for kube-proxy [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623] ...
I0828 19:09:46.980061 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:47.023718 635612 logs.go:123] Gathering logs for kindnet [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11] ...
I0828 19:09:47.023747 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:47.073250 635612 logs.go:123] Gathering logs for container status ...
I0828 19:09:47.073287 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:09:47.137663 635612 logs.go:123] Gathering logs for dmesg ...
I0828 19:09:47.137699 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:09:47.158656 635612 logs.go:123] Gathering logs for coredns [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16] ...
I0828 19:09:47.158722 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:47.205508 635612 logs.go:123] Gathering logs for storage-provisioner [00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30] ...
I0828 19:09:47.205539 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:47.242809 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:47.242837 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0828 19:09:47.242887 635612 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0828 19:09:47.242904 635612 out.go:270] Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:47.242914 635612 out.go:270] Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:47.242936 635612 out.go:270] Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:47.242946 635612 out.go:270] Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:47.242980 635612 out.go:270] Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0828 19:09:47.242995 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:47.243001 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:09:57.245126 635612 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0828 19:09:57.258795 635612 api_server.go:72] duration metric: took 5m54.643340669s to wait for apiserver process to appear ...
I0828 19:09:57.258830 635612 api_server.go:88] waiting for apiserver healthz status ...
I0828 19:09:57.258871 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:09:57.258935 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:09:57.309643 635612 cri.go:89] found id: "5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:57.309671 635612 cri.go:89] found id: ""
I0828 19:09:57.309680 635612 logs.go:276] 1 containers: [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319]
I0828 19:09:57.309735 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.313665 635612 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:09:57.313748 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:09:57.357570 635612 cri.go:89] found id: "e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:57.357588 635612 cri.go:89] found id: ""
I0828 19:09:57.357596 635612 logs.go:276] 1 containers: [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3]
I0828 19:09:57.357650 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.363443 635612 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:09:57.363523 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:09:57.401091 635612 cri.go:89] found id: "a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:57.401111 635612 cri.go:89] found id: ""
I0828 19:09:57.401119 635612 logs.go:276] 1 containers: [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16]
I0828 19:09:57.401180 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.404820 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:09:57.404892 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:09:57.441795 635612 cri.go:89] found id: "f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:57.441812 635612 cri.go:89] found id: ""
I0828 19:09:57.441820 635612 logs.go:276] 1 containers: [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0]
I0828 19:09:57.441876 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.445389 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:09:57.445463 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:09:57.481786 635612 cri.go:89] found id: "ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:57.481807 635612 cri.go:89] found id: ""
I0828 19:09:57.481815 635612 logs.go:276] 1 containers: [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623]
I0828 19:09:57.481875 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.485535 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:09:57.485616 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:09:57.526371 635612 cri.go:89] found id: "e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:57.526396 635612 cri.go:89] found id: ""
I0828 19:09:57.526404 635612 logs.go:276] 1 containers: [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2]
I0828 19:09:57.526471 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.530191 635612 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:09:57.530266 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:09:57.574140 635612 cri.go:89] found id: "917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:57.574167 635612 cri.go:89] found id: ""
I0828 19:09:57.574175 635612 logs.go:276] 1 containers: [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11]
I0828 19:09:57.574232 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.577884 635612 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:09:57.577963 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:09:57.625232 635612 cri.go:89] found id: "6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:57.625299 635612 cri.go:89] found id: ""
I0828 19:09:57.625335 635612 logs.go:276] 1 containers: [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21]
I0828 19:09:57.625395 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.628836 635612 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:09:57.628951 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:09:57.675453 635612 cri.go:89] found id: "77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:57.675476 635612 cri.go:89] found id: "00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:57.675480 635612 cri.go:89] found id: ""
I0828 19:09:57.675487 635612 logs.go:276] 2 containers: [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30]
I0828 19:09:57.675544 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.679544 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.682894 635612 logs.go:123] Gathering logs for kube-proxy [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623] ...
I0828 19:09:57.682919 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:57.730503 635612 logs.go:123] Gathering logs for container status ...
I0828 19:09:57.730532 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:09:57.773015 635612 logs.go:123] Gathering logs for kube-apiserver [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319] ...
I0828 19:09:57.773043 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:57.846966 635612 logs.go:123] Gathering logs for coredns [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16] ...
I0828 19:09:57.847003 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:57.896268 635612 logs.go:123] Gathering logs for kindnet [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11] ...
I0828 19:09:57.896300 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:57.944005 635612 logs.go:123] Gathering logs for kubelet ...
I0828 19:09:57.944039 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0828 19:09:57.994339 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875224 732 reflector.go:138] object-"kube-system"/"metrics-server-token-glpj8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-glpj8" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.994628 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875341 732 reflector.go:138] object-"kube-system"/"coredns-token-24mr5": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-24mr5" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.994855 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875410 732 reflector.go:138] object-"kube-system"/"kindnet-token-9j8wt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-9j8wt" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995069 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875469 732 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995292 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875528 732 reflector.go:138] object-"kube-system"/"kube-proxy-token-h26lx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-h26lx" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995504 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875586 732 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995739 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875638 732 reflector.go:138] object-"kube-system"/"storage-provisioner-token-6t6fb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-6t6fb" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995954 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875728 732 reflector.go:138] object-"default"/"default-token-986j2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-986j2" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:58.007422 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:22 old-k8s-version-530457 kubelet[732]: E0828 19:04:22.754021 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.008467 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:23 old-k8s-version-530457 kubelet[732]: E0828 19:04:23.617200 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.010557 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:37 old-k8s-version-530457 kubelet[732]: E0828 19:04:37.592436 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.012403 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:52 old-k8s-version-530457 kubelet[732]: E0828 19:04:52.576595 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.013000 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:57 old-k8s-version-530457 kubelet[732]: E0828 19:04:57.827984 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.013543 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:58 old-k8s-version-530457 kubelet[732]: E0828 19:04:58.830137 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.015601 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:03 old-k8s-version-530457 kubelet[732]: E0828 19:05:03.597500 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.015933 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:04 old-k8s-version-530457 kubelet[732]: E0828 19:05:04.983929 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.016119 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:15 old-k8s-version-530457 kubelet[732]: E0828 19:05:15.576664 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.016708 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:18 old-k8s-version-530457 kubelet[732]: E0828 19:05:18.866240 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017093 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:24 old-k8s-version-530457 kubelet[732]: E0828 19:05:24.985883 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017281 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:26 old-k8s-version-530457 kubelet[732]: E0828 19:05:26.577717 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.017621 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:37 old-k8s-version-530457 kubelet[732]: E0828 19:05:37.575455 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017811 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:40 old-k8s-version-530457 kubelet[732]: E0828 19:05:40.577507 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.018412 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:48 old-k8s-version-530457 kubelet[732]: E0828 19:05:48.915152 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.020489 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:51 old-k8s-version-530457 kubelet[732]: E0828 19:05:51.588013 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.020823 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:54 old-k8s-version-530457 kubelet[732]: E0828 19:05:54.984049 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.021009 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:06 old-k8s-version-530457 kubelet[732]: E0828 19:06:06.575922 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.021343 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:08 old-k8s-version-530457 kubelet[732]: E0828 19:06:08.575436 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.021531 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:20 old-k8s-version-530457 kubelet[732]: E0828 19:06:20.576354 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.021861 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:23 old-k8s-version-530457 kubelet[732]: E0828 19:06:23.575443 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.022044 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:32 old-k8s-version-530457 kubelet[732]: E0828 19:06:32.577946 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.022634 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:36 old-k8s-version-530457 kubelet[732]: E0828 19:06:36.981996 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.022817 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:43 old-k8s-version-530457 kubelet[732]: E0828 19:06:43.575953 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.023144 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:44 old-k8s-version-530457 kubelet[732]: E0828 19:06:44.984324 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.023333 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:56 old-k8s-version-530457 kubelet[732]: E0828 19:06:56.575945 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.023661 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:58 old-k8s-version-530457 kubelet[732]: E0828 19:06:58.575915 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.023973 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576302 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.024171 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576985 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.024497 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:24 old-k8s-version-530457 kubelet[732]: E0828 19:07:24.575460 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.026567 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:25 old-k8s-version-530457 kubelet[732]: E0828 19:07:25.586438 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.026756 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:36 old-k8s-version-530457 kubelet[732]: E0828 19:07:36.577014 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.027094 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:38 old-k8s-version-530457 kubelet[732]: E0828 19:07:38.575534 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.027280 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:48 old-k8s-version-530457 kubelet[732]: E0828 19:07:48.576137 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.027608 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:49 old-k8s-version-530457 kubelet[732]: E0828 19:07:49.575420 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.027792 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:00 old-k8s-version-530457 kubelet[732]: E0828 19:08:00.576139 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.028380 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:06 old-k8s-version-530457 kubelet[732]: E0828 19:08:06.130079 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.028563 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.576054 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.028891 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.983952 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029219 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:25 old-k8s-version-530457 kubelet[732]: E0828 19:08:25.575482 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029448 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:29 old-k8s-version-530457 kubelet[732]: E0828 19:08:29.575996 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.029778 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:37 old-k8s-version-530457 kubelet[732]: E0828 19:08:37.575411 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029963 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:43 old-k8s-version-530457 kubelet[732]: E0828 19:08:43.575929 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.030289 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: E0828 19:08:50.577123 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.030479 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:56 old-k8s-version-530457 kubelet[732]: E0828 19:08:56.576060 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.030807 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: E0828 19:09:04.575429 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.031061 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:07 old-k8s-version-530457 kubelet[732]: E0828 19:09:07.576124 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.031639 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: E0828 19:09:17.575422 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.031826 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.032153 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.032340 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.032672 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.032858 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.033172 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.033376 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
I0828 19:09:58.033389 635612 logs.go:123] Gathering logs for kubernetes-dashboard [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21] ...
I0828 19:09:58.033406 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:58.089976 635612 logs.go:123] Gathering logs for storage-provisioner [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb] ...
I0828 19:09:58.090005 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:58.135562 635612 logs.go:123] Gathering logs for storage-provisioner [00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30] ...
I0828 19:09:58.135591 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:58.173412 635612 logs.go:123] Gathering logs for kube-scheduler [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0] ...
I0828 19:09:58.173445 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:58.237093 635612 logs.go:123] Gathering logs for describe nodes ...
I0828 19:09:58.237121 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0828 19:09:58.383455 635612 logs.go:123] Gathering logs for etcd [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3] ...
I0828 19:09:58.383491 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:58.434930 635612 logs.go:123] Gathering logs for kube-controller-manager [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2] ...
I0828 19:09:58.434968 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:58.517578 635612 logs.go:123] Gathering logs for CRI-O ...
I0828 19:09:58.517612 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0828 19:09:58.606571 635612 logs.go:123] Gathering logs for dmesg ...
I0828 19:09:58.606606 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:09:58.626574 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:58.626597 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0828 19:09:58.626648 635612 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0828 19:09:58.626663 635612 out.go:270] Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626669 635612 out.go:270] Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.626676 635612 out.go:270] Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626686 635612 out.go:270] Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626695 635612 out.go:270] Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
I0828 19:09:58.626705 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:58.626710 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:10:08.628180 635612 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0828 19:10:08.641074 635612 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0828 19:10:08.644092 635612 out.go:201]
W0828 19:10:08.646685 635612 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0828 19:10:08.646728 635612 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0828 19:10:08.646753 635612 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0828 19:10:08.646763 635612 out.go:270] *
*
W0828 19:10:08.647693 635612 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0828 19:10:08.649411 635612 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-530457 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-530457
helpers_test.go:235: (dbg) docker inspect old-k8s-version-530457:
-- stdout --
[
{
"Id": "4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e",
"Created": "2024-08-28T19:01:02.109715325Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 636045,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-08-28T19:03:53.644399874Z",
"FinishedAt": "2024-08-28T19:03:51.974971743Z"
},
"Image": "sha256:2cc8dc59c2b679153d99f84cc70dab3e87225f8a0d04f61969b54714a9c4cd4d",
"ResolvConfPath": "/var/lib/docker/containers/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e/hostname",
"HostsPath": "/var/lib/docker/containers/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e/hosts",
"LogPath": "/var/lib/docker/containers/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e-json.log",
"Name": "/old-k8s-version-530457",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-530457:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-530457",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/26399591475f32175ab80eaca3c8d9e7df7de7d8b0fd3454a294bc0222dad1bb-init/diff:/var/lib/docker/overlay2/6bfa77c07dd97043d71deb05d517c77255c781df0cdd58b7d0f863c8462d9eed/diff",
"MergedDir": "/var/lib/docker/overlay2/26399591475f32175ab80eaca3c8d9e7df7de7d8b0fd3454a294bc0222dad1bb/merged",
"UpperDir": "/var/lib/docker/overlay2/26399591475f32175ab80eaca3c8d9e7df7de7d8b0fd3454a294bc0222dad1bb/diff",
"WorkDir": "/var/lib/docker/overlay2/26399591475f32175ab80eaca3c8d9e7df7de7d8b0fd3454a294bc0222dad1bb/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-530457",
"Source": "/var/lib/docker/volumes/old-k8s-version-530457/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-530457",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-530457",
"name.minikube.sigs.k8s.io": "old-k8s-version-530457",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "ecc6803fe53c2ef83a1e17b354ff114091306af98c3a15b442a5766e2fc3a92a",
"SandboxKey": "/var/run/docker/netns/ecc6803fe53c",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33450"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33451"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33454"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33452"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33453"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-530457": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"DriverOpts": null,
"NetworkID": "ebeebcaf3573cf18e25eadcab770164d6fd0e41e322fc450a5936aaa10246117",
"EndpointID": "6b541ad8c206fce82e15f44f9c31154f2bbf41eb9d9c54b53c96abe7edf45af3",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-530457",
"4dbb4904f00c"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-530457 -n old-k8s-version-530457
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-530457 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-530457 logs -n 25: (1.885872835s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| ssh | -p cilium-058163 sudo cat | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | /etc/containerd/config.toml | | | | | |
| ssh | -p cilium-058163 sudo | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | containerd config dump | | | | | |
| ssh | -p cilium-058163 sudo | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | systemctl status crio --all | | | | | |
| | --full --no-pager | | | | | |
| ssh | -p cilium-058163 sudo | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | systemctl cat crio --no-pager | | | | | |
| ssh | -p cilium-058163 sudo find | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p cilium-058163 sudo crio | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | |
| | config | | | | | |
| delete | -p cilium-058163 | cilium-058163 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | 28 Aug 24 18:59 UTC |
| start | -p cert-expiration-756409 | cert-expiration-756409 | jenkins | v1.33.1 | 28 Aug 24 18:59 UTC | 28 Aug 24 19:00 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| delete | -p force-systemd-env-275491 | force-systemd-env-275491 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:00 UTC |
| start | -p cert-options-990758 | cert-options-990758 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:00 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| ssh | cert-options-990758 ssh | cert-options-990758 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:00 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-990758 -- sudo | cert-options-990758 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:00 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-990758 | cert-options-990758 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:00 UTC |
| start | -p old-k8s-version-530457 | old-k8s-version-530457 | jenkins | v1.33.1 | 28 Aug 24 19:00 UTC | 28 Aug 24 19:03 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-756409 | cert-expiration-756409 | jenkins | v1.33.1 | 28 Aug 24 19:03 UTC | 28 Aug 24 19:04 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| addons | enable metrics-server -p old-k8s-version-530457 | old-k8s-version-530457 | jenkins | v1.33.1 | 28 Aug 24 19:03 UTC | 28 Aug 24 19:03 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-530457 | old-k8s-version-530457 | jenkins | v1.33.1 | 28 Aug 24 19:03 UTC | 28 Aug 24 19:03 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-530457 | old-k8s-version-530457 | jenkins | v1.33.1 | 28 Aug 24 19:03 UTC | 28 Aug 24 19:03 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-530457 | old-k8s-version-530457 | jenkins | v1.33.1 | 28 Aug 24 19:03 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| delete | -p cert-expiration-756409 | cert-expiration-756409 | jenkins | v1.33.1 | 28 Aug 24 19:04 UTC | 28 Aug 24 19:04 UTC |
| start | -p no-preload-811745 | no-preload-811745 | jenkins | v1.33.1 | 28 Aug 24 19:04 UTC | 28 Aug 24 19:05 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
| addons | enable metrics-server -p no-preload-811745 | no-preload-811745 | jenkins | v1.33.1 | 28 Aug 24 19:05 UTC | 28 Aug 24 19:05 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-811745 | no-preload-811745 | jenkins | v1.33.1 | 28 Aug 24 19:05 UTC | 28 Aug 24 19:05 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-811745 | no-preload-811745 | jenkins | v1.33.1 | 28 Aug 24 19:05 UTC | 28 Aug 24 19:05 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-811745 | no-preload-811745 | jenkins | v1.33.1 | 28 Aug 24 19:05 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/08/28 19:05:45
Running on machine: ip-172-31-29-130
Binary: Built with gc go1.22.5 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0828 19:05:45.392832 643069 out.go:345] Setting OutFile to fd 1 ...
I0828 19:05:45.392953 643069 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:05:45.392964 643069 out.go:358] Setting ErrFile to fd 2...
I0828 19:05:45.392969 643069 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:05:45.393201 643069 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19529-440552/.minikube/bin
I0828 19:05:45.393638 643069 out.go:352] Setting JSON to false
I0828 19:05:45.394780 643069 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":10094,"bootTime":1724861852,"procs":221,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1068-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0828 19:05:45.394857 643069 start.go:139] virtualization:
I0828 19:05:45.396604 643069 out.go:177] * [no-preload-811745] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0828 19:05:45.398320 643069 out.go:177] - MINIKUBE_LOCATION=19529
I0828 19:05:45.398462 643069 notify.go:220] Checking for updates...
I0828 19:05:45.400574 643069 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0828 19:05:45.402700 643069 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:05:45.404261 643069 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19529-440552/.minikube
I0828 19:05:45.405379 643069 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0828 19:05:45.406956 643069 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0828 19:05:45.408420 643069 config.go:182] Loaded profile config "no-preload-811745": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.31.0
I0828 19:05:45.408945 643069 driver.go:392] Setting default libvirt URI to qemu:///system
I0828 19:05:45.432561 643069 docker.go:123] docker version: linux-27.2.0:Docker Engine - Community
I0828 19:05:45.432693 643069 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0828 19:05:45.513479 643069 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-28 19:05:45.502416684 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.2.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:472731909fa34bd7bc9c087e4c27943f9835f111 Expected:472731909fa34bd7bc9c087e4c27943f9835f111} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0828 19:05:45.513595 643069 docker.go:307] overlay module found
I0828 19:05:45.515355 643069 out.go:177] * Using the docker driver based on existing profile
I0828 19:05:45.516497 643069 start.go:297] selected driver: docker
I0828 19:05:45.516518 643069 start.go:901] validating driver "docker" against &{Name:no-preload-811745 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-811745 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home
/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:05:45.516649 643069 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0828 19:05:45.517370 643069 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0828 19:05:45.602184 643069 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:39 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-28 19:05:45.571384018 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1068-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214831104 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.2.0 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:472731909fa34bd7bc9c087e4c27943f9835f111 Expected:472731909fa34bd7bc9c087e4c27943f9835f111} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.2]] Warnings:<nil>}}
I0828 19:05:45.602651 643069 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0828 19:05:45.602692 643069 cni.go:84] Creating CNI manager for ""
I0828 19:05:45.602702 643069 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0828 19:05:45.602743 643069 start.go:340] cluster config:
{Name:no-preload-811745 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-811745 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMS
ize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:05:45.605257 643069 out.go:177] * Starting "no-preload-811745" primary control-plane node in "no-preload-811745" cluster
I0828 19:05:45.606505 643069 cache.go:121] Beginning downloading kic base image for docker with crio
I0828 19:05:45.607831 643069 out.go:177] * Pulling base image v0.0.44-1724775115-19521 ...
I0828 19:05:45.609455 643069 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime crio
I0828 19:05:45.609540 643069 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local docker daemon
I0828 19:05:45.609617 643069 profile.go:143] Saving config to /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/config.json ...
I0828 19:05:45.609977 643069 cache.go:107] acquiring lock: {Name:mk4e48afaaa101f08f5a2de26320181b369d06eb Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610075 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0828 19:05:45.610092 643069 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 120.793µs
I0828 19:05:45.610108 643069 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0828 19:05:45.610121 643069 cache.go:107] acquiring lock: {Name:mk87ce53d187e1245ffcaa634d307ce6842a1894 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610168 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0 exists
I0828 19:05:45.610178 643069 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.31.0" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0" took 57.952µs
I0828 19:05:45.610185 643069 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.31.0 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0 succeeded
I0828 19:05:45.610207 643069 cache.go:107] acquiring lock: {Name:mke114dcb765ae8c8c2dc83b7edf9669c55fceaf Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610242 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0 exists
I0828 19:05:45.610252 643069 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.31.0" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0" took 47.056µs
I0828 19:05:45.610259 643069 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.31.0 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0 succeeded
I0828 19:05:45.610271 643069 cache.go:107] acquiring lock: {Name:mk73791ffbcc7fb6b3b51944e7b4e67d008c88eb Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610304 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0 exists
I0828 19:05:45.610310 643069 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.31.0" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0" took 41.246µs
I0828 19:05:45.610322 643069 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.31.0 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0 succeeded
I0828 19:05:45.610338 643069 cache.go:107] acquiring lock: {Name:mk2b6922607f35668ba84f006bf34c5f38011aca Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610389 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0 exists
I0828 19:05:45.610395 643069 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.31.0" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0" took 64.614µs
I0828 19:05:45.610410 643069 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.31.0 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0 succeeded
I0828 19:05:45.610429 643069 cache.go:107] acquiring lock: {Name:mkd042459c48f85f27638ab891d72e1a61d59194 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610468 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0828 19:05:45.610478 643069 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 49.698µs
I0828 19:05:45.610484 643069 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0828 19:05:45.610493 643069 cache.go:107] acquiring lock: {Name:mkc00ac7fab3b09aa577b451a9e8000b0d88591d Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610523 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 exists
I0828 19:05:45.610533 643069 cache.go:96] cache image "registry.k8s.io/etcd:3.5.15-0" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0" took 41.271µs
I0828 19:05:45.610538 643069 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.15-0 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 succeeded
I0828 19:05:45.610550 643069 cache.go:107] acquiring lock: {Name:mk2778d77f9ab0d73f06edeaef987b4123d449c1 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.610582 643069 cache.go:115] /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 exists
I0828 19:05:45.610590 643069 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.1" -> "/home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1" took 43.494µs
I0828 19:05:45.610596 643069 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.1 -> /home/jenkins/minikube-integration/19529-440552/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 succeeded
I0828 19:05:45.610602 643069 cache.go:87] Successfully saved all images to host disk.
W0828 19:05:45.630918 643069 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce is of wrong architecture
I0828 19:05:45.630961 643069 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce to local cache
I0828 19:05:45.631046 643069 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local cache directory
I0828 19:05:45.631070 643069 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce in local cache directory, skipping pull
I0828 19:05:45.631078 643069 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce exists in cache, skipping pull
I0828 19:05:45.631089 643069 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce as a tarball
I0828 19:05:45.631095 643069 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce from local cache
I0828 19:05:45.762580 643069 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce from cached tarball
I0828 19:05:45.762627 643069 cache.go:194] Successfully downloaded all kic artifacts
I0828 19:05:45.762667 643069 start.go:360] acquireMachinesLock for no-preload-811745: {Name:mk2bab32d99ea0630b31dc0b7f6bccdfce646c96 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0828 19:05:45.762740 643069 start.go:364] duration metric: took 51.149µs to acquireMachinesLock for "no-preload-811745"
I0828 19:05:45.762771 643069 start.go:96] Skipping create...Using existing machine configuration
I0828 19:05:45.762780 643069 fix.go:54] fixHost starting:
I0828 19:05:45.763071 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:45.779092 643069 fix.go:112] recreateIfNeeded on no-preload-811745: state=Stopped err=<nil>
W0828 19:05:45.779125 643069 fix.go:138] unexpected machine state, will restart: <nil>
I0828 19:05:45.780534 643069 out.go:177] * Restarting existing docker container for "no-preload-811745" ...
I0828 19:05:44.313663 635612 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:45.814931 635612 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.814957 635612 pod_ready.go:82] duration metric: took 1m12.006976966s for pod "kube-controller-manager-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.814970 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-mzrqc" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.820896 635612 pod_ready.go:93] pod "kube-proxy-mzrqc" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.820917 635612 pod_ready.go:82] duration metric: took 5.939148ms for pod "kube-proxy-mzrqc" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.820927 635612 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.828415 635612 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:45.828506 635612 pod_ready.go:82] duration metric: took 7.56952ms for pod "kube-scheduler-old-k8s-version-530457" in "kube-system" namespace to be "Ready" ...
I0828 19:05:45.828533 635612 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace to be "Ready" ...
I0828 19:05:47.835145 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:45.781633 643069 cli_runner.go:164] Run: docker start no-preload-811745
I0828 19:05:46.107533 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:46.131485 643069 kic.go:430] container "no-preload-811745" state is running.
I0828 19:05:46.131950 643069 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-811745
I0828 19:05:46.154877 643069 profile.go:143] Saving config to /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/config.json ...
I0828 19:05:46.155120 643069 machine.go:93] provisionDockerMachine start ...
I0828 19:05:46.155186 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:46.177468 643069 main.go:141] libmachine: Using SSH client type: native
I0828 19:05:46.177728 643069 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33460 <nil> <nil>}
I0828 19:05:46.177752 643069 main.go:141] libmachine: About to run SSH command:
hostname
I0828 19:05:46.178454 643069 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0828 19:05:49.316593 643069 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-811745
I0828 19:05:49.316616 643069 ubuntu.go:169] provisioning hostname "no-preload-811745"
I0828 19:05:49.316680 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:49.341973 643069 main.go:141] libmachine: Using SSH client type: native
I0828 19:05:49.342221 643069 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33460 <nil> <nil>}
I0828 19:05:49.342236 643069 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-811745 && echo "no-preload-811745" | sudo tee /etc/hostname
I0828 19:05:49.490446 643069 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-811745
I0828 19:05:49.490531 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:49.513299 643069 main.go:141] libmachine: Using SSH client type: native
I0828 19:05:49.513596 643069 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33460 <nil> <nil>}
I0828 19:05:49.513620 643069 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-811745' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-811745/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-811745' | sudo tee -a /etc/hosts;
fi
fi
I0828 19:05:49.645215 643069 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0828 19:05:49.645239 643069 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19529-440552/.minikube CaCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19529-440552/.minikube}
I0828 19:05:49.645291 643069 ubuntu.go:177] setting up certificates
I0828 19:05:49.645302 643069 provision.go:84] configureAuth start
I0828 19:05:49.645424 643069 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-811745
I0828 19:05:49.663330 643069 provision.go:143] copyHostCerts
I0828 19:05:49.663399 643069 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem, removing ...
I0828 19:05:49.663407 643069 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem
I0828 19:05:49.663482 643069 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/ca.pem (1082 bytes)
I0828 19:05:49.663604 643069 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem, removing ...
I0828 19:05:49.663610 643069 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem
I0828 19:05:49.663635 643069 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/cert.pem (1123 bytes)
I0828 19:05:49.663696 643069 exec_runner.go:144] found /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem, removing ...
I0828 19:05:49.663701 643069 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem
I0828 19:05:49.663723 643069 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19529-440552/.minikube/key.pem (1679 bytes)
I0828 19:05:49.663818 643069 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem org=jenkins.no-preload-811745 san=[127.0.0.1 192.168.85.2 localhost minikube no-preload-811745]
I0828 19:05:49.981329 643069 provision.go:177] copyRemoteCerts
I0828 19:05:49.981396 643069 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0828 19:05:49.981455 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:49.997541 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:50.111100 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0828 19:05:50.138486 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0828 19:05:50.164055 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0828 19:05:50.190612 643069 provision.go:87] duration metric: took 545.262199ms to configureAuth
I0828 19:05:50.190639 643069 ubuntu.go:193] setting minikube options for container-runtime
I0828 19:05:50.190849 643069 config.go:182] Loaded profile config "no-preload-811745": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.31.0
I0828 19:05:50.190966 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:50.207761 643069 main.go:141] libmachine: Using SSH client type: native
I0828 19:05:50.208026 643069 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33460 <nil> <nil>}
I0828 19:05:50.208046 643069 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0828 19:05:50.580876 643069 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0828 19:05:50.580900 643069 machine.go:96] duration metric: took 4.42576207s to provisionDockerMachine
I0828 19:05:50.580913 643069 start.go:293] postStartSetup for "no-preload-811745" (driver="docker")
I0828 19:05:50.580924 643069 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0828 19:05:50.580987 643069 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0828 19:05:50.581039 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:50.600390 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:50.705904 643069 ssh_runner.go:195] Run: cat /etc/os-release
I0828 19:05:50.709272 643069 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0828 19:05:50.709365 643069 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0828 19:05:50.709382 643069 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0828 19:05:50.709390 643069 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0828 19:05:50.709404 643069 filesync.go:126] Scanning /home/jenkins/minikube-integration/19529-440552/.minikube/addons for local assets ...
I0828 19:05:50.709455 643069 filesync.go:126] Scanning /home/jenkins/minikube-integration/19529-440552/.minikube/files for local assets ...
I0828 19:05:50.709563 643069 filesync.go:149] local asset: /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem -> 4459402.pem in /etc/ssl/certs
I0828 19:05:50.709672 643069 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0828 19:05:50.719395 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem --> /etc/ssl/certs/4459402.pem (1708 bytes)
I0828 19:05:50.743240 643069 start.go:296] duration metric: took 162.312261ms for postStartSetup
I0828 19:05:50.743341 643069 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0828 19:05:50.743419 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:50.759944 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:50.850553 643069 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0828 19:05:50.855670 643069 fix.go:56] duration metric: took 5.092883137s for fixHost
I0828 19:05:50.855696 643069 start.go:83] releasing machines lock for "no-preload-811745", held for 5.092944322s
I0828 19:05:50.855768 643069 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-811745
I0828 19:05:50.872612 643069 ssh_runner.go:195] Run: cat /version.json
I0828 19:05:50.872669 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:50.872946 643069 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0828 19:05:50.873018 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:50.892473 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:50.904464 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:50.988807 643069 ssh_runner.go:195] Run: systemctl --version
I0828 19:05:51.141386 643069 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0828 19:05:51.288620 643069 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0828 19:05:51.292978 643069 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0828 19:05:51.301985 643069 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0828 19:05:51.302064 643069 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0828 19:05:51.311245 643069 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0828 19:05:51.311269 643069 start.go:495] detecting cgroup driver to use...
I0828 19:05:51.311303 643069 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0828 19:05:51.311360 643069 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0828 19:05:51.323808 643069 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0828 19:05:51.341495 643069 docker.go:217] disabling cri-docker service (if available) ...
I0828 19:05:51.341560 643069 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0828 19:05:51.356690 643069 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0828 19:05:51.368586 643069 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0828 19:05:51.463735 643069 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0828 19:05:51.553872 643069 docker.go:233] disabling docker service ...
I0828 19:05:51.554009 643069 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0828 19:05:51.567594 643069 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0828 19:05:51.586135 643069 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0828 19:05:51.683839 643069 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0828 19:05:51.773986 643069 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0828 19:05:51.786651 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0828 19:05:51.811804 643069 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.10" pause image...
I0828 19:05:51.811874 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.10"|' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.823129 643069 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0828 19:05:51.823211 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.839524 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.850511 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.861478 643069 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0828 19:05:51.873059 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *"net.ipv4.ip_unprivileged_port_start=.*"/d' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.883765 643069 ssh_runner.go:195] Run: sh -c "sudo grep -q "^ *default_sysctls" /etc/crio/crio.conf.d/02-crio.conf || sudo sed -i '/conmon_cgroup = .*/a default_sysctls = \[\n\]' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.893536 643069 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^default_sysctls *= *\[|&\n "net.ipv4.ip_unprivileged_port_start=0",|' /etc/crio/crio.conf.d/02-crio.conf"
I0828 19:05:51.903817 643069 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0828 19:05:51.914091 643069 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0828 19:05:51.923831 643069 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:05:52.022990 643069 ssh_runner.go:195] Run: sudo systemctl restart crio
I0828 19:05:52.138341 643069 start.go:542] Will wait 60s for socket path /var/run/crio/crio.sock
I0828 19:05:52.138416 643069 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0828 19:05:52.142076 643069 start.go:563] Will wait 60s for crictl version
I0828 19:05:52.142142 643069 ssh_runner.go:195] Run: which crictl
I0828 19:05:52.146097 643069 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0828 19:05:52.183904 643069 start.go:579] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0828 19:05:52.183989 643069 ssh_runner.go:195] Run: crio --version
I0828 19:05:52.226650 643069 ssh_runner.go:195] Run: crio --version
I0828 19:05:52.273912 643069 out.go:177] * Preparing Kubernetes v1.31.0 on CRI-O 1.24.6 ...
I0828 19:05:49.836051 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:51.836953 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:52.275112 643069 cli_runner.go:164] Run: docker network inspect no-preload-811745 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0828 19:05:52.291485 643069 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0828 19:05:52.295397 643069 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0828 19:05:52.307239 643069 kubeadm.go:883] updating cluster {Name:no-preload-811745 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-811745 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikub
e-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0828 19:05:52.307370 643069 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime crio
I0828 19:05:52.307424 643069 ssh_runner.go:195] Run: sudo crictl images --output json
I0828 19:05:52.367898 643069 crio.go:514] all images are preloaded for cri-o runtime.
I0828 19:05:52.367924 643069 cache_images.go:84] Images are preloaded, skipping loading
I0828 19:05:52.367932 643069 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.31.0 crio true true} ...
I0828 19:05:52.368045 643069 kubeadm.go:946] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=no-preload-811745 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.31.0 ClusterName:no-preload-811745 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0828 19:05:52.368140 643069 ssh_runner.go:195] Run: crio config
I0828 19:05:52.429777 643069 cni.go:84] Creating CNI manager for ""
I0828 19:05:52.429807 643069 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0828 19:05:52.429818 643069 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0828 19:05:52.429841 643069 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.31.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-811745 NodeName:no-preload-811745 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc
/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0828 19:05:52.429999 643069 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/crio/crio.sock
name: "no-preload-811745"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/crio/crio.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0828 19:05:52.430073 643069 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.0
I0828 19:05:52.439682 643069 binaries.go:44] Found k8s binaries, skipping transfer
I0828 19:05:52.439766 643069 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0828 19:05:52.448455 643069 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (367 bytes)
I0828 19:05:52.466514 643069 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0828 19:05:52.484274 643069 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2155 bytes)
I0828 19:05:52.502463 643069 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0828 19:05:52.506009 643069 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0828 19:05:52.517696 643069 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:05:52.625154 643069 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0828 19:05:52.640820 643069 certs.go:68] Setting up /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745 for IP: 192.168.85.2
I0828 19:05:52.640846 643069 certs.go:194] generating shared ca certs ...
I0828 19:05:52.640866 643069 certs.go:226] acquiring lock for ca certs: {Name:mk3dbd0c4288cf1e03eeb5b77f9120790fc1f5e6 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:05:52.641002 643069 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19529-440552/.minikube/ca.key
I0828 19:05:52.641057 643069 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.key
I0828 19:05:52.641070 643069 certs.go:256] generating profile certs ...
I0828 19:05:52.641152 643069 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/client.key
I0828 19:05:52.641262 643069 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/apiserver.key.0a8b1dbe
I0828 19:05:52.641320 643069 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/proxy-client.key
I0828 19:05:52.641456 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940.pem (1338 bytes)
W0828 19:05:52.641495 643069 certs.go:480] ignoring /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940_empty.pem, impossibly tiny 0 bytes
I0828 19:05:52.641508 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca-key.pem (1675 bytes)
I0828 19:05:52.641535 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/ca.pem (1082 bytes)
I0828 19:05:52.641563 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/cert.pem (1123 bytes)
I0828 19:05:52.641586 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/certs/key.pem (1679 bytes)
I0828 19:05:52.641632 643069 certs.go:484] found cert: /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem (1708 bytes)
I0828 19:05:52.642314 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0828 19:05:52.673270 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0828 19:05:52.718761 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0828 19:05:52.773358 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0828 19:05:52.833315 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0828 19:05:52.860736 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0828 19:05:52.888488 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0828 19:05:52.913606 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/profiles/no-preload-811745/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0828 19:05:52.940787 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0828 19:05:52.965480 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/certs/445940.pem --> /usr/share/ca-certificates/445940.pem (1338 bytes)
I0828 19:05:52.990974 643069 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19529-440552/.minikube/files/etc/ssl/certs/4459402.pem --> /usr/share/ca-certificates/4459402.pem (1708 bytes)
I0828 19:05:53.023816 643069 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0828 19:05:53.042959 643069 ssh_runner.go:195] Run: openssl version
I0828 19:05:53.050971 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0828 19:05:53.061480 643069 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0828 19:05:53.065103 643069 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 28 18:03 /usr/share/ca-certificates/minikubeCA.pem
I0828 19:05:53.065185 643069 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0828 19:05:53.072183 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0828 19:05:53.081647 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/445940.pem && ln -fs /usr/share/ca-certificates/445940.pem /etc/ssl/certs/445940.pem"
I0828 19:05:53.091717 643069 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/445940.pem
I0828 19:05:53.095452 643069 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 28 18:21 /usr/share/ca-certificates/445940.pem
I0828 19:05:53.095521 643069 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/445940.pem
I0828 19:05:53.102831 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/445940.pem /etc/ssl/certs/51391683.0"
I0828 19:05:53.112015 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/4459402.pem && ln -fs /usr/share/ca-certificates/4459402.pem /etc/ssl/certs/4459402.pem"
I0828 19:05:53.121673 643069 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/4459402.pem
I0828 19:05:53.125134 643069 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 28 18:21 /usr/share/ca-certificates/4459402.pem
I0828 19:05:53.125240 643069 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/4459402.pem
I0828 19:05:53.132798 643069 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/4459402.pem /etc/ssl/certs/3ec20f2e.0"
I0828 19:05:53.142398 643069 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0828 19:05:53.146230 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0828 19:05:53.153483 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0828 19:05:53.160888 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0828 19:05:53.169169 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0828 19:05:53.177354 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0828 19:05:53.185508 643069 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0828 19:05:53.194887 643069 kubeadm.go:392] StartCluster: {Name:no-preload-811745 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1724775115-19521@sha256:5e61ebc6e68d69e31cadead040aa9b41aa36d281b29a7d562fa41077c99ed3ce Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-811745 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-h
ost Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0828 19:05:53.194991 643069 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0828 19:05:53.195080 643069 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0828 19:05:53.258306 643069 cri.go:89] found id: ""
I0828 19:05:53.258409 643069 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0828 19:05:53.270192 643069 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0828 19:05:53.270214 643069 kubeadm.go:593] restartPrimaryControlPlane start ...
I0828 19:05:53.270294 643069 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0828 19:05:53.291836 643069 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0828 19:05:53.292458 643069 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-811745" does not appear in /home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:05:53.292777 643069 kubeconfig.go:62] /home/jenkins/minikube-integration/19529-440552/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-811745" cluster setting kubeconfig missing "no-preload-811745" context setting]
I0828 19:05:53.293413 643069 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19529-440552/kubeconfig: {Name:mk2bbb990ac0e5b155d3e1ee34adea25cdbf73a4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:05:53.294859 643069 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0828 19:05:53.324800 643069 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0828 19:05:53.324835 643069 kubeadm.go:597] duration metric: took 54.613712ms to restartPrimaryControlPlane
I0828 19:05:53.324846 643069 kubeadm.go:394] duration metric: took 129.970687ms to StartCluster
I0828 19:05:53.324868 643069 settings.go:142] acquiring lock: {Name:mk3578bce439064a3b256aa0408671abd470922d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:05:53.324933 643069 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19529-440552/kubeconfig
I0828 19:05:53.325891 643069 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19529-440552/kubeconfig: {Name:mk2bbb990ac0e5b155d3e1ee34adea25cdbf73a4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0828 19:05:53.326101 643069 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0828 19:05:53.326478 643069 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0828 19:05:53.326558 643069 addons.go:69] Setting storage-provisioner=true in profile "no-preload-811745"
I0828 19:05:53.326581 643069 addons.go:234] Setting addon storage-provisioner=true in "no-preload-811745"
W0828 19:05:53.326597 643069 addons.go:243] addon storage-provisioner should already be in state true
I0828 19:05:53.326622 643069 host.go:66] Checking if "no-preload-811745" exists ...
I0828 19:05:53.327087 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:53.327393 643069 config.go:182] Loaded profile config "no-preload-811745": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.31.0
I0828 19:05:53.327467 643069 addons.go:69] Setting default-storageclass=true in profile "no-preload-811745"
I0828 19:05:53.327509 643069 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-811745"
I0828 19:05:53.327752 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:53.328232 643069 addons.go:69] Setting dashboard=true in profile "no-preload-811745"
I0828 19:05:53.328260 643069 addons.go:234] Setting addon dashboard=true in "no-preload-811745"
W0828 19:05:53.328267 643069 addons.go:243] addon dashboard should already be in state true
I0828 19:05:53.328293 643069 host.go:66] Checking if "no-preload-811745" exists ...
I0828 19:05:53.328680 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:53.330787 643069 out.go:177] * Verifying Kubernetes components...
I0828 19:05:53.330832 643069 addons.go:69] Setting metrics-server=true in profile "no-preload-811745"
I0828 19:05:53.330860 643069 addons.go:234] Setting addon metrics-server=true in "no-preload-811745"
W0828 19:05:53.330867 643069 addons.go:243] addon metrics-server should already be in state true
I0828 19:05:53.331121 643069 host.go:66] Checking if "no-preload-811745" exists ...
I0828 19:05:53.335343 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:53.336659 643069 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0828 19:05:53.397715 643069 addons.go:234] Setting addon default-storageclass=true in "no-preload-811745"
W0828 19:05:53.397736 643069 addons.go:243] addon default-storageclass should already be in state true
I0828 19:05:53.397759 643069 host.go:66] Checking if "no-preload-811745" exists ...
I0828 19:05:53.398166 643069 cli_runner.go:164] Run: docker container inspect no-preload-811745 --format={{.State.Status}}
I0828 19:05:53.401340 643069 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0828 19:05:53.402717 643069 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:05:53.402735 643069 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0828 19:05:53.402808 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:53.415757 643069 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0828 19:05:53.416845 643069 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0828 19:05:53.418566 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0828 19:05:53.418595 643069 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0828 19:05:53.418682 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:53.433731 643069 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0828 19:05:53.434851 643069 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0828 19:05:53.434872 643069 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0828 19:05:53.434951 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:53.479923 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:53.490782 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:53.495213 643069 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0828 19:05:53.495236 643069 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0828 19:05:53.498903 643069 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-811745
I0828 19:05:53.512741 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:53.535632 643069 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33460 SSHKeyPath:/home/jenkins/minikube-integration/19529-440552/.minikube/machines/no-preload-811745/id_rsa Username:docker}
I0828 19:05:53.732414 643069 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0828 19:05:53.778345 643069 node_ready.go:35] waiting up to 6m0s for node "no-preload-811745" to be "Ready" ...
I0828 19:05:53.795213 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:05:53.795745 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0828 19:05:53.795764 643069 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0828 19:05:53.813451 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:05:53.817042 643069 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0828 19:05:53.817110 643069 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0828 19:05:53.884420 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0828 19:05:53.884493 643069 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0828 19:05:53.893396 643069 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0828 19:05:53.893460 643069 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0828 19:05:53.949359 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0828 19:05:53.949440 643069 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0828 19:05:53.964235 643069 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:05:53.964309 643069 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
W0828 19:05:54.013783 643069 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0828 19:05:54.013872 643069 retry.go:31] will retry after 322.619162ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
W0828 19:05:54.014241 643069 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0828 19:05:54.014297 643069 retry.go:31] will retry after 191.39354ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0828 19:05:54.022766 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0828 19:05:54.022842 643069 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0828 19:05:54.032875 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:05:54.059600 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0828 19:05:54.059681 643069 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0828 19:05:54.116872 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0828 19:05:54.116944 643069 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0828 19:05:54.145336 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0828 19:05:54.145415 643069 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0828 19:05:54.185054 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0828 19:05:54.185133 643069 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
W0828 19:05:54.195392 643069 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0828 19:05:54.195480 643069 retry.go:31] will retry after 293.188942ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0828 19:05:54.206698 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0828 19:05:54.226460 643069 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:05:54.226540 643069 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0828 19:05:54.262256 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0828 19:05:54.336943 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0828 19:05:54.489427 643069 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0828 19:05:54.334874 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:56.335646 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:05:59.042857 643069 node_ready.go:49] node "no-preload-811745" has status "Ready":"True"
I0828 19:05:59.042884 643069 node_ready.go:38] duration metric: took 5.264490992s for node "no-preload-811745" to be "Ready" ...
I0828 19:05:59.042897 643069 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0828 19:05:59.154433 643069 pod_ready.go:79] waiting up to 6m0s for pod "coredns-6f6b679f8f-bz5q4" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.237113 643069 pod_ready.go:93] pod "coredns-6f6b679f8f-bz5q4" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.237186 643069 pod_ready.go:82] duration metric: took 82.670336ms for pod "coredns-6f6b679f8f-bz5q4" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.237219 643069 pod_ready.go:79] waiting up to 6m0s for pod "coredns-6f6b679f8f-d85tk" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.261221 643069 pod_ready.go:93] pod "coredns-6f6b679f8f-d85tk" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.261247 643069 pod_ready.go:82] duration metric: took 24.003193ms for pod "coredns-6f6b679f8f-d85tk" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.261258 643069 pod_ready.go:79] waiting up to 6m0s for pod "etcd-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.287580 643069 pod_ready.go:93] pod "etcd-no-preload-811745" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.287608 643069 pod_ready.go:82] duration metric: took 26.341044ms for pod "etcd-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.287623 643069 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.294849 643069 pod_ready.go:93] pod "kube-apiserver-no-preload-811745" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.294875 643069 pod_ready.go:82] duration metric: took 7.244555ms for pod "kube-apiserver-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.294888 643069 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.300905 643069 pod_ready.go:93] pod "kube-controller-manager-no-preload-811745" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.300931 643069 pod_ready.go:82] duration metric: took 6.034481ms for pod "kube-controller-manager-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.300945 643069 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-54khk" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.660399 643069 pod_ready.go:93] pod "kube-proxy-54khk" in "kube-system" namespace has status "Ready":"True"
I0828 19:05:59.660429 643069 pod_ready.go:82] duration metric: took 359.474824ms for pod "kube-proxy-54khk" in "kube-system" namespace to be "Ready" ...
I0828 19:05:59.660441 643069 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:06:00.116263 643069 pod_ready.go:93] pod "kube-scheduler-no-preload-811745" in "kube-system" namespace has status "Ready":"True"
I0828 19:06:00.116300 643069 pod_ready.go:82] duration metric: took 455.849943ms for pod "kube-scheduler-no-preload-811745" in "kube-system" namespace to be "Ready" ...
I0828 19:06:00.116322 643069 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace to be "Ready" ...
I0828 19:06:00.321387 643069 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (6.114582469s)
I0828 19:06:01.235996 643069 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (6.973647209s)
I0828 19:06:01.236322 643069 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (6.746808564s)
I0828 19:06:01.236378 643069 addons.go:475] Verifying addon metrics-server=true in "no-preload-811745"
I0828 19:06:01.236189 643069 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (6.899175285s)
I0828 19:06:01.238965 643069 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-811745 addons enable metrics-server
I0828 19:06:01.256281 643069 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0828 19:05:58.835270 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:00.835849 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:01.259055 643069 addons.go:510] duration metric: took 7.932567454s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0828 19:06:02.122242 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:04.123156 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:03.348129 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:05.834927 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:07.835375 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:06.622086 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:08.622730 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:10.335998 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:12.338041 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:10.626029 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:12.626424 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:15.124200 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:14.841486 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:17.334506 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:17.622293 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:19.622567 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:19.834875 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:21.891228 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:21.623188 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:24.122362 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:24.335098 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:26.335545 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:26.622588 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:28.622646 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:28.840021 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:31.338658 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:31.125261 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:33.622238 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:33.836358 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:35.837036 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:37.846348 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:35.622674 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:37.622781 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:40.124507 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:40.334278 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:42.336240 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:42.622951 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:45.126225 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:44.839981 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:46.842503 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:47.622462 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:49.624560 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:49.335849 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:51.841467 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:52.123664 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:54.622363 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:54.336015 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:56.836114 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:56.622408 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:59.122542 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:06:59.335150 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:01.339095 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:01.123628 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:03.123903 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:05.127897 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:03.839487 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:06.335897 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:07.622195 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:09.625946 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:08.835302 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:10.847072 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:12.123311 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:14.622142 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:13.335296 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:15.335968 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:17.840577 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:16.622533 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:18.622994 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:20.334622 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:22.404395 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:21.122078 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:23.122879 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:25.123378 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:24.841108 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:27.336643 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:27.622522 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:30.123140 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:29.844943 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:32.335817 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:32.622286 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:35.123126 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:34.838622 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:36.840754 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:37.125652 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:39.622910 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:39.334289 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:41.338500 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:42.124268 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:44.622431 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:43.838685 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:46.334910 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:46.622904 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:49.122931 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:48.335240 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:50.839720 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:52.845484 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:51.622251 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:53.622846 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:55.333808 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:57.335080 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:56.122931 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:58.622443 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:07:59.836333 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:02.336123 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:01.123221 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:03.124114 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:04.841753 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:07.337634 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:05.622513 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:08.122916 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:09.835428 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:11.837265 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:10.623565 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:13.122138 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:15.123016 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:13.840664 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:16.334234 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:17.623015 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:19.623610 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:18.335444 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:20.840057 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:22.122538 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:24.622010 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:22.917136 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:25.337027 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:27.842024 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:27.122222 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:29.122683 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:29.843950 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:32.334667 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:31.123287 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:33.123511 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:34.334970 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:36.335232 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:35.622188 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:37.622501 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:40.123803 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:38.840600 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:41.334802 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:42.622382 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:44.622664 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:43.338760 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:45.842787 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:47.122974 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:49.130016 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:48.335332 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:50.840584 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:52.841465 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:51.621868 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:53.622829 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:55.337126 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:57.840165 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:56.122370 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:58.122700 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:00.132318 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:08:59.841202 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:01.844267 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:02.623063 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:05.122321 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:04.335030 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:06.335270 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:07.122520 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:09.123132 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:08.834703 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:10.835441 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:11.624079 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:14.123204 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:13.407484 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:15.846822 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:16.622952 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:19.122414 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:18.335367 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:20.840449 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:22.842273 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:21.122461 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:23.124544 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:25.335256 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:27.335969 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:25.623061 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:28.122201 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:30.124134 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:29.839992 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:31.840538 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:32.622393 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:35.123735 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:34.335033 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:36.836165 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:37.622597 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:40.123106 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:38.840731 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:41.335746 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:42.123940 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:44.622866 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:43.839360 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:45.840744 635612 pod_ready.go:103] pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:45.840772 635612 pod_ready.go:82] duration metric: took 4m0.012205772s for pod "metrics-server-9975d5f86-vfdm2" in "kube-system" namespace to be "Ready" ...
E0828 19:09:45.840793 635612 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0828 19:09:45.840804 635612 pod_ready.go:39] duration metric: took 5m24.765235278s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0828 19:09:45.840845 635612 api_server.go:52] waiting for apiserver process to appear ...
I0828 19:09:45.840891 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:09:45.840962 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:09:45.887113 635612 cri.go:89] found id: "5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:45.887194 635612 cri.go:89] found id: ""
I0828 19:09:45.887210 635612 logs.go:276] 1 containers: [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319]
I0828 19:09:45.887275 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.890895 635612 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:09:45.890968 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:09:45.930751 635612 cri.go:89] found id: "e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:45.930776 635612 cri.go:89] found id: ""
I0828 19:09:45.930785 635612 logs.go:276] 1 containers: [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3]
I0828 19:09:45.930843 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.934450 635612 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:09:45.934576 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:09:45.975797 635612 cri.go:89] found id: "a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:45.975822 635612 cri.go:89] found id: ""
I0828 19:09:45.975830 635612 logs.go:276] 1 containers: [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16]
I0828 19:09:45.975903 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:45.979458 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:09:45.979543 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:09:46.018424 635612 cri.go:89] found id: "f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:46.018450 635612 cri.go:89] found id: ""
I0828 19:09:46.018458 635612 logs.go:276] 1 containers: [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0]
I0828 19:09:46.018551 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.023097 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:09:46.023171 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:09:46.076288 635612 cri.go:89] found id: "ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:46.076311 635612 cri.go:89] found id: ""
I0828 19:09:46.076319 635612 logs.go:276] 1 containers: [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623]
I0828 19:09:46.076409 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.080293 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:09:46.080400 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:09:46.136404 635612 cri.go:89] found id: "e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:46.136428 635612 cri.go:89] found id: ""
I0828 19:09:46.136436 635612 logs.go:276] 1 containers: [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2]
I0828 19:09:46.136526 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.140231 635612 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:09:46.140350 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:09:46.182052 635612 cri.go:89] found id: "917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:46.182121 635612 cri.go:89] found id: ""
I0828 19:09:46.182143 635612 logs.go:276] 1 containers: [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11]
I0828 19:09:46.182207 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.185610 635612 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:09:46.185677 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:09:46.230085 635612 cri.go:89] found id: "6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:46.230111 635612 cri.go:89] found id: ""
I0828 19:09:46.230118 635612 logs.go:276] 1 containers: [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21]
I0828 19:09:46.230179 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.233921 635612 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:09:46.234029 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:09:46.272391 635612 cri.go:89] found id: "77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:46.272415 635612 cri.go:89] found id: "00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:46.272419 635612 cri.go:89] found id: ""
I0828 19:09:46.272426 635612 logs.go:276] 2 containers: [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30]
I0828 19:09:46.272512 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.275880 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:46.279309 635612 logs.go:123] Gathering logs for etcd [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3] ...
I0828 19:09:46.279335 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:46.332432 635612 logs.go:123] Gathering logs for kube-controller-manager [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2] ...
I0828 19:09:46.332463 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:46.414187 635612 logs.go:123] Gathering logs for kubernetes-dashboard [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21] ...
I0828 19:09:46.414229 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:46.454309 635612 logs.go:123] Gathering logs for CRI-O ...
I0828 19:09:46.454338 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0828 19:09:46.537491 635612 logs.go:123] Gathering logs for storage-provisioner [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb] ...
I0828 19:09:46.537526 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:46.587143 635612 logs.go:123] Gathering logs for kubelet ...
I0828 19:09:46.587174 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0828 19:09:46.639142 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875224 732 reflector.go:138] object-"kube-system"/"metrics-server-token-glpj8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-glpj8" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639377 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875341 732 reflector.go:138] object-"kube-system"/"coredns-token-24mr5": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-24mr5" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639592 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875410 732 reflector.go:138] object-"kube-system"/"kindnet-token-9j8wt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-9j8wt" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.639794 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875469 732 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640010 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875528 732 reflector.go:138] object-"kube-system"/"kube-proxy-token-h26lx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-h26lx" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640217 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875586 732 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640445 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875638 732 reflector.go:138] object-"kube-system"/"storage-provisioner-token-6t6fb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-6t6fb" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.640680 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875728 732 reflector.go:138] object-"default"/"default-token-986j2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-986j2" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:46.651199 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:22 old-k8s-version-530457 kubelet[732]: E0828 19:04:22.754021 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.652234 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:23 old-k8s-version-530457 kubelet[732]: E0828 19:04:23.617200 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.654357 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:37 old-k8s-version-530457 kubelet[732]: E0828 19:04:37.592436 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.656104 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:52 old-k8s-version-530457 kubelet[732]: E0828 19:04:52.576595 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.656693 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:57 old-k8s-version-530457 kubelet[732]: E0828 19:04:57.827984 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.657152 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:58 old-k8s-version-530457 kubelet[732]: E0828 19:04:58.830137 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.659245 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:03 old-k8s-version-530457 kubelet[732]: E0828 19:05:03.597500 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.659602 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:04 old-k8s-version-530457 kubelet[732]: E0828 19:05:04.983929 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.659789 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:15 old-k8s-version-530457 kubelet[732]: E0828 19:05:15.576664 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.660382 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:18 old-k8s-version-530457 kubelet[732]: E0828 19:05:18.866240 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.660709 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:24 old-k8s-version-530457 kubelet[732]: E0828 19:05:24.985883 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.660917 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:26 old-k8s-version-530457 kubelet[732]: E0828 19:05:26.577717 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.661245 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:37 old-k8s-version-530457 kubelet[732]: E0828 19:05:37.575455 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.661449 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:40 old-k8s-version-530457 kubelet[732]: E0828 19:05:40.577507 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.662040 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:48 old-k8s-version-530457 kubelet[732]: E0828 19:05:48.915152 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.664090 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:51 old-k8s-version-530457 kubelet[732]: E0828 19:05:51.588013 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.664430 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:54 old-k8s-version-530457 kubelet[732]: E0828 19:05:54.984049 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.664614 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:06 old-k8s-version-530457 kubelet[732]: E0828 19:06:06.575922 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.664941 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:08 old-k8s-version-530457 kubelet[732]: E0828 19:06:08.575436 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.665124 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:20 old-k8s-version-530457 kubelet[732]: E0828 19:06:20.576354 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.665502 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:23 old-k8s-version-530457 kubelet[732]: E0828 19:06:23.575443 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.665689 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:32 old-k8s-version-530457 kubelet[732]: E0828 19:06:32.577946 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.666344 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:36 old-k8s-version-530457 kubelet[732]: E0828 19:06:36.981996 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.666539 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:43 old-k8s-version-530457 kubelet[732]: E0828 19:06:43.575953 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.666866 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:44 old-k8s-version-530457 kubelet[732]: E0828 19:06:44.984324 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.667051 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:56 old-k8s-version-530457 kubelet[732]: E0828 19:06:56.575945 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.667383 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:58 old-k8s-version-530457 kubelet[732]: E0828 19:06:58.575915 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.667696 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576302 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.667893 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576985 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.668223 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:24 old-k8s-version-530457 kubelet[732]: E0828 19:07:24.575460 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.670290 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:25 old-k8s-version-530457 kubelet[732]: E0828 19:07:25.586438 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:46.670477 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:36 old-k8s-version-530457 kubelet[732]: E0828 19:07:36.577014 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.670808 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:38 old-k8s-version-530457 kubelet[732]: E0828 19:07:38.575534 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.670991 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:48 old-k8s-version-530457 kubelet[732]: E0828 19:07:48.576137 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.672571 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:49 old-k8s-version-530457 kubelet[732]: E0828 19:07:49.575420 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.672780 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:00 old-k8s-version-530457 kubelet[732]: E0828 19:08:00.576139 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.673388 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:06 old-k8s-version-530457 kubelet[732]: E0828 19:08:06.130079 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.673573 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.576054 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.673902 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.983952 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674234 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:25 old-k8s-version-530457 kubelet[732]: E0828 19:08:25.575482 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674417 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:29 old-k8s-version-530457 kubelet[732]: E0828 19:08:29.575996 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.674752 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:37 old-k8s-version-530457 kubelet[732]: E0828 19:08:37.575411 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.674937 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:43 old-k8s-version-530457 kubelet[732]: E0828 19:08:43.575929 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.675265 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: E0828 19:08:50.577123 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.675449 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:56 old-k8s-version-530457 kubelet[732]: E0828 19:08:56.576060 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.675780 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: E0828 19:09:04.575429 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.675975 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:07 old-k8s-version-530457 kubelet[732]: E0828 19:09:07.576124 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.676552 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: E0828 19:09:17.575422 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.676737 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.677066 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.677250 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:46.677586 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:46.677772 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0828 19:09:46.677782 635612 logs.go:123] Gathering logs for describe nodes ...
I0828 19:09:46.677797 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0828 19:09:46.858651 635612 logs.go:123] Gathering logs for kube-apiserver [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319] ...
I0828 19:09:46.858682 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:46.933892 635612 logs.go:123] Gathering logs for kube-scheduler [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0] ...
I0828 19:09:46.933928 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:46.980032 635612 logs.go:123] Gathering logs for kube-proxy [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623] ...
I0828 19:09:46.980061 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:47.023718 635612 logs.go:123] Gathering logs for kindnet [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11] ...
I0828 19:09:47.023747 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:47.073250 635612 logs.go:123] Gathering logs for container status ...
I0828 19:09:47.073287 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:09:47.137663 635612 logs.go:123] Gathering logs for dmesg ...
I0828 19:09:47.137699 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:09:47.158656 635612 logs.go:123] Gathering logs for coredns [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16] ...
I0828 19:09:47.158722 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:47.205508 635612 logs.go:123] Gathering logs for storage-provisioner [00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30] ...
I0828 19:09:47.205539 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:47.242809 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:47.242837 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0828 19:09:47.242887 635612 out.go:270] X Problems detected in kubelet:
W0828 19:09:47.242904 635612 out.go:270] Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:47.242914 635612 out.go:270] Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:47.242936 635612 out.go:270] Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:47.242946 635612 out.go:270] Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:47.242980 635612 out.go:270] Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0828 19:09:47.242995 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:47.243001 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:09:47.122466 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:49.122920 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:51.123762 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:53.622152 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:57.245126 635612 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0828 19:09:57.258795 635612 api_server.go:72] duration metric: took 5m54.643340669s to wait for apiserver process to appear ...
I0828 19:09:57.258830 635612 api_server.go:88] waiting for apiserver healthz status ...
I0828 19:09:57.258871 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:09:57.258935 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:09:57.309643 635612 cri.go:89] found id: "5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:57.309671 635612 cri.go:89] found id: ""
I0828 19:09:57.309680 635612 logs.go:276] 1 containers: [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319]
I0828 19:09:57.309735 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.313665 635612 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:09:57.313748 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:09:57.357570 635612 cri.go:89] found id: "e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:57.357588 635612 cri.go:89] found id: ""
I0828 19:09:57.357596 635612 logs.go:276] 1 containers: [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3]
I0828 19:09:57.357650 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.363443 635612 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:09:57.363523 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:09:57.401091 635612 cri.go:89] found id: "a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:57.401111 635612 cri.go:89] found id: ""
I0828 19:09:57.401119 635612 logs.go:276] 1 containers: [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16]
I0828 19:09:57.401180 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.404820 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:09:57.404892 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:09:57.441795 635612 cri.go:89] found id: "f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:57.441812 635612 cri.go:89] found id: ""
I0828 19:09:57.441820 635612 logs.go:276] 1 containers: [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0]
I0828 19:09:57.441876 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.445389 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:09:57.445463 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:09:57.481786 635612 cri.go:89] found id: "ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:57.481807 635612 cri.go:89] found id: ""
I0828 19:09:57.481815 635612 logs.go:276] 1 containers: [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623]
I0828 19:09:57.481875 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.485535 635612 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:09:57.485616 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:09:57.526371 635612 cri.go:89] found id: "e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:57.526396 635612 cri.go:89] found id: ""
I0828 19:09:57.526404 635612 logs.go:276] 1 containers: [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2]
I0828 19:09:57.526471 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.530191 635612 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:09:57.530266 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:09:57.574140 635612 cri.go:89] found id: "917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:57.574167 635612 cri.go:89] found id: ""
I0828 19:09:57.574175 635612 logs.go:276] 1 containers: [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11]
I0828 19:09:57.574232 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.577884 635612 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:09:57.577963 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:09:57.625232 635612 cri.go:89] found id: "6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:57.625299 635612 cri.go:89] found id: ""
I0828 19:09:57.625335 635612 logs.go:276] 1 containers: [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21]
I0828 19:09:57.625395 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.628836 635612 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:09:57.628951 635612 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:09:57.675453 635612 cri.go:89] found id: "77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:57.675476 635612 cri.go:89] found id: "00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:57.675480 635612 cri.go:89] found id: ""
I0828 19:09:57.675487 635612 logs.go:276] 2 containers: [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30]
I0828 19:09:57.675544 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.679544 635612 ssh_runner.go:195] Run: which crictl
I0828 19:09:57.682894 635612 logs.go:123] Gathering logs for kube-proxy [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623] ...
I0828 19:09:57.682919 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623"
I0828 19:09:57.730503 635612 logs.go:123] Gathering logs for container status ...
I0828 19:09:57.730532 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:09:57.773015 635612 logs.go:123] Gathering logs for kube-apiserver [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319] ...
I0828 19:09:57.773043 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319"
I0828 19:09:57.846966 635612 logs.go:123] Gathering logs for coredns [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16] ...
I0828 19:09:57.847003 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16"
I0828 19:09:55.623016 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:09:58.122817 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:10:00.147019 643069 pod_ready.go:103] pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace has status "Ready":"False"
I0828 19:10:00.147138 643069 pod_ready.go:82] duration metric: took 4m0.030804473s for pod "metrics-server-6867b74b74-5rnp9" in "kube-system" namespace to be "Ready" ...
E0828 19:10:00.147187 643069 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0828 19:10:00.147228 643069 pod_ready.go:39] duration metric: took 4m1.104320173s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0828 19:10:00.147266 643069 api_server.go:52] waiting for apiserver process to appear ...
I0828 19:10:00.147333 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:10:00.147423 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:10:00.282844 643069 cri.go:89] found id: "fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c"
I0828 19:10:00.282867 643069 cri.go:89] found id: ""
I0828 19:10:00.282876 643069 logs.go:276] 1 containers: [fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c]
I0828 19:10:00.282942 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.296739 643069 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:10:00.296829 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:10:00.382473 643069 cri.go:89] found id: "c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae"
I0828 19:10:00.382497 643069 cri.go:89] found id: ""
I0828 19:10:00.382506 643069 logs.go:276] 1 containers: [c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae]
I0828 19:10:00.382573 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.386849 643069 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:10:00.386969 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:09:57.896268 635612 logs.go:123] Gathering logs for kindnet [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11] ...
I0828 19:09:57.896300 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11"
I0828 19:09:57.944005 635612 logs.go:123] Gathering logs for kubelet ...
I0828 19:09:57.944039 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0828 19:09:57.994339 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875224 732 reflector.go:138] object-"kube-system"/"metrics-server-token-glpj8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-glpj8" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.994628 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875341 732 reflector.go:138] object-"kube-system"/"coredns-token-24mr5": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-24mr5" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.994855 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875410 732 reflector.go:138] object-"kube-system"/"kindnet-token-9j8wt": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-9j8wt" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995069 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875469 732 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995292 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875528 732 reflector.go:138] object-"kube-system"/"kube-proxy-token-h26lx": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-h26lx" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995504 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875586 732 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995739 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875638 732 reflector.go:138] object-"kube-system"/"storage-provisioner-token-6t6fb": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-6t6fb" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:57.995954 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:20 old-k8s-version-530457 kubelet[732]: E0828 19:04:20.875728 732 reflector.go:138] object-"default"/"default-token-986j2": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-986j2" is forbidden: User "system:node:old-k8s-version-530457" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-530457' and this object
W0828 19:09:58.007422 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:22 old-k8s-version-530457 kubelet[732]: E0828 19:04:22.754021 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.008467 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:23 old-k8s-version-530457 kubelet[732]: E0828 19:04:23.617200 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.010557 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:37 old-k8s-version-530457 kubelet[732]: E0828 19:04:37.592436 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.012403 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:52 old-k8s-version-530457 kubelet[732]: E0828 19:04:52.576595 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.013000 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:57 old-k8s-version-530457 kubelet[732]: E0828 19:04:57.827984 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.013543 635612 logs.go:138] Found kubelet problem: Aug 28 19:04:58 old-k8s-version-530457 kubelet[732]: E0828 19:04:58.830137 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.015601 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:03 old-k8s-version-530457 kubelet[732]: E0828 19:05:03.597500 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.015933 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:04 old-k8s-version-530457 kubelet[732]: E0828 19:05:04.983929 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.016119 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:15 old-k8s-version-530457 kubelet[732]: E0828 19:05:15.576664 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.016708 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:18 old-k8s-version-530457 kubelet[732]: E0828 19:05:18.866240 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017093 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:24 old-k8s-version-530457 kubelet[732]: E0828 19:05:24.985883 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017281 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:26 old-k8s-version-530457 kubelet[732]: E0828 19:05:26.577717 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.017621 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:37 old-k8s-version-530457 kubelet[732]: E0828 19:05:37.575455 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.017811 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:40 old-k8s-version-530457 kubelet[732]: E0828 19:05:40.577507 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.018412 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:48 old-k8s-version-530457 kubelet[732]: E0828 19:05:48.915152 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.020489 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:51 old-k8s-version-530457 kubelet[732]: E0828 19:05:51.588013 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.020823 635612 logs.go:138] Found kubelet problem: Aug 28 19:05:54 old-k8s-version-530457 kubelet[732]: E0828 19:05:54.984049 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.021009 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:06 old-k8s-version-530457 kubelet[732]: E0828 19:06:06.575922 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.021343 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:08 old-k8s-version-530457 kubelet[732]: E0828 19:06:08.575436 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.021531 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:20 old-k8s-version-530457 kubelet[732]: E0828 19:06:20.576354 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.021861 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:23 old-k8s-version-530457 kubelet[732]: E0828 19:06:23.575443 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.022044 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:32 old-k8s-version-530457 kubelet[732]: E0828 19:06:32.577946 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.022634 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:36 old-k8s-version-530457 kubelet[732]: E0828 19:06:36.981996 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.022817 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:43 old-k8s-version-530457 kubelet[732]: E0828 19:06:43.575953 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.023144 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:44 old-k8s-version-530457 kubelet[732]: E0828 19:06:44.984324 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.023333 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:56 old-k8s-version-530457 kubelet[732]: E0828 19:06:56.575945 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.023661 635612 logs.go:138] Found kubelet problem: Aug 28 19:06:58 old-k8s-version-530457 kubelet[732]: E0828 19:06:58.575915 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.023973 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576302 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.024171 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:11 old-k8s-version-530457 kubelet[732]: E0828 19:07:11.576985 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.024497 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:24 old-k8s-version-530457 kubelet[732]: E0828 19:07:24.575460 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.026567 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:25 old-k8s-version-530457 kubelet[732]: E0828 19:07:25.586438 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0828 19:09:58.026756 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:36 old-k8s-version-530457 kubelet[732]: E0828 19:07:36.577014 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.027094 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:38 old-k8s-version-530457 kubelet[732]: E0828 19:07:38.575534 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.027280 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:48 old-k8s-version-530457 kubelet[732]: E0828 19:07:48.576137 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.027608 635612 logs.go:138] Found kubelet problem: Aug 28 19:07:49 old-k8s-version-530457 kubelet[732]: E0828 19:07:49.575420 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.027792 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:00 old-k8s-version-530457 kubelet[732]: E0828 19:08:00.576139 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.028380 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:06 old-k8s-version-530457 kubelet[732]: E0828 19:08:06.130079 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.028563 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.576054 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.028891 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:14 old-k8s-version-530457 kubelet[732]: E0828 19:08:14.983952 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029219 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:25 old-k8s-version-530457 kubelet[732]: E0828 19:08:25.575482 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029448 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:29 old-k8s-version-530457 kubelet[732]: E0828 19:08:29.575996 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.029778 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:37 old-k8s-version-530457 kubelet[732]: E0828 19:08:37.575411 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.029963 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:43 old-k8s-version-530457 kubelet[732]: E0828 19:08:43.575929 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.030289 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: E0828 19:08:50.577123 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.030479 635612 logs.go:138] Found kubelet problem: Aug 28 19:08:56 old-k8s-version-530457 kubelet[732]: E0828 19:08:56.576060 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.030807 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: E0828 19:09:04.575429 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.031061 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:07 old-k8s-version-530457 kubelet[732]: E0828 19:09:07.576124 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.031639 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: E0828 19:09:17.575422 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.031826 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.032153 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.032340 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.032672 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.032858 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.033172 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.033376 635612 logs.go:138] Found kubelet problem: Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
I0828 19:09:58.033389 635612 logs.go:123] Gathering logs for kubernetes-dashboard [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21] ...
I0828 19:09:58.033406 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21"
I0828 19:09:58.089976 635612 logs.go:123] Gathering logs for storage-provisioner [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb] ...
I0828 19:09:58.090005 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb"
I0828 19:09:58.135562 635612 logs.go:123] Gathering logs for storage-provisioner [00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30] ...
I0828 19:09:58.135591 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30"
I0828 19:09:58.173412 635612 logs.go:123] Gathering logs for kube-scheduler [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0] ...
I0828 19:09:58.173445 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0"
I0828 19:09:58.237093 635612 logs.go:123] Gathering logs for describe nodes ...
I0828 19:09:58.237121 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0828 19:09:58.383455 635612 logs.go:123] Gathering logs for etcd [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3] ...
I0828 19:09:58.383491 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3"
I0828 19:09:58.434930 635612 logs.go:123] Gathering logs for kube-controller-manager [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2] ...
I0828 19:09:58.434968 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2"
I0828 19:09:58.517578 635612 logs.go:123] Gathering logs for CRI-O ...
I0828 19:09:58.517612 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0828 19:09:58.606571 635612 logs.go:123] Gathering logs for dmesg ...
I0828 19:09:58.606606 635612 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:09:58.626574 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:58.626597 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0828 19:09:58.626648 635612 out.go:270] X Problems detected in kubelet:
W0828 19:09:58.626663 635612 out.go:270] Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626669 635612 out.go:270] Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
W0828 19:09:58.626676 635612 out.go:270] Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626686 635612 out.go:270] Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0828 19:09:58.626695 635612 out.go:270] Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
I0828 19:09:58.626705 635612 out.go:358] Setting ErrFile to fd 2...
I0828 19:09:58.626710 635612 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0828 19:10:00.462985 643069 cri.go:89] found id: "b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6"
I0828 19:10:00.463013 643069 cri.go:89] found id: "ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e"
I0828 19:10:00.463019 643069 cri.go:89] found id: ""
I0828 19:10:00.463026 643069 logs.go:276] 2 containers: [b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6 ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e]
I0828 19:10:00.463138 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.467172 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.471105 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:10:00.471251 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:10:00.514143 643069 cri.go:89] found id: "db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939"
I0828 19:10:00.514176 643069 cri.go:89] found id: ""
I0828 19:10:00.514185 643069 logs.go:276] 1 containers: [db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939]
I0828 19:10:00.514264 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.518130 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:10:00.518206 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:10:00.562561 643069 cri.go:89] found id: "6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7"
I0828 19:10:00.562588 643069 cri.go:89] found id: ""
I0828 19:10:00.562596 643069 logs.go:276] 1 containers: [6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7]
I0828 19:10:00.562659 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.566900 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:10:00.566978 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:10:00.615042 643069 cri.go:89] found id: "6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098"
I0828 19:10:00.615076 643069 cri.go:89] found id: ""
I0828 19:10:00.615085 643069 logs.go:276] 1 containers: [6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098]
I0828 19:10:00.615154 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.619267 643069 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:10:00.619413 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:10:00.659631 643069 cri.go:89] found id: "1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361"
I0828 19:10:00.659655 643069 cri.go:89] found id: ""
I0828 19:10:00.659663 643069 logs.go:276] 1 containers: [1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361]
I0828 19:10:00.659729 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.664325 643069 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:10:00.664432 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:10:00.718257 643069 cri.go:89] found id: "3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de"
I0828 19:10:00.718280 643069 cri.go:89] found id: ""
I0828 19:10:00.718288 643069 logs.go:276] 1 containers: [3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de]
I0828 19:10:00.718374 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.722081 643069 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:10:00.722197 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:10:00.762908 643069 cri.go:89] found id: "657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101"
I0828 19:10:00.762980 643069 cri.go:89] found id: "fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad"
I0828 19:10:00.762993 643069 cri.go:89] found id: ""
I0828 19:10:00.763002 643069 logs.go:276] 2 containers: [657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101 fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad]
I0828 19:10:00.763063 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.766763 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:00.770169 643069 logs.go:123] Gathering logs for kube-proxy [6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7] ...
I0828 19:10:00.770194 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7"
I0828 19:10:00.812556 643069 logs.go:123] Gathering logs for CRI-O ...
I0828 19:10:00.812583 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0828 19:10:00.895582 643069 logs.go:123] Gathering logs for dmesg ...
I0828 19:10:00.895620 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:10:00.913619 643069 logs.go:123] Gathering logs for describe nodes ...
I0828 19:10:00.913648 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0828 19:10:01.060935 643069 logs.go:123] Gathering logs for kube-scheduler [db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939] ...
I0828 19:10:01.060971 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939"
I0828 19:10:01.104736 643069 logs.go:123] Gathering logs for kindnet [1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361] ...
I0828 19:10:01.104764 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361"
I0828 19:10:01.161537 643069 logs.go:123] Gathering logs for storage-provisioner [fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad] ...
I0828 19:10:01.161573 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad"
I0828 19:10:01.209945 643069 logs.go:123] Gathering logs for etcd [c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae] ...
I0828 19:10:01.209978 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae"
I0828 19:10:01.264284 643069 logs.go:123] Gathering logs for coredns [b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6] ...
I0828 19:10:01.264320 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6"
I0828 19:10:01.318474 643069 logs.go:123] Gathering logs for kube-controller-manager [6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098] ...
I0828 19:10:01.318510 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098"
I0828 19:10:01.416953 643069 logs.go:123] Gathering logs for kubernetes-dashboard [3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de] ...
I0828 19:10:01.416999 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de"
I0828 19:10:01.477601 643069 logs.go:123] Gathering logs for storage-provisioner [657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101] ...
I0828 19:10:01.477633 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101"
I0828 19:10:01.537777 643069 logs.go:123] Gathering logs for container status ...
I0828 19:10:01.537811 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:10:01.609469 643069 logs.go:123] Gathering logs for kubelet ...
I0828 19:10:01.609505 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0828 19:10:01.704063 643069 logs.go:123] Gathering logs for kube-apiserver [fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c] ...
I0828 19:10:01.704157 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c"
I0828 19:10:01.768159 643069 logs.go:123] Gathering logs for coredns [ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e] ...
I0828 19:10:01.768255 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e"
I0828 19:10:04.344741 643069 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0828 19:10:04.357498 643069 api_server.go:72] duration metric: took 4m11.031357048s to wait for apiserver process to appear ...
I0828 19:10:04.357524 643069 api_server.go:88] waiting for apiserver healthz status ...
I0828 19:10:04.357569 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0828 19:10:04.357629 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0828 19:10:04.403793 643069 cri.go:89] found id: "fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c"
I0828 19:10:04.403815 643069 cri.go:89] found id: ""
I0828 19:10:04.403835 643069 logs.go:276] 1 containers: [fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c]
I0828 19:10:04.403894 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.407493 643069 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0828 19:10:04.407562 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0828 19:10:04.462316 643069 cri.go:89] found id: "c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae"
I0828 19:10:04.462387 643069 cri.go:89] found id: ""
I0828 19:10:04.462409 643069 logs.go:276] 1 containers: [c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae]
I0828 19:10:04.462493 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.466211 643069 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0828 19:10:04.466282 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0828 19:10:04.506725 643069 cri.go:89] found id: "b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6"
I0828 19:10:04.506747 643069 cri.go:89] found id: "ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e"
I0828 19:10:04.506751 643069 cri.go:89] found id: ""
I0828 19:10:04.506759 643069 logs.go:276] 2 containers: [b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6 ff55524b25cf965c1c624bddcd34a292c72c11411684458dd98c83956600dc4e]
I0828 19:10:04.506817 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.510787 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.514106 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0828 19:10:04.514171 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0828 19:10:04.555350 643069 cri.go:89] found id: "db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939"
I0828 19:10:04.555374 643069 cri.go:89] found id: ""
I0828 19:10:04.555382 643069 logs.go:276] 1 containers: [db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939]
I0828 19:10:04.555461 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.559018 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0828 19:10:04.559122 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0828 19:10:04.597548 643069 cri.go:89] found id: "6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7"
I0828 19:10:04.597610 643069 cri.go:89] found id: ""
I0828 19:10:04.597625 643069 logs.go:276] 1 containers: [6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7]
I0828 19:10:04.597687 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.601121 643069 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0828 19:10:04.601186 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0828 19:10:04.638702 643069 cri.go:89] found id: "6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098"
I0828 19:10:04.638781 643069 cri.go:89] found id: ""
I0828 19:10:04.638796 643069 logs.go:276] 1 containers: [6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098]
I0828 19:10:04.638867 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.642465 643069 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0828 19:10:04.642542 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0828 19:10:04.682791 643069 cri.go:89] found id: "1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361"
I0828 19:10:04.682856 643069 cri.go:89] found id: ""
I0828 19:10:04.682871 643069 logs.go:276] 1 containers: [1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361]
I0828 19:10:04.682933 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.686637 643069 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0828 19:10:04.686712 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0828 19:10:04.739225 643069 cri.go:89] found id: "657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101"
I0828 19:10:04.739247 643069 cri.go:89] found id: "fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad"
I0828 19:10:04.739253 643069 cri.go:89] found id: ""
I0828 19:10:04.739262 643069 logs.go:276] 2 containers: [657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101 fe9e2440bb39979b8224da5dfeb5a11363ef0e2471b8ec95c13d9ccb9e3649ad]
I0828 19:10:04.739335 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.742990 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.746298 643069 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0828 19:10:04.746360 643069 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0828 19:10:04.784419 643069 cri.go:89] found id: "3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de"
I0828 19:10:04.784442 643069 cri.go:89] found id: ""
I0828 19:10:04.784451 643069 logs.go:276] 1 containers: [3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de]
I0828 19:10:04.784528 643069 ssh_runner.go:195] Run: which crictl
I0828 19:10:04.788928 643069 logs.go:123] Gathering logs for kube-apiserver [fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c] ...
I0828 19:10:04.788954 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fecbc065934f21b6cc7459e66f9eded6742793dd95182326bcb601ca8c87fd0c"
I0828 19:10:04.841568 643069 logs.go:123] Gathering logs for etcd [c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae] ...
I0828 19:10:04.841655 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c9c9adb4f8c5de0cfe6da656d38cf04de580c2828545972926a8d0f064c32dae"
I0828 19:10:04.891799 643069 logs.go:123] Gathering logs for kubelet ...
I0828 19:10:04.891832 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
I0828 19:10:04.983224 643069 logs.go:123] Gathering logs for dmesg ...
I0828 19:10:04.983257 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0828 19:10:04.999577 643069 logs.go:123] Gathering logs for kube-scheduler [db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939] ...
I0828 19:10:04.999605 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 db7e6149d905cc2e9516dfadd06f5e5c632aac9cc219f80f5aa3fffc3f2f6939"
I0828 19:10:05.049539 643069 logs.go:123] Gathering logs for kube-proxy [6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7] ...
I0828 19:10:05.049569 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6e2c7760e7ab0ddf978d7892de14cb1d771c99456d9e4444f9eda310a6a480f7"
I0828 19:10:05.094654 643069 logs.go:123] Gathering logs for kindnet [1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361] ...
I0828 19:10:05.094685 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b55716fb7cf2747633664f6564b3d084ba80fd7fa14b1544baf8b37ccafc361"
I0828 19:10:05.149995 643069 logs.go:123] Gathering logs for container status ...
I0828 19:10:05.150024 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0828 19:10:05.201372 643069 logs.go:123] Gathering logs for coredns [b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6] ...
I0828 19:10:05.201403 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b1d3bd2bfdb0d82b830dc736594b119384d021b484088b36acb4f541416e33d6"
I0828 19:10:05.259217 643069 logs.go:123] Gathering logs for kube-controller-manager [6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098] ...
I0828 19:10:05.259249 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6768c1525e8d4cef6e31525976a5adfe14ba7d2cf06a6929f7234af80ae60098"
I0828 19:10:05.321066 643069 logs.go:123] Gathering logs for storage-provisioner [657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101] ...
I0828 19:10:05.321096 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 657d9514239b2c92bf12023f12b91704959c35777811969efe7f991a69a7c101"
I0828 19:10:05.361351 643069 logs.go:123] Gathering logs for kubernetes-dashboard [3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de] ...
I0828 19:10:05.361431 643069 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3d92bc35633f8d53546a496de7d9050c5cd2f62bfb4d24e7aa5f0df46894d3de"
I0828 19:10:08.628180 635612 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0828 19:10:08.641074 635612 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0828 19:10:08.644092 635612 out.go:201]
W0828 19:10:08.646685 635612 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0828 19:10:08.646728 635612 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0828 19:10:08.646753 635612 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0828 19:10:08.646763 635612 out.go:270] *
W0828 19:10:08.647693 635612 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0828 19:10:08.649411 635612 out.go:201]
==> CRI-O <==
Aug 28 19:08:05 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:05.146186350Z" level=info msg="Removed container 8c727b58911f673d7fd73cd5e31b424c8f33bbc9c22a1ad13b6d0bae9a04ccbe: kubernetes-dashboard/dashboard-metrics-scraper-8d5bb5db8-k85jn/dashboard-metrics-scraper" id=bcfa9c0e-c53a-4e10-963b-0748589c7c5d name=/runtime.v1alpha2.RuntimeService/RemoveContainer
Aug 28 19:08:14 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:14.575430999Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=3938b818-73e8-461b-8da0-ab13c47bad93 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:14 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:14.575672044Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=3938b818-73e8-461b-8da0-ab13c47bad93 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:29 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:29.575469490Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=c8fe18e7-043a-4650-8950-c90f9a3deb4b name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:29 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:29.575708655Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=c8fe18e7-043a-4650-8950-c90f9a3deb4b name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:43 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:43.575467162Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=4b99686d-fdf7-436f-8895-d764dc242e62 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:43 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:43.575709814Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=4b99686d-fdf7-436f-8895-d764dc242e62 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:56 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:56.575608581Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=74100247-a208-418a-8327-c4f8eddd6d5e name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:08:56 old-k8s-version-530457 crio[621]: time="2024-08-28 19:08:56.575850380Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=74100247-a208-418a-8327-c4f8eddd6d5e name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:07.575461800Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=0cddefc2-0adf-401a-a99a-7bff7e461559 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:07.575738117Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=0cddefc2-0adf-401a-a99a-7bff7e461559 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:10 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:10.462040100Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.2" id=94604464-fe99-431c-9909-41a6624de28e name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:10 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:10.462290038Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2a060e2e7101d419352bf82c613158587400be743482d9a537ec4a9d1b4eb93c,RepoTags:[k8s.gcr.io/pause:3.2 registry.k8s.io/pause:3.2],RepoDigests:[k8s.gcr.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f registry.k8s.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 registry.k8s.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f],Size_:489397,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=94604464-fe99-431c-9909-41a6624de28e name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:22 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:22.575420643Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=720bd8c7-855f-4155-8c51-dd8543668636 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:22 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:22.575663542Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=720bd8c7-855f-4155-8c51-dd8543668636 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:33 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:33.575783132Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=da0a4b62-7660-4201-8b43-352100159690 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:33 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:33.576033259Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=da0a4b62-7660-4201-8b43-352100159690 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:44 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:44.575413571Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=7de54b73-5cb5-48ad-96d9-26ec913a141b name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:44 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:44.575640076Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=7de54b73-5cb5-48ad-96d9-26ec913a141b name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:55 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:55.575969268Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=29bc1aad-38ae-4c17-adda-54b2799afef4 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:09:55 old-k8s-version-530457 crio[621]: time="2024-08-28 19:09:55.576230881Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=29bc1aad-38ae-4c17-adda-54b2799afef4 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:10:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:10:07.575484111Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=b289479c-652e-4888-9028-787708c27a98 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:10:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:10:07.575751378Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=b289479c-652e-4888-9028-787708c27a98 name=/runtime.v1alpha2.ImageService/ImageStatus
Aug 28 19:10:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:10:07.576472475Z" level=info msg="Pulling image: fake.domain/registry.k8s.io/echoserver:1.4" id=10f36b2c-730a-4ea1-b9b3-dee04d1a77b5 name=/runtime.v1alpha2.ImageService/PullImage
Aug 28 19:10:07 old-k8s-version-530457 crio[621]: time="2024-08-28 19:10:07.581447622Z" level=info msg="Trying to access \"fake.domain/registry.k8s.io/echoserver:1.4\""
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
d972bae8e8721 a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 2 minutes ago Exited dashboard-metrics-scraper 5 aa936d1024cca dashboard-metrics-scraper-8d5bb5db8-k85jn
77caa541b8678 ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Running storage-provisioner 1 34688991080d7 storage-provisioner
6ade1a63523dd docker.io/kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93 5 minutes ago Running kubernetes-dashboard 0 f0bd4d302099e kubernetes-dashboard-cd95d586-hts44
a69857fb717e7 db91994f4ee8f894a1e8a6c1a76f615da8fc3c019300a3686291ce6fcbc57895 5 minutes ago Running coredns 0 7e7fa4d4b94cc coredns-74ff55c5b-z6sb7
917e18e8ed4ea 6a23fa8fd2b78ab58e42ba273808edc936a9c53d8ac4a919f6337be094843a51 5 minutes ago Running kindnet-cni 0 363fe20f4c433 kindnet-4pdhj
ee22e16883b2e 25a5233254979d0678a2db1d15b76b73dc380d81bc5eed93916ba5638b3cd894 5 minutes ago Running kube-proxy 0 d39c1c57ef636 kube-proxy-mzrqc
00f6d3ef896cb ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Exited storage-provisioner 0 34688991080d7 storage-provisioner
9ae5faad8bbc1 1611cd07b61d57dbbfebe6db242513fd51e1c02d20ba08af17a45837d86a8a8c 5 minutes ago Running busybox 0 11c7e4106d9cd busybox
f3c41ad57f8c9 e7605f88f17d6a4c3f083ef9c6f5f19b39f87e4d4406a05a8612b54a6ea57051 5 minutes ago Running kube-scheduler 0 87ff9e09082f0 kube-scheduler-old-k8s-version-530457
e98d5ab738e76 05b738aa1bc6355db8a2ee8639f3631b908286e43f584a3d2ee0c472de033c28 5 minutes ago Running etcd 0 a78fb8c182078 etcd-old-k8s-version-530457
e93b5ce24805a 1df8a2b116bd16f7070fd383a6769c8d644b365575e8ffa3e492b84e4f05fc74 5 minutes ago Running kube-controller-manager 0 9e087a3bd72a5 kube-controller-manager-old-k8s-version-530457
5c39e778b2c41 2c08bbbc02d3aa5dfbf4e79f15c0a61424049288917aa10364464ca1f7de7157 5 minutes ago Running kube-apiserver 0 fe053bbf2290b kube-apiserver-old-k8s-version-530457
==> coredns [a69857fb717e701f0ce8eb4b23787eaab7ff7a135acbba2ddc996facdfdedd16] <==
I0828 19:04:53.857754 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-28 19:04:23.850179499 +0000 UTC m=+0.102669137) (total time: 30.00748018s):
Trace[2019727887]: [30.00748018s] [30.00748018s] END
E0828 19:04:53.857792 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0828 19:04:53.893544 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-28 19:04:23.858825015 +0000 UTC m=+0.111314645) (total time: 30.034686951s):
Trace[939984059]: [30.034686951s] [30.034686951s] END
E0828 19:04:53.893564 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0828 19:04:53.893644 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-28 19:04:23.850644334 +0000 UTC m=+0.103133964) (total time: 30.042988582s):
Trace[1474941318]: [30.042988582s] [30.042988582s] END
E0828 19:04:53.893657 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:50592 - 12407 "HINFO IN 3346656017079628500.1588017741253354354. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.053801328s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
[INFO] plugin/ready: Still waiting on: "kubernetes"
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:49279 - 1138 "HINFO IN 3957492191768777129.9186778910910367549. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.028807276s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
==> describe nodes <==
Name: old-k8s-version-530457
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-530457
kubernetes.io/os=linux
minikube.k8s.io/commit=6f256f0bf490fd67de29a75a245d072e85b1b216
minikube.k8s.io/name=old-k8s-version-530457
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_08_28T19_01_54_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 28 Aug 2024 19:01:51 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-530457
AcquireTime: <unset>
RenewTime: Wed, 28 Aug 2024 19:10:03 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Wed, 28 Aug 2024 19:05:12 +0000 Wed, 28 Aug 2024 19:01:45 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Wed, 28 Aug 2024 19:05:12 +0000 Wed, 28 Aug 2024 19:01:45 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Wed, 28 Aug 2024 19:05:12 +0000 Wed, 28 Aug 2024 19:01:45 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Wed, 28 Aug 2024 19:05:12 +0000 Wed, 28 Aug 2024 19:02:32 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-530457
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022296Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022296Ki
pods: 110
System Info:
Machine ID: 7d76f2458b714210be0a986221b796be
System UUID: 6c41387e-40cd-427e-9202-36c9b49c692c
Boot ID: 44112deb-fcf3-4af3-8a5d-e2078ba5f67b
Kernel Version: 5.15.0-1068-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: cri-o://1.24.6
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m43s
kube-system coredns-74ff55c5b-z6sb7 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m
kube-system etcd-old-k8s-version-530457 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m8s
kube-system kindnet-4pdhj 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m
kube-system kube-apiserver-old-k8s-version-530457 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m8s
kube-system kube-controller-manager-old-k8s-version-530457 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m8s
kube-system kube-proxy-mzrqc 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m
kube-system kube-scheduler-old-k8s-version-530457 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m8s
kube-system metrics-server-9975d5f86-vfdm2 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m30s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 7m58s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-k85jn 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m29s
kubernetes-dashboard kubernetes-dashboard-cd95d586-hts44 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m29s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m27s (x5 over 8m27s) kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m27s (x5 over 8m27s) kubelet Node old-k8s-version-530457 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m27s (x4 over 8m27s) kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientPID
Normal Starting 8m8s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m8s kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m8s kubelet Node old-k8s-version-530457 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m8s kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientPID
Normal Starting 7m58s kube-proxy Starting kube-proxy.
Normal NodeReady 7m38s kubelet Node old-k8s-version-530457 status is now: NodeReady
Normal Starting 6m kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m (x8 over 6m) kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m (x8 over 6m) kubelet Node old-k8s-version-530457 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m (x8 over 6m) kubelet Node old-k8s-version-530457 status is now: NodeHasSufficientPID
Normal Starting 5m46s kube-proxy Starting kube-proxy.
==> dmesg <==
[Aug28 17:30] kmem.limit_in_bytes is deprecated and will be removed. Please report your usecase to linux-mm@kvack.org if you depend on this functionality.
[Aug28 18:58] overlayfs: '/var/lib/containers/storage/overlay/l/Q2QJNMTVZL6GMULS36RA5ZJGSA' not a directory
==> etcd [e98d5ab738e766e2c37e681767c94d6dc4d5544f18f7033b432f85daa0f66ba3] <==
2024-08-28 19:06:06.533064 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:06:16.532921 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:06:26.532945 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:06:36.533663 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:06:46.532917 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:06:56.532903 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:06.533110 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:16.533962 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:26.533400 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:36.533264 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:46.532937 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:07:56.532966 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:06.532922 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:16.533022 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:26.532948 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:36.533018 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:46.532944 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:08:56.532984 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:06.532900 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:16.532979 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:26.533182 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:36.532864 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:46.533058 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:09:56.532903 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-28 19:10:06.532980 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
19:10:10 up 2:52, 0 users, load average: 1.08, 1.99, 2.53
Linux old-k8s-version-530457 5.15.0-1068-aws #74~20.04.1-Ubuntu SMP Tue Aug 6 19:45:17 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [917e18e8ed4ea8dc322d07b6a50300ed1ad4ca545a01e161f583aaf33b723f11] <==
I0828 19:08:03.870588 1 main.go:299] handling current node
I0828 19:08:13.873887 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:08:13.873926 1 main.go:299] handling current node
I0828 19:08:23.867652 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:08:23.867686 1 main.go:299] handling current node
I0828 19:08:33.874843 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:08:33.874878 1 main.go:299] handling current node
I0828 19:08:43.875588 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:08:43.875626 1 main.go:299] handling current node
I0828 19:08:53.874194 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:08:53.874234 1 main.go:299] handling current node
I0828 19:09:03.870864 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:03.870906 1 main.go:299] handling current node
I0828 19:09:13.875847 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:13.875899 1 main.go:299] handling current node
I0828 19:09:23.866953 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:23.866990 1 main.go:299] handling current node
I0828 19:09:33.873887 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:33.873923 1 main.go:299] handling current node
I0828 19:09:43.875767 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:43.875886 1 main.go:299] handling current node
I0828 19:09:53.874862 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:09:53.874980 1 main.go:299] handling current node
I0828 19:10:03.871384 1 main.go:295] Handling node with IPs: map[192.168.76.2:{}]
I0828 19:10:03.871420 1 main.go:299] handling current node
==> kube-apiserver [5c39e778b2c4147c38d84c75f0df8495da5f5306eab9e651d85d7013c5d7b319] <==
I0828 19:07:03.804908 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:07:03.804917 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0828 19:07:25.343062 1 handler_proxy.go:102] no RequestInfo found in the context
E0828 19:07:25.343141 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0828 19:07:25.343150 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0828 19:07:34.199779 1 client.go:360] parsed scheme: "passthrough"
I0828 19:07:34.199829 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:07:34.199838 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0828 19:08:04.454327 1 client.go:360] parsed scheme: "passthrough"
I0828 19:08:04.454373 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:08:04.454382 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0828 19:08:42.342570 1 client.go:360] parsed scheme: "passthrough"
I0828 19:08:42.342614 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:08:42.342623 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0828 19:09:20.752791 1 client.go:360] parsed scheme: "passthrough"
I0828 19:09:20.752833 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:09:20.752842 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0828 19:09:22.085221 1 handler_proxy.go:102] no RequestInfo found in the context
E0828 19:09:22.085419 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0828 19:09:22.085435 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0828 19:09:58.259842 1 client.go:360] parsed scheme: "passthrough"
I0828 19:09:58.259901 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0828 19:09:58.259910 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [e93b5ce24805aa41b12dc684a90729c61bd115c5efd804aa2e1a837df21182d2] <==
W0828 19:05:47.003107 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:06:14.726403 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:06:18.654442 1 request.go:655] Throttling request took 1.048312852s, request: GET:https://192.168.76.2:8443/apis/events.k8s.io/v1beta1?timeout=32s
W0828 19:06:19.505943 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:06:45.229462 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:06:51.156380 1 request.go:655] Throttling request took 1.048211066s, request: GET:https://192.168.76.2:8443/apis/authentication.k8s.io/v1?timeout=32s
W0828 19:06:52.015095 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:07:15.731289 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:07:23.666300 1 request.go:655] Throttling request took 1.048009516s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0828 19:07:24.517330 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:07:46.233986 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:07:56.167767 1 request.go:655] Throttling request took 1.04823307s, request: GET:https://192.168.76.2:8443/apis/events.k8s.io/v1?timeout=32s
W0828 19:07:57.019267 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:08:16.735693 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:08:28.669692 1 request.go:655] Throttling request took 1.046023769s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0828 19:08:29.521102 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:08:47.237466 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:09:01.171528 1 request.go:655] Throttling request took 1.042318477s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0828 19:09:02.023459 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:09:17.739300 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:09:33.673897 1 request.go:655] Throttling request took 1.047534348s, request: GET:https://192.168.76.2:8443/apis/networking.k8s.io/v1beta1?timeout=32s
W0828 19:09:34.525331 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0828 19:09:48.241114 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0828 19:10:06.175795 1 request.go:655] Throttling request took 1.04828415s, request: GET:https://192.168.76.2:8443/apis/admissionregistration.k8s.io/v1?timeout=32s
W0828 19:10:07.027351 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-proxy [ee22e16883b2ea885911b340c812a158b2d8a8b904215b10b9f099503cab5623] <==
I0828 19:02:11.971308 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0828 19:02:12.000731 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0828 19:02:12.238766 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0828 19:02:12.239237 1 server_others.go:185] Using iptables Proxier.
I0828 19:02:12.239519 1 server.go:650] Version: v1.20.0
I0828 19:02:12.240794 1 config.go:315] Starting service config controller
I0828 19:02:12.240858 1 shared_informer.go:240] Waiting for caches to sync for service config
I0828 19:02:12.240910 1 config.go:224] Starting endpoint slice config controller
I0828 19:02:12.240968 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0828 19:02:12.341129 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0828 19:02:12.341190 1 shared_informer.go:247] Caches are synced for service config
I0828 19:04:24.410026 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0828 19:04:24.410126 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0828 19:04:24.451652 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0828 19:04:24.451820 1 server_others.go:185] Using iptables Proxier.
I0828 19:04:24.452088 1 server.go:650] Version: v1.20.0
I0828 19:04:24.463833 1 config.go:315] Starting service config controller
I0828 19:04:24.463921 1 shared_informer.go:240] Waiting for caches to sync for service config
I0828 19:04:24.463988 1 config.go:224] Starting endpoint slice config controller
I0828 19:04:24.464027 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0828 19:04:24.564070 1 shared_informer.go:247] Caches are synced for service config
I0828 19:04:24.564176 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [f3c41ad57f8c9b59a9c49516c7842c079f372fd0a0523b146a90016a1eb459c0] <==
E0828 19:01:51.042872 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0828 19:01:51.058912 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0828 19:01:51.059404 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0828 19:01:51.059600 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0828 19:01:51.059880 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0828 19:01:51.060069 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0828 19:01:51.060292 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0828 19:01:51.060532 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0828 19:01:51.060806 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0828 19:01:51.869228 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0828 19:01:51.916369 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0828 19:01:51.921639 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0828 19:01:52.069710 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0828 19:01:52.071673 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0828 19:01:53.806686 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0828 19:04:14.075498 1 serving.go:331] Generated self-signed cert in-memory
W0828 19:04:20.828626 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0828 19:04:20.828764 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0828 19:04:20.828805 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0828 19:04:20.828847 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0828 19:04:21.094931 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0828 19:04:21.097382 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0828 19:04:21.097474 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0828 19:04:21.097814 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0828 19:04:21.302897 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: I0828 19:08:50.576152 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:08:50 old-k8s-version-530457 kubelet[732]: E0828 19:08:50.577123 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:08:56 old-k8s-version-530457 kubelet[732]: E0828 19:08:56.576060 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: I0828 19:09:04.575084 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:09:04 old-k8s-version-530457 kubelet[732]: E0828 19:09:04.575429 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:07 old-k8s-version-530457 kubelet[732]: E0828 19:09:07.576124 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:10 old-k8s-version-530457 kubelet[732]: E0828 19:09:10.520833 732 container_manager_linux.go:533] failed to find cgroups of kubelet - cpu and memory cgroup hierarchy not unified. cpu: /docker/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e, memory: /docker/4dbb4904f00c546c62950204b6a2eea75f992cc16257185f022f7b40308b508e/system.slice/kubelet.service
Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: I0828 19:09:17.575067 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:09:17 old-k8s-version-530457 kubelet[732]: E0828 19:09:17.575422 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:22 old-k8s-version-530457 kubelet[732]: E0828 19:09:22.575924 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: I0828 19:09:30.576181 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:09:30 old-k8s-version-530457 kubelet[732]: E0828 19:09:30.576492 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:33 old-k8s-version-530457 kubelet[732]: E0828 19:09:33.576250 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: I0828 19:09:41.575167 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:09:41 old-k8s-version-530457 kubelet[732]: E0828 19:09:41.575504 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:09:44 old-k8s-version-530457 kubelet[732]: E0828 19:09:44.576299 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: I0828 19:09:55.575176 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.576490 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 28 19:09:55 old-k8s-version-530457 kubelet[732]: E0828 19:09:55.577024 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:10:06 old-k8s-version-530457 kubelet[732]: I0828 19:10:06.575198 732 scope.go:95] [topologymanager] RemoveContainer - Container ID: d972bae8e872112c9d83dd641f0ff2d1301099575a045c0d81a0bba21c050f3e
Aug 28 19:10:06 old-k8s-version-530457 kubelet[732]: E0828 19:10:06.575574 732 pod_workers.go:191] Error syncing pod fa43accd-503c-4863-9c97-168f2bb218cf ("dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-k85jn_kubernetes-dashboard(fa43accd-503c-4863-9c97-168f2bb218cf)"
Aug 28 19:10:07 old-k8s-version-530457 kubelet[732]: E0828 19:10:07.588432 732 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = pinging container registry fake.domain: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain: no such host
Aug 28 19:10:07 old-k8s-version-530457 kubelet[732]: E0828 19:10:07.588503 732 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = pinging container registry fake.domain: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain: no such host
Aug 28 19:10:07 old-k8s-version-530457 kubelet[732]: E0828 19:10:07.589156 732 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-glpj8,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-vfdm2_kube-system(298c932
1-b9ba-4811-837c-2a53881ff451): ErrImagePull: rpc error: code = Unknown desc = pinging container registry fake.domain: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain: no such host
Aug 28 19:10:07 old-k8s-version-530457 kubelet[732]: E0828 19:10:07.589196 732 pod_workers.go:191] Error syncing pod 298c9321-b9ba-4811-837c-2a53881ff451 ("metrics-server-9975d5f86-vfdm2_kube-system(298c9321-b9ba-4811-837c-2a53881ff451)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
==> kubernetes-dashboard [6ade1a63523dd30b4378c10828bbfca3325f96b80c1b2270fe130f5186446b21] <==
2024/08/28 19:04:48 Using namespace: kubernetes-dashboard
2024/08/28 19:04:48 Using in-cluster config to connect to apiserver
2024/08/28 19:04:48 Using secret token for csrf signing
2024/08/28 19:04:48 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/08/28 19:04:48 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/08/28 19:04:48 Successful initial request to the apiserver, version: v1.20.0
2024/08/28 19:04:48 Generating JWE encryption key
2024/08/28 19:04:48 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/08/28 19:04:48 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/08/28 19:04:53 Initializing JWE encryption key from synchronized object
2024/08/28 19:04:53 Creating in-cluster Sidecar client
2024/08/28 19:04:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:04:53 Serving insecurely on HTTP port: 9090
2024/08/28 19:05:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:05:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:06:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:06:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:07:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:07:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:08:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:08:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:09:23 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:09:53 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/28 19:04:48 Starting overwatch
==> storage-provisioner [00f6d3ef896cb5c94b32b002838e20e9f8932eef759fb250721779ddd4dc4a30] <==
I0828 19:02:37.217683 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0828 19:02:37.258933 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0828 19:02:37.259188 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0828 19:02:37.282353 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0828 19:02:37.282599 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-530457_5473e768-8c4f-43c7-9009-69f57c03a5dd!
I0828 19:02:37.282665 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"a5354950-b9a8-4633-8fe7-79fc8b4e3d20", APIVersion:"v1", ResourceVersion:"489", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-530457_5473e768-8c4f-43c7-9009-69f57c03a5dd became leader
I0828 19:02:37.383109 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-530457_5473e768-8c4f-43c7-9009-69f57c03a5dd!
I0828 19:04:22.870782 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0828 19:04:52.881204 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [77caa541b8678631e669df9f00fd2cf105eba77630dc5635e27c58b083f812cb] <==
I0828 19:04:56.204276 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0828 19:04:56.237724 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0828 19:04:56.238524 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0828 19:05:13.716893 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0828 19:05:13.717500 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"a5354950-b9a8-4633-8fe7-79fc8b4e3d20", APIVersion:"v1", ResourceVersion:"802", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-530457_73beb236-7804-4616-8eb7-2023ee61e701 became leader
I0828 19:05:13.717689 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-530457_73beb236-7804-4616-8eb7-2023ee61e701!
I0828 19:05:13.818562 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-530457_73beb236-7804-4616-8eb7-2023ee61e701!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-530457 -n old-k8s-version-530457
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-530457 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-vfdm2
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-530457 describe pod metrics-server-9975d5f86-vfdm2
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-530457 describe pod metrics-server-9975d5f86-vfdm2: exit status 1 (108.096297ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-vfdm2" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-530457 describe pod metrics-server-9975d5f86-vfdm2: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (379.09s)