=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-137556 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0
E0528 22:24:22.540685 1355197 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/functional-486060/client.crt: no such file or directory
E0528 22:24:28.138509 1355197 cert_rotation.go:168] key failed with : open /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/addons-504712/client.crt: no such file or directory
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-137556 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0: exit status 102 (6m9.166653624s)
-- stdout --
* [old-k8s-version-137556] minikube v1.33.1 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18966
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18966-1349783/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18966-1349783/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.30.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-137556" primary control-plane node in "old-k8s-version-137556" cluster
* Pulling base image v0.0.44-1716228441-18934 ...
* Restarting existing docker container for "old-k8s-version-137556" ...
* Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
* Verifying Kubernetes components...
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-137556 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
-- /stdout --
** stderr **
I0528 22:23:34.595516 1540905 out.go:291] Setting OutFile to fd 1 ...
I0528 22:23:34.595712 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:23:34.595739 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:23:34.595758 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:23:34.596008 1540905 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18966-1349783/.minikube/bin
I0528 22:23:34.596382 1540905 out.go:298] Setting JSON to false
I0528 22:23:34.597304 1540905 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":21963,"bootTime":1716913052,"procs":177,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0528 22:23:34.597393 1540905 start.go:139] virtualization:
I0528 22:23:34.599792 1540905 out.go:177] * [old-k8s-version-137556] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0528 22:23:34.602075 1540905 out.go:177] - MINIKUBE_LOCATION=18966
I0528 22:23:34.603686 1540905 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0528 22:23:34.602152 1540905 notify.go:220] Checking for updates...
I0528 22:23:34.607038 1540905 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:23:34.609134 1540905 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18966-1349783/.minikube
I0528 22:23:34.610645 1540905 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0528 22:23:34.612326 1540905 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0528 22:23:34.614444 1540905 config.go:182] Loaded profile config "old-k8s-version-137556": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0528 22:23:34.616704 1540905 out.go:177] * Kubernetes 1.30.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.1
I0528 22:23:34.618596 1540905 driver.go:392] Setting default libvirt URI to qemu:///system
I0528 22:23:34.640092 1540905 docker.go:122] docker version: linux-26.1.3:Docker Engine - Community
I0528 22:23:34.640208 1540905 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0528 22:23:34.728024 1540905 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:59 SystemTime:2024-05-28 22:23:34.718978421 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1062-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214974464 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89 Expected:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0528 22:23:34.728140 1540905 docker.go:295] overlay module found
I0528 22:23:34.731109 1540905 out.go:177] * Using the docker driver based on existing profile
I0528 22:23:34.733406 1540905 start.go:297] selected driver: docker
I0528 22:23:34.733427 1540905 start.go:901] validating driver "docker" against &{Name:old-k8s-version-137556 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-137556 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountStr
ing:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:23:34.733534 1540905 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0528 22:23:34.734221 1540905 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0528 22:23:34.828328 1540905 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:59 SystemTime:2024-05-28 22:23:34.816395389 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1062-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214974464 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89 Expected:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0528 22:23:34.828784 1540905 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0528 22:23:34.828818 1540905 cni.go:84] Creating CNI manager for ""
I0528 22:23:34.828831 1540905 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0528 22:23:34.828875 1540905 start.go:340] cluster config:
{Name:old-k8s-version-137556 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-137556 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP
: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:23:34.831613 1540905 out.go:177] * Starting "old-k8s-version-137556" primary control-plane node in "old-k8s-version-137556" cluster
I0528 22:23:34.833699 1540905 cache.go:121] Beginning downloading kic base image for docker with crio
I0528 22:23:34.836731 1540905 out.go:177] * Pulling base image v0.0.44-1716228441-18934 ...
I0528 22:23:34.838934 1540905 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0528 22:23:34.838988 1540905 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4
I0528 22:23:34.838996 1540905 cache.go:56] Caching tarball of preloaded images
I0528 22:23:34.839095 1540905 preload.go:173] Found /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-cri-o-overlay-arm64.tar.lz4 in cache, skipping download
I0528 22:23:34.839105 1540905 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on crio
I0528 22:23:34.839228 1540905 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/config.json ...
I0528 22:23:34.839463 1540905 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 in local docker daemon
I0528 22:23:34.855004 1540905 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 in local docker daemon, skipping pull
I0528 22:23:34.855026 1540905 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 exists in daemon, skipping load
I0528 22:23:34.855055 1540905 cache.go:194] Successfully downloaded all kic artifacts
I0528 22:23:34.855084 1540905 start.go:360] acquireMachinesLock for old-k8s-version-137556: {Name:mk2c00f24a3d04faf9982ed0805071e70a489b60 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:23:34.855144 1540905 start.go:364] duration metric: took 39.548µs to acquireMachinesLock for "old-k8s-version-137556"
I0528 22:23:34.855164 1540905 start.go:96] Skipping create...Using existing machine configuration
I0528 22:23:34.855177 1540905 fix.go:54] fixHost starting:
I0528 22:23:34.855660 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:34.879573 1540905 fix.go:112] recreateIfNeeded on old-k8s-version-137556: state=Stopped err=<nil>
W0528 22:23:34.879601 1540905 fix.go:138] unexpected machine state, will restart: <nil>
I0528 22:23:34.882748 1540905 out.go:177] * Restarting existing docker container for "old-k8s-version-137556" ...
I0528 22:23:34.884948 1540905 cli_runner.go:164] Run: docker start old-k8s-version-137556
I0528 22:23:35.244793 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:35.261420 1540905 kic.go:430] container "old-k8s-version-137556" state is running.
I0528 22:23:35.261816 1540905 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-137556
I0528 22:23:35.287358 1540905 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/config.json ...
I0528 22:23:35.287590 1540905 machine.go:94] provisionDockerMachine start ...
I0528 22:23:35.287656 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:35.312878 1540905 main.go:141] libmachine: Using SSH client type: native
I0528 22:23:35.313144 1540905 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34589 <nil> <nil>}
I0528 22:23:35.313159 1540905 main.go:141] libmachine: About to run SSH command:
hostname
I0528 22:23:35.313859 1540905 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0528 22:23:38.470318 1540905 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-137556
I0528 22:23:38.470344 1540905 ubuntu.go:169] provisioning hostname "old-k8s-version-137556"
I0528 22:23:38.470446 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:38.546503 1540905 main.go:141] libmachine: Using SSH client type: native
I0528 22:23:38.547469 1540905 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34589 <nil> <nil>}
I0528 22:23:38.547488 1540905 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-137556 && echo "old-k8s-version-137556" | sudo tee /etc/hostname
I0528 22:23:38.736393 1540905 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-137556
I0528 22:23:38.736480 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:38.775756 1540905 main.go:141] libmachine: Using SSH client type: native
I0528 22:23:38.776005 1540905 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34589 <nil> <nil>}
I0528 22:23:38.776028 1540905 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-137556' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-137556/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-137556' | sudo tee -a /etc/hosts;
fi
fi
I0528 22:23:38.971016 1540905 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0528 22:23:38.971052 1540905 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18966-1349783/.minikube CaCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18966-1349783/.minikube}
I0528 22:23:38.971085 1540905 ubuntu.go:177] setting up certificates
I0528 22:23:38.971097 1540905 provision.go:84] configureAuth start
I0528 22:23:38.971194 1540905 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-137556
I0528 22:23:39.011757 1540905 provision.go:143] copyHostCerts
I0528 22:23:39.011834 1540905 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem, removing ...
I0528 22:23:39.011847 1540905 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem
I0528 22:23:39.011938 1540905 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem (1082 bytes)
I0528 22:23:39.012035 1540905 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem, removing ...
I0528 22:23:39.012047 1540905 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem
I0528 22:23:39.012080 1540905 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem (1123 bytes)
I0528 22:23:39.012186 1540905 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem, removing ...
I0528 22:23:39.012196 1540905 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem
I0528 22:23:39.012226 1540905 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem (1675 bytes)
I0528 22:23:39.012277 1540905 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-137556 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-137556]
I0528 22:23:39.351123 1540905 provision.go:177] copyRemoteCerts
I0528 22:23:39.351200 1540905 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0528 22:23:39.351251 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:39.372776 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:39.465452 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0528 22:23:39.492905 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0528 22:23:39.520784 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0528 22:23:39.547728 1540905 provision.go:87] duration metric: took 576.616647ms to configureAuth
I0528 22:23:39.547757 1540905 ubuntu.go:193] setting minikube options for container-runtime
I0528 22:23:39.547946 1540905 config.go:182] Loaded profile config "old-k8s-version-137556": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0528 22:23:39.548051 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:39.566340 1540905 main.go:141] libmachine: Using SSH client type: native
I0528 22:23:39.566597 1540905 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34589 <nil> <nil>}
I0528 22:23:39.566619 1540905 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %s "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0528 22:23:40.023948 1540905 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0528 22:23:40.023978 1540905 machine.go:97] duration metric: took 4.736370738s to provisionDockerMachine
I0528 22:23:40.023990 1540905 start.go:293] postStartSetup for "old-k8s-version-137556" (driver="docker")
I0528 22:23:40.024002 1540905 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0528 22:23:40.024069 1540905 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0528 22:23:40.024115 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:40.061084 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:40.171999 1540905 ssh_runner.go:195] Run: cat /etc/os-release
I0528 22:23:40.177519 1540905 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0528 22:23:40.177558 1540905 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0528 22:23:40.177570 1540905 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0528 22:23:40.177577 1540905 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0528 22:23:40.177590 1540905 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-1349783/.minikube/addons for local assets ...
I0528 22:23:40.177649 1540905 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-1349783/.minikube/files for local assets ...
I0528 22:23:40.177734 1540905 filesync.go:149] local asset: /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem -> 13551972.pem in /etc/ssl/certs
I0528 22:23:40.177881 1540905 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0528 22:23:40.189319 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem --> /etc/ssl/certs/13551972.pem (1708 bytes)
I0528 22:23:40.227585 1540905 start.go:296] duration metric: took 203.580636ms for postStartSetup
I0528 22:23:40.227669 1540905 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0528 22:23:40.227715 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:40.251134 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:40.339350 1540905 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0528 22:23:40.343854 1540905 fix.go:56] duration metric: took 5.488676702s for fixHost
I0528 22:23:40.343882 1540905 start.go:83] releasing machines lock for "old-k8s-version-137556", held for 5.48872704s
I0528 22:23:40.343949 1540905 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-137556
I0528 22:23:40.366265 1540905 ssh_runner.go:195] Run: cat /version.json
I0528 22:23:40.366324 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:40.366568 1540905 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0528 22:23:40.366631 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:40.400045 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:40.400647 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:40.621301 1540905 ssh_runner.go:195] Run: systemctl --version
I0528 22:23:40.626636 1540905 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0528 22:23:40.781367 1540905 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0528 22:23:40.785926 1540905 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0528 22:23:40.795264 1540905 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0528 22:23:40.795341 1540905 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0528 22:23:40.805010 1540905 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0528 22:23:40.805038 1540905 start.go:494] detecting cgroup driver to use...
I0528 22:23:40.805070 1540905 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0528 22:23:40.805118 1540905 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0528 22:23:40.817168 1540905 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0528 22:23:40.829811 1540905 docker.go:217] disabling cri-docker service (if available) ...
I0528 22:23:40.829894 1540905 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0528 22:23:40.844809 1540905 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0528 22:23:40.856838 1540905 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0528 22:23:40.970031 1540905 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0528 22:23:41.075165 1540905 docker.go:233] disabling docker service ...
I0528 22:23:41.075243 1540905 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0528 22:23:41.089723 1540905 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0528 22:23:41.102619 1540905 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0528 22:23:41.216889 1540905 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0528 22:23:41.340423 1540905 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0528 22:23:41.353416 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0528 22:23:41.370946 1540905 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.2" pause image...
I0528 22:23:41.371068 1540905 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.2"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:23:41.381589 1540905 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0528 22:23:41.381725 1540905 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:23:41.393372 1540905 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:23:41.404550 1540905 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:23:41.415180 1540905 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0528 22:23:41.424850 1540905 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0528 22:23:41.434545 1540905 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0528 22:23:41.443966 1540905 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:23:41.553700 1540905 ssh_runner.go:195] Run: sudo systemctl restart crio
I0528 22:23:42.248763 1540905 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0528 22:23:42.248944 1540905 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0528 22:23:42.253122 1540905 start.go:562] Will wait 60s for crictl version
I0528 22:23:42.253234 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:23:42.256948 1540905 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0528 22:23:42.309543 1540905 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0528 22:23:42.309661 1540905 ssh_runner.go:195] Run: crio --version
I0528 22:23:42.359879 1540905 ssh_runner.go:195] Run: crio --version
I0528 22:23:42.419569 1540905 out.go:177] * Preparing Kubernetes v1.20.0 on CRI-O 1.24.6 ...
I0528 22:23:42.421573 1540905 cli_runner.go:164] Run: docker network inspect old-k8s-version-137556 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0528 22:23:42.442451 1540905 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0528 22:23:42.447462 1540905 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 22:23:42.458273 1540905 kubeadm.go:877] updating cluster {Name:old-k8s-version-137556 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-137556 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0528 22:23:42.458436 1540905 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime crio
I0528 22:23:42.458518 1540905 ssh_runner.go:195] Run: sudo crictl images --output json
I0528 22:23:42.513391 1540905 crio.go:514] all images are preloaded for cri-o runtime.
I0528 22:23:42.513415 1540905 crio.go:433] Images already preloaded, skipping extraction
I0528 22:23:42.513470 1540905 ssh_runner.go:195] Run: sudo crictl images --output json
I0528 22:23:42.554094 1540905 crio.go:514] all images are preloaded for cri-o runtime.
I0528 22:23:42.554118 1540905 cache_images.go:84] Images are preloaded, skipping loading
I0528 22:23:42.554126 1540905 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.20.0 crio true true} ...
I0528 22:23:42.554240 1540905 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --enforce-node-allocatable= --hostname-override=old-k8s-version-137556 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-137556 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0528 22:23:42.554324 1540905 ssh_runner.go:195] Run: crio config
I0528 22:23:42.620481 1540905 cni.go:84] Creating CNI manager for ""
I0528 22:23:42.620502 1540905 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0528 22:23:42.620518 1540905 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0528 22:23:42.620545 1540905 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-137556 NodeName:old-k8s-version-137556 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0528 22:23:42.620713 1540905 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/crio/crio.sock
name: "old-k8s-version-137556"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0528 22:23:42.620816 1540905 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0528 22:23:42.629939 1540905 binaries.go:44] Found k8s binaries, skipping transfer
I0528 22:23:42.630033 1540905 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0528 22:23:42.639522 1540905 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (480 bytes)
I0528 22:23:42.658193 1540905 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0528 22:23:42.676618 1540905 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2117 bytes)
I0528 22:23:42.694571 1540905 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0528 22:23:42.698350 1540905 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 22:23:42.708655 1540905 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:23:42.803236 1540905 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0528 22:23:42.817937 1540905 certs.go:68] Setting up /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556 for IP: 192.168.85.2
I0528 22:23:42.818007 1540905 certs.go:194] generating shared ca certs ...
I0528 22:23:42.818059 1540905 certs.go:226] acquiring lock for ca certs: {Name:mk3b01431a293453662fa80a6161920f23c6c736 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:23:42.818252 1540905 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.key
I0528 22:23:42.818335 1540905 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.key
I0528 22:23:42.818372 1540905 certs.go:256] generating profile certs ...
I0528 22:23:42.818500 1540905 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/client.key
I0528 22:23:42.818623 1540905 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/apiserver.key.984daade
I0528 22:23:42.818709 1540905 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/proxy-client.key
I0528 22:23:42.818882 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197.pem (1338 bytes)
W0528 22:23:42.818957 1540905 certs.go:480] ignoring /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197_empty.pem, impossibly tiny 0 bytes
I0528 22:23:42.818995 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem (1679 bytes)
I0528 22:23:42.819049 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem (1082 bytes)
I0528 22:23:42.819106 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem (1123 bytes)
I0528 22:23:42.819165 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem (1675 bytes)
I0528 22:23:42.819257 1540905 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem (1708 bytes)
I0528 22:23:42.820102 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0528 22:23:42.844947 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0528 22:23:42.870136 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0528 22:23:42.916516 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0528 22:23:42.958812 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0528 22:23:42.982057 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0528 22:23:43.007463 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0528 22:23:43.044148 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/old-k8s-version-137556/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0528 22:23:43.089564 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197.pem --> /usr/share/ca-certificates/1355197.pem (1338 bytes)
I0528 22:23:43.117543 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem --> /usr/share/ca-certificates/13551972.pem (1708 bytes)
I0528 22:23:43.145287 1540905 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0528 22:23:43.172661 1540905 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0528 22:23:43.193499 1540905 ssh_runner.go:195] Run: openssl version
I0528 22:23:43.199335 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/13551972.pem && ln -fs /usr/share/ca-certificates/13551972.pem /etc/ssl/certs/13551972.pem"
I0528 22:23:43.209421 1540905 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/13551972.pem
I0528 22:23:43.212925 1540905 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 May 28 21:42 /usr/share/ca-certificates/13551972.pem
I0528 22:23:43.213038 1540905 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/13551972.pem
I0528 22:23:43.220098 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/13551972.pem /etc/ssl/certs/3ec20f2e.0"
I0528 22:23:43.230471 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0528 22:23:43.241087 1540905 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0528 22:23:43.245194 1540905 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 May 28 21:31 /usr/share/ca-certificates/minikubeCA.pem
I0528 22:23:43.245254 1540905 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0528 22:23:43.253058 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0528 22:23:43.263559 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1355197.pem && ln -fs /usr/share/ca-certificates/1355197.pem /etc/ssl/certs/1355197.pem"
I0528 22:23:43.274177 1540905 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1355197.pem
I0528 22:23:43.278304 1540905 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 May 28 21:42 /usr/share/ca-certificates/1355197.pem
I0528 22:23:43.278363 1540905 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1355197.pem
I0528 22:23:43.285731 1540905 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1355197.pem /etc/ssl/certs/51391683.0"
I0528 22:23:43.295743 1540905 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0528 22:23:43.299816 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0528 22:23:43.307205 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0528 22:23:43.314632 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0528 22:23:43.321940 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0528 22:23:43.329236 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0528 22:23:43.336412 1540905 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0528 22:23:43.343612 1540905 kubeadm.go:391] StartCluster: {Name:old-k8s-version-137556 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-137556 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/m
inikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:23:43.343709 1540905 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0528 22:23:43.343765 1540905 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0528 22:23:43.401070 1540905 cri.go:89] found id: ""
I0528 22:23:43.401166 1540905 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0528 22:23:43.412801 1540905 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0528 22:23:43.412823 1540905 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0528 22:23:43.412832 1540905 kubeadm.go:587] restartPrimaryControlPlane start ...
I0528 22:23:43.412889 1540905 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0528 22:23:43.423703 1540905 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0528 22:23:43.424164 1540905 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-137556" does not appear in /home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:23:43.424293 1540905 kubeconfig.go:62] /home/jenkins/minikube-integration/18966-1349783/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-137556" cluster setting kubeconfig missing "old-k8s-version-137556" context setting]
I0528 22:23:43.424626 1540905 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-1349783/kubeconfig: {Name:mkaf5e1534f034576a412c2bb12acf3530c82fbc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:23:43.426437 1540905 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0528 22:23:43.437846 1540905 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.85.2
I0528 22:23:43.437880 1540905 kubeadm.go:591] duration metric: took 25.041806ms to restartPrimaryControlPlane
I0528 22:23:43.437894 1540905 kubeadm.go:393] duration metric: took 94.292091ms to StartCluster
I0528 22:23:43.437909 1540905 settings.go:142] acquiring lock: {Name:mk3ead4661b05edfaa64061283a93c6a76969cd7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:23:43.437977 1540905 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:23:43.438650 1540905 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-1349783/kubeconfig: {Name:mkaf5e1534f034576a412c2bb12acf3530c82fbc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:23:43.438874 1540905 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:crio ControlPlane:true Worker:true}
I0528 22:23:43.442369 1540905 out.go:177] * Verifying Kubernetes components...
I0528 22:23:43.439292 1540905 config.go:182] Loaded profile config "old-k8s-version-137556": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.20.0
I0528 22:23:43.439251 1540905 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0528 22:23:43.443890 1540905 addons.go:69] Setting dashboard=true in profile "old-k8s-version-137556"
I0528 22:23:43.443918 1540905 addons.go:234] Setting addon dashboard=true in "old-k8s-version-137556"
W0528 22:23:43.443925 1540905 addons.go:243] addon dashboard should already be in state true
I0528 22:23:43.443954 1540905 host.go:66] Checking if "old-k8s-version-137556" exists ...
I0528 22:23:43.444398 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:43.444559 1540905 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-137556"
I0528 22:23:43.444599 1540905 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-137556"
W0528 22:23:43.444606 1540905 addons.go:243] addon storage-provisioner should already be in state true
I0528 22:23:43.444627 1540905 host.go:66] Checking if "old-k8s-version-137556" exists ...
I0528 22:23:43.445001 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:43.449278 1540905 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-137556"
I0528 22:23:43.449314 1540905 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-137556"
I0528 22:23:43.449369 1540905 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:23:43.449629 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:43.450153 1540905 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-137556"
I0528 22:23:43.450179 1540905 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-137556"
W0528 22:23:43.450186 1540905 addons.go:243] addon metrics-server should already be in state true
I0528 22:23:43.450212 1540905 host.go:66] Checking if "old-k8s-version-137556" exists ...
I0528 22:23:43.450707 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:43.496250 1540905 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0528 22:23:43.503523 1540905 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0528 22:23:43.503554 1540905 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0528 22:23:43.505410 1540905 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0528 22:23:43.503820 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:43.518087 1540905 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0528 22:23:43.510279 1540905 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:23:43.521838 1540905 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0528 22:23:43.519921 1540905 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0528 22:23:43.521048 1540905 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-137556"
W0528 22:23:43.523459 1540905 addons.go:243] addon default-storageclass should already be in state true
I0528 22:23:43.523463 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0528 22:23:43.523488 1540905 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0528 22:23:43.523492 1540905 host.go:66] Checking if "old-k8s-version-137556" exists ...
I0528 22:23:43.523558 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:43.523922 1540905 cli_runner.go:164] Run: docker container inspect old-k8s-version-137556 --format={{.State.Status}}
I0528 22:23:43.524175 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:43.547202 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:43.579720 1540905 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0528 22:23:43.579740 1540905 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0528 22:23:43.579801 1540905 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-137556
I0528 22:23:43.622180 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:43.622208 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:43.631899 1540905 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34589 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/old-k8s-version-137556/id_rsa Username:docker}
I0528 22:23:43.751825 1540905 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0528 22:23:43.805452 1540905 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-137556" to be "Ready" ...
I0528 22:23:43.814743 1540905 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0528 22:23:43.814763 1540905 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0528 22:23:43.822565 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0528 22:23:43.822589 1540905 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0528 22:23:43.868748 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0528 22:23:43.883690 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:23:43.912264 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0528 22:23:43.912338 1540905 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0528 22:23:43.913200 1540905 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0528 22:23:43.913239 1540905 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0528 22:23:44.040611 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0528 22:23:44.040687 1540905 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0528 22:23:44.046482 1540905 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:44.046561 1540905 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0528 22:23:44.141465 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:44.195449 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0528 22:23:44.195518 1540905 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0528 22:23:44.210842 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.210958 1540905 retry.go:31] will retry after 180.35272ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:44.272549 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.272629 1540905 retry.go:31] will retry after 288.281649ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.284484 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0528 22:23:44.284557 1540905 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0528 22:23:44.313612 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0528 22:23:44.313640 1540905 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0528 22:23:44.370304 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.370339 1540905 retry.go:31] will retry after 129.224302ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.371251 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0528 22:23:44.371273 1540905 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0528 22:23:44.391442 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0528 22:23:44.404483 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0528 22:23:44.404507 1540905 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0528 22:23:44.472373 1540905 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:23:44.472422 1540905 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0528 22:23:44.500343 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:44.553025 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:23:44.561699 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:44.609423 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.609458 1540905 retry.go:31] will retry after 261.069566ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:44.824797 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:44.824917 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.824947 1540905 retry.go:31] will retry after 310.857777ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.824832 1540905 retry.go:31] will retry after 468.850958ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:44.825259 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.825278 1540905 retry.go:31] will retry after 503.54013ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.871599 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:44.963356 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:44.963387 1540905 retry.go:31] will retry after 547.819511ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.136865 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0528 22:23:45.244461 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.244547 1540905 retry.go:31] will retry after 562.457992ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.294859 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:45.329725 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:45.428528 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.428562 1540905 retry.go:31] will retry after 516.034734ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:45.491399 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.491433 1540905 retry.go:31] will retry after 752.230555ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.511681 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:45.617335 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.617363 1540905 retry.go:31] will retry after 1.038992506s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.807043 1540905 node_ready.go:53] error getting node "old-k8s-version-137556": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-137556": dial tcp 192.168.85.2:8443: connect: connection refused
I0528 22:23:45.807139 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0528 22:23:45.891076 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.891110 1540905 retry.go:31] will retry after 576.186797ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:45.945282 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0528 22:23:46.041053 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.041100 1540905 retry.go:31] will retry after 837.107834ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.244576 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:46.348847 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.348876 1540905 retry.go:31] will retry after 953.190511ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.468136 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0528 22:23:46.572021 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.572153 1540905 retry.go:31] will retry after 479.607386ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.657216 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:46.770835 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.770967 1540905 retry.go:31] will retry after 880.017504ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:46.878744 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0528 22:23:47.005127 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.005209 1540905 retry.go:31] will retry after 1.831529389s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.052400 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0528 22:23:47.149143 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.149253 1540905 retry.go:31] will retry after 1.449851435s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.302538 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:47.399664 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.399740 1540905 retry.go:31] will retry after 1.521086778s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.652219 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:47.738570 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:47.738605 1540905 retry.go:31] will retry after 1.932102836s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:48.306097 1540905 node_ready.go:53] error getting node "old-k8s-version-137556": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-137556": dial tcp 192.168.85.2:8443: connect: connection refused
I0528 22:23:48.599722 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0528 22:23:48.735285 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:48.735323 1540905 retry.go:31] will retry after 2.60272942s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:48.837257 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:48.921649 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:48.945551 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:48.945585 1540905 retry.go:31] will retry after 2.839374882s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:49.025761 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:49.025796 1540905 retry.go:31] will retry after 1.914360905s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:49.671259 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:49.769523 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:49.769551 1540905 retry.go:31] will retry after 1.689950637s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:50.806101 1540905 node_ready.go:53] error getting node "old-k8s-version-137556": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-137556": dial tcp 192.168.85.2:8443: connect: connection refused
I0528 22:23:50.940369 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0528 22:23:51.054899 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.054956 1540905 retry.go:31] will retry after 3.003281468s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.338895 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:23:51.460517 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0528 22:23:51.641696 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.641738 1540905 retry.go:31] will retry after 3.176506053s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0528 22:23:51.714974 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.715011 1540905 retry.go:31] will retry after 6.391394852s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.785328 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0528 22:23:51.913885 1540905 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:51.913994 1540905 retry.go:31] will retry after 1.481274555s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0528 22:23:53.396468 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:23:54.059041 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:23:54.818820 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:23:58.106606 1540905 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0528 22:24:03.189264 1540905 node_ready.go:49] node "old-k8s-version-137556" has status "Ready":"True"
I0528 22:24:03.189288 1540905 node_ready.go:38] duration metric: took 19.383760468s for node "old-k8s-version-137556" to be "Ready" ...
I0528 22:24:03.189301 1540905 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0528 22:24:03.568080 1540905 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-rkctv" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.156013 1540905 pod_ready.go:92] pod "coredns-74ff55c5b-rkctv" in "kube-system" namespace has status "Ready":"True"
I0528 22:24:04.156036 1540905 pod_ready.go:81] duration metric: took 587.87846ms for pod "coredns-74ff55c5b-rkctv" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.156048 1540905 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.369111 1540905 pod_ready.go:92] pod "etcd-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:24:04.369191 1540905 pod_ready.go:81] duration metric: took 213.133487ms for pod "etcd-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.369219 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.439403 1540905 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:24:04.439473 1540905 pod_ready.go:81] duration metric: took 70.233627ms for pod "kube-apiserver-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.439498 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:24:04.763459 1540905 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (10.704382353s)
I0528 22:24:04.763596 1540905 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (11.367094061s)
I0528 22:24:04.763646 1540905 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-137556"
I0528 22:24:05.381839 1540905 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (10.562975904s)
I0528 22:24:05.383931 1540905 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-137556 addons enable metrics-server
I0528 22:24:05.381996 1540905 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (7.275365367s)
I0528 22:24:05.409528 1540905 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0528 22:24:05.411495 1540905 addons.go:510] duration metric: took 21.972239619s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0528 22:24:06.451849 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:08.945929 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:10.949092 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:13.446094 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:15.946734 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:18.445761 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:20.446869 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:22.952513 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:25.447217 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:27.476754 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:29.955367 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:32.495602 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:34.949753 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:36.953814 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:39.445565 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:41.446225 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:43.945951 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:45.946277 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:47.947364 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:49.950331 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:52.445956 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:54.945052 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:56.946014 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:24:58.946614 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:01.446499 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:03.946841 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:05.947874 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:08.447084 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:10.946031 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:11.951732 1540905 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:11.951759 1540905 pod_ready.go:81] duration metric: took 1m7.512242141s for pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.951771 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-8jz6w" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.974855 1540905 pod_ready.go:92] pod "kube-proxy-8jz6w" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:11.974883 1540905 pod_ready.go:81] duration metric: took 23.104329ms for pod "kube-proxy-8jz6w" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.974920 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:13.982194 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:16.480792 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:18.482060 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:20.482153 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:20.982171 1540905 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:20.982200 1540905 pod_ready.go:81] duration metric: took 9.007259537s for pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:20.982213 1540905 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace to be "Ready" ...
I0528 22:25:22.989102 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:24.989608 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:27.489518 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:29.988533 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:31.991247 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:34.488194 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:36.495271 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:38.990348 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:41.495366 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:43.521873 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:45.988876 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:47.989265 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:50.488844 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:52.488950 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:54.492792 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:56.989265 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:59.488620 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:01.488762 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:03.988913 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:06.488936 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:08.988177 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:10.988907 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:12.989263 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:15.489223 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:17.988232 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:19.988565 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:22.488224 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:24.489557 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:26.988463 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:28.990525 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:31.487940 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:33.488677 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:35.988481 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:37.988691 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:39.989047 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:42.497971 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:44.988931 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:46.988984 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:49.488471 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:51.988480 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:54.487836 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:56.488805 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:58.988626 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:00.988931 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:02.989274 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:05.487582 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:07.487769 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:09.488409 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:11.987932 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:13.993302 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:16.488026 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:18.988722 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:20.988794 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:23.488388 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:25.489230 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:27.988524 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:29.989139 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:31.989250 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:34.488910 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:36.988638 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:39.488046 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:41.488536 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:43.488804 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:45.988591 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:47.989534 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:50.488161 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:52.989006 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:55.015860 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:57.488310 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:59.488865 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:01.493382 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:03.988248 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:05.989110 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:08.492906 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:10.988521 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:12.989371 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:15.488132 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:17.988756 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:20.488836 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:22.988169 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:25.492744 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:27.987967 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:29.988864 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:31.989186 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:34.488439 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:36.488805 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:38.988909 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:41.488703 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:43.988369 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:45.988453 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:47.989135 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:49.989359 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:52.487694 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:54.488326 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:56.489000 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:58.989339 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:01.488277 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:03.988703 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:05.989324 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:07.990979 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:10.489138 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:12.989242 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:15.488807 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:17.989551 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:20.489092 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:20.988019 1540905 pod_ready.go:81] duration metric: took 4m0.005793638s for pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace to be "Ready" ...
E0528 22:29:20.988044 1540905 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0528 22:29:20.988093 1540905 pod_ready.go:38] duration metric: took 5m17.798780687s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0528 22:29:20.988114 1540905 api_server.go:52] waiting for apiserver process to appear ...
I0528 22:29:20.988153 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:20.988223 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:21.032451 1540905 cri.go:89] found id: "9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:21.032474 1540905 cri.go:89] found id: ""
I0528 22:29:21.032489 1540905 logs.go:276] 1 containers: [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3]
I0528 22:29:21.032561 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.036290 1540905 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:21.036360 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:21.078257 1540905 cri.go:89] found id: "c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:21.078279 1540905 cri.go:89] found id: ""
I0528 22:29:21.078288 1540905 logs.go:276] 1 containers: [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8]
I0528 22:29:21.078346 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.081979 1540905 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:21.082082 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:21.128720 1540905 cri.go:89] found id: "cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:21.128744 1540905 cri.go:89] found id: ""
I0528 22:29:21.128752 1540905 logs.go:276] 1 containers: [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a]
I0528 22:29:21.128828 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.132418 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:21.132487 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:21.182402 1540905 cri.go:89] found id: "b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:21.182464 1540905 cri.go:89] found id: ""
I0528 22:29:21.182485 1540905 logs.go:276] 1 containers: [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105]
I0528 22:29:21.182568 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.186351 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:21.186428 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:21.236128 1540905 cri.go:89] found id: "7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:21.236152 1540905 cri.go:89] found id: ""
I0528 22:29:21.236160 1540905 logs.go:276] 1 containers: [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b]
I0528 22:29:21.236250 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.239897 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:21.239973 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:21.276341 1540905 cri.go:89] found id: "41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:21.276373 1540905 cri.go:89] found id: ""
I0528 22:29:21.276382 1540905 logs.go:276] 1 containers: [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29]
I0528 22:29:21.276440 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.280074 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:21.280157 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:21.329000 1540905 cri.go:89] found id: "dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:21.329022 1540905 cri.go:89] found id: ""
I0528 22:29:21.329030 1540905 logs.go:276] 1 containers: [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5]
I0528 22:29:21.329099 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.333459 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:21.333533 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:21.376968 1540905 cri.go:89] found id: "bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:21.377039 1540905 cri.go:89] found id: ""
I0528 22:29:21.377060 1540905 logs.go:276] 1 containers: [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527]
I0528 22:29:21.377147 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.381185 1540905 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:21.381306 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:21.421871 1540905 cri.go:89] found id: "4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:21.421943 1540905 cri.go:89] found id: "05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:21.421962 1540905 cri.go:89] found id: ""
I0528 22:29:21.421984 1540905 logs.go:276] 2 containers: [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998]
I0528 22:29:21.422120 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.425791 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.429266 1540905 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:21.429294 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:21.449313 1540905 logs.go:123] Gathering logs for kube-apiserver [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3] ...
I0528 22:29:21.449351 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:21.519486 1540905 logs.go:123] Gathering logs for etcd [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8] ...
I0528 22:29:21.519521 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:21.569520 1540905 logs.go:123] Gathering logs for coredns [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a] ...
I0528 22:29:21.569549 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:21.613509 1540905 logs.go:123] Gathering logs for kube-controller-manager [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29] ...
I0528 22:29:21.613536 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:21.686097 1540905 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:21.686137 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:21.831954 1540905 logs.go:123] Gathering logs for storage-provisioner [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33] ...
I0528 22:29:21.831985 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:21.881237 1540905 logs.go:123] Gathering logs for kube-scheduler [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105] ...
I0528 22:29:21.881266 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:21.927283 1540905 logs.go:123] Gathering logs for kindnet [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5] ...
I0528 22:29:21.927310 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:21.967506 1540905 logs.go:123] Gathering logs for storage-provisioner [05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998] ...
I0528 22:29:21.967537 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:22.013255 1540905 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:22.013284 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:22.069448 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.186601 738 reflector.go:138] object-"kube-system"/"kindnet-token-ztnvd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ztnvd" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.069707 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187153 738 reflector.go:138] object-"kube-system"/"kube-proxy-token-cs2pc": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-cs2pc" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.069920 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187714 738 reflector.go:138] object-"default"/"default-token-k8l4j": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-k8l4j" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070157 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.194216 738 reflector.go:138] object-"kube-system"/"storage-provisioner-token-cswbw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-cswbw" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070359 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199146 738 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070571 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199216 738 reflector.go:138] object-"kube-system"/"coredns-token-q7qdg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-q7qdg" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070775 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199262 738 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.075454 1540905 logs.go:138] Found kubelet problem: May 28 22:24:04 old-k8s-version-137556 kubelet[738]: E0528 22:24:04.243616 738 reflector.go:138] object-"kube-system"/"metrics-server-token-42d6m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-42d6m" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.079777 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.119557 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.079965 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.781597 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.082067 1540905 logs.go:138] Found kubelet problem: May 28 22:24:20 old-k8s-version-137556 kubelet[738]: E0528 22:24:20.203798 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.083824 1540905 logs.go:138] Found kubelet problem: May 28 22:24:33 old-k8s-version-137556 kubelet[738]: E0528 22:24:33.708202 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.084544 1540905 logs.go:138] Found kubelet problem: May 28 22:24:38 old-k8s-version-137556 kubelet[738]: E0528 22:24:38.049446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.084872 1540905 logs.go:138] Found kubelet problem: May 28 22:24:39 old-k8s-version-137556 kubelet[738]: E0528 22:24:39.048542 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.085200 1540905 logs.go:138] Found kubelet problem: May 28 22:24:46 old-k8s-version-137556 kubelet[738]: E0528 22:24:46.690430 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.087272 1540905 logs.go:138] Found kubelet problem: May 28 22:24:48 old-k8s-version-137556 kubelet[738]: E0528 22:24:48.729167 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.087463 1540905 logs.go:138] Found kubelet problem: May 28 22:25:00 old-k8s-version-137556 kubelet[738]: E0528 22:25:00.705605 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.088052 1540905 logs.go:138] Found kubelet problem: May 28 22:25:02 old-k8s-version-137556 kubelet[738]: E0528 22:25:02.084189 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.088378 1540905 logs.go:138] Found kubelet problem: May 28 22:25:06 old-k8s-version-137556 kubelet[738]: E0528 22:25:06.690524 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.088562 1540905 logs.go:138] Found kubelet problem: May 28 22:25:13 old-k8s-version-137556 kubelet[738]: E0528 22:25:13.705095 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.088886 1540905 logs.go:138] Found kubelet problem: May 28 22:25:19 old-k8s-version-137556 kubelet[738]: E0528 22:25:19.704584 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.089072 1540905 logs.go:138] Found kubelet problem: May 28 22:25:24 old-k8s-version-137556 kubelet[738]: E0528 22:25:24.704685 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.089655 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.141913 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.091744 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.718176 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.092073 1540905 logs.go:138] Found kubelet problem: May 28 22:25:36 old-k8s-version-137556 kubelet[738]: E0528 22:25:36.690449 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.092399 1540905 logs.go:138] Found kubelet problem: May 28 22:25:47 old-k8s-version-137556 kubelet[738]: E0528 22:25:47.705446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.092585 1540905 logs.go:138] Found kubelet problem: May 28 22:25:49 old-k8s-version-137556 kubelet[738]: E0528 22:25:49.705980 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.092918 1540905 logs.go:138] Found kubelet problem: May 28 22:26:00 old-k8s-version-137556 kubelet[738]: E0528 22:26:00.704155 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.093104 1540905 logs.go:138] Found kubelet problem: May 28 22:26:01 old-k8s-version-137556 kubelet[738]: E0528 22:26:01.704811 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.093692 1540905 logs.go:138] Found kubelet problem: May 28 22:26:15 old-k8s-version-137556 kubelet[738]: E0528 22:26:15.207095 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094030 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.690320 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094212 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.704608 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.094541 1540905 logs.go:138] Found kubelet problem: May 28 22:26:28 old-k8s-version-137556 kubelet[738]: E0528 22:26:28.704143 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094726 1540905 logs.go:138] Found kubelet problem: May 28 22:26:29 old-k8s-version-137556 kubelet[738]: E0528 22:26:29.704616 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.095052 1540905 logs.go:138] Found kubelet problem: May 28 22:26:39 old-k8s-version-137556 kubelet[738]: E0528 22:26:39.704876 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.095237 1540905 logs.go:138] Found kubelet problem: May 28 22:26:43 old-k8s-version-137556 kubelet[738]: E0528 22:26:43.705110 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.096084 1540905 logs.go:138] Found kubelet problem: May 28 22:26:53 old-k8s-version-137556 kubelet[738]: E0528 22:26:53.704310 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.096269 1540905 logs.go:138] Found kubelet problem: May 28 22:26:54 old-k8s-version-137556 kubelet[738]: E0528 22:26:54.704655 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.096595 1540905 logs.go:138] Found kubelet problem: May 28 22:27:06 old-k8s-version-137556 kubelet[738]: E0528 22:27:06.704128 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.098660 1540905 logs.go:138] Found kubelet problem: May 28 22:27:09 old-k8s-version-137556 kubelet[738]: E0528 22:27:09.715928 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.098988 1540905 logs.go:138] Found kubelet problem: May 28 22:27:17 old-k8s-version-137556 kubelet[738]: E0528 22:27:17.704167 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.099172 1540905 logs.go:138] Found kubelet problem: May 28 22:27:21 old-k8s-version-137556 kubelet[738]: E0528 22:27:21.704885 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.099502 1540905 logs.go:138] Found kubelet problem: May 28 22:27:30 old-k8s-version-137556 kubelet[738]: E0528 22:27:30.704161 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.099687 1540905 logs.go:138] Found kubelet problem: May 28 22:27:33 old-k8s-version-137556 kubelet[738]: E0528 22:27:33.704586 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.100291 1540905 logs.go:138] Found kubelet problem: May 28 22:27:44 old-k8s-version-137556 kubelet[738]: E0528 22:27:44.340886 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.100622 1540905 logs.go:138] Found kubelet problem: May 28 22:27:46 old-k8s-version-137556 kubelet[738]: E0528 22:27:46.690347 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.100807 1540905 logs.go:138] Found kubelet problem: May 28 22:27:48 old-k8s-version-137556 kubelet[738]: E0528 22:27:48.704625 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101135 1540905 logs.go:138] Found kubelet problem: May 28 22:27:58 old-k8s-version-137556 kubelet[738]: E0528 22:27:58.704805 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.101320 1540905 logs.go:138] Found kubelet problem: May 28 22:28:00 old-k8s-version-137556 kubelet[738]: E0528 22:28:00.704891 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101634 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705029 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101831 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705742 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.102015 1540905 logs.go:138] Found kubelet problem: May 28 22:28:24 old-k8s-version-137556 kubelet[738]: E0528 22:28:24.704663 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.102358 1540905 logs.go:138] Found kubelet problem: May 28 22:28:26 old-k8s-version-137556 kubelet[738]: E0528 22:28:26.704162 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.102544 1540905 logs.go:138] Found kubelet problem: May 28 22:28:39 old-k8s-version-137556 kubelet[738]: E0528 22:28:39.704787 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.102873 1540905 logs.go:138] Found kubelet problem: May 28 22:28:40 old-k8s-version-137556 kubelet[738]: E0528 22:28:40.704255 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.103452 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.704205 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.103640 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.103825 1540905 logs.go:138] Found kubelet problem: May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.104151 1540905 logs.go:138] Found kubelet problem: May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.104340 1540905 logs.go:138] Found kubelet problem: May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.104665 1540905 logs.go:138] Found kubelet problem: May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
I0528 22:29:22.104674 1540905 logs.go:123] Gathering logs for kube-proxy [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b] ...
I0528 22:29:22.104688 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:22.141117 1540905 logs.go:123] Gathering logs for kubernetes-dashboard [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527] ...
I0528 22:29:22.141148 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:22.180279 1540905 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:22.180303 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:22.275290 1540905 logs.go:123] Gathering logs for container status ...
I0528 22:29:22.275341 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:22.320717 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:22.320742 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:22.320829 1540905 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0528 22:29:22.320847 1540905 out.go:239] May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.320988 1540905 out.go:239] May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.321016 1540905 out.go:239] May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.321024 1540905 out.go:239] May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.321042 1540905 out.go:239] May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
I0528 22:29:22.321050 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:22.321057 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:32.323043 1540905 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0528 22:29:32.335698 1540905 api_server.go:72] duration metric: took 5m48.896786162s to wait for apiserver process to appear ...
I0528 22:29:32.335723 1540905 api_server.go:88] waiting for apiserver healthz status ...
I0528 22:29:32.335762 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:32.335824 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:32.374464 1540905 cri.go:89] found id: "9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:32.374485 1540905 cri.go:89] found id: ""
I0528 22:29:32.374494 1540905 logs.go:276] 1 containers: [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3]
I0528 22:29:32.374556 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.378665 1540905 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:32.378735 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:32.428280 1540905 cri.go:89] found id: "c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:32.428310 1540905 cri.go:89] found id: ""
I0528 22:29:32.428319 1540905 logs.go:276] 1 containers: [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8]
I0528 22:29:32.428377 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.432088 1540905 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:32.432153 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:32.474292 1540905 cri.go:89] found id: "cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:32.474312 1540905 cri.go:89] found id: ""
I0528 22:29:32.474319 1540905 logs.go:276] 1 containers: [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a]
I0528 22:29:32.474376 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.477891 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:32.477960 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:32.521895 1540905 cri.go:89] found id: "b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:32.521917 1540905 cri.go:89] found id: ""
I0528 22:29:32.521925 1540905 logs.go:276] 1 containers: [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105]
I0528 22:29:32.521982 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.526385 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:32.526465 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:32.568221 1540905 cri.go:89] found id: "7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:32.568241 1540905 cri.go:89] found id: ""
I0528 22:29:32.568249 1540905 logs.go:276] 1 containers: [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b]
I0528 22:29:32.568304 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.571975 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:32.572079 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:32.609486 1540905 cri.go:89] found id: "41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:32.609510 1540905 cri.go:89] found id: ""
I0528 22:29:32.609517 1540905 logs.go:276] 1 containers: [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29]
I0528 22:29:32.609574 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.612947 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:32.613016 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:32.652655 1540905 cri.go:89] found id: "dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:32.652680 1540905 cri.go:89] found id: ""
I0528 22:29:32.652689 1540905 logs.go:276] 1 containers: [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5]
I0528 22:29:32.652748 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.656315 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:32.656442 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:32.693909 1540905 cri.go:89] found id: "bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:32.693941 1540905 cri.go:89] found id: ""
I0528 22:29:32.693949 1540905 logs.go:276] 1 containers: [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527]
I0528 22:29:32.694072 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.697734 1540905 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:32.697815 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:32.736027 1540905 cri.go:89] found id: "4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:32.736053 1540905 cri.go:89] found id: "05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:32.736058 1540905 cri.go:89] found id: ""
I0528 22:29:32.736065 1540905 logs.go:276] 2 containers: [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998]
I0528 22:29:32.736150 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.742328 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.746335 1540905 logs.go:123] Gathering logs for storage-provisioner [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33] ...
I0528 22:29:32.746363 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:32.783928 1540905 logs.go:123] Gathering logs for storage-provisioner [05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998] ...
I0528 22:29:32.783955 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:32.821570 1540905 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:32.821598 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:32.907550 1540905 logs.go:123] Gathering logs for kubernetes-dashboard [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527] ...
I0528 22:29:32.907629 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:32.952689 1540905 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:32.952720 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:32.971568 1540905 logs.go:123] Gathering logs for etcd [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8] ...
I0528 22:29:32.971598 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:33.039002 1540905 logs.go:123] Gathering logs for kube-scheduler [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105] ...
I0528 22:29:33.039034 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:33.090654 1540905 logs.go:123] Gathering logs for kube-proxy [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b] ...
I0528 22:29:33.090687 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:33.136556 1540905 logs.go:123] Gathering logs for kindnet [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5] ...
I0528 22:29:33.136585 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:33.192155 1540905 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:33.192228 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:33.246432 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.186601 738 reflector.go:138] object-"kube-system"/"kindnet-token-ztnvd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ztnvd" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.246672 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187153 738 reflector.go:138] object-"kube-system"/"kube-proxy-token-cs2pc": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-cs2pc" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.246887 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187714 738 reflector.go:138] object-"default"/"default-token-k8l4j": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-k8l4j" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247119 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.194216 738 reflector.go:138] object-"kube-system"/"storage-provisioner-token-cswbw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-cswbw" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247320 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199146 738 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247555 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199216 738 reflector.go:138] object-"kube-system"/"coredns-token-q7qdg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-q7qdg" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247760 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199262 738 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.252483 1540905 logs.go:138] Found kubelet problem: May 28 22:24:04 old-k8s-version-137556 kubelet[738]: E0528 22:24:04.243616 738 reflector.go:138] object-"kube-system"/"metrics-server-token-42d6m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-42d6m" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.256917 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.119557 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.257113 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.781597 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.259198 1540905 logs.go:138] Found kubelet problem: May 28 22:24:20 old-k8s-version-137556 kubelet[738]: E0528 22:24:20.203798 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.260974 1540905 logs.go:138] Found kubelet problem: May 28 22:24:33 old-k8s-version-137556 kubelet[738]: E0528 22:24:33.708202 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.261692 1540905 logs.go:138] Found kubelet problem: May 28 22:24:38 old-k8s-version-137556 kubelet[738]: E0528 22:24:38.049446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.262026 1540905 logs.go:138] Found kubelet problem: May 28 22:24:39 old-k8s-version-137556 kubelet[738]: E0528 22:24:39.048542 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.262358 1540905 logs.go:138] Found kubelet problem: May 28 22:24:46 old-k8s-version-137556 kubelet[738]: E0528 22:24:46.690430 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.265262 1540905 logs.go:138] Found kubelet problem: May 28 22:24:48 old-k8s-version-137556 kubelet[738]: E0528 22:24:48.729167 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.265472 1540905 logs.go:138] Found kubelet problem: May 28 22:25:00 old-k8s-version-137556 kubelet[738]: E0528 22:25:00.705605 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.266085 1540905 logs.go:138] Found kubelet problem: May 28 22:25:02 old-k8s-version-137556 kubelet[738]: E0528 22:25:02.084189 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.266416 1540905 logs.go:138] Found kubelet problem: May 28 22:25:06 old-k8s-version-137556 kubelet[738]: E0528 22:25:06.690524 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.266603 1540905 logs.go:138] Found kubelet problem: May 28 22:25:13 old-k8s-version-137556 kubelet[738]: E0528 22:25:13.705095 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.266933 1540905 logs.go:138] Found kubelet problem: May 28 22:25:19 old-k8s-version-137556 kubelet[738]: E0528 22:25:19.704584 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.267118 1540905 logs.go:138] Found kubelet problem: May 28 22:25:24 old-k8s-version-137556 kubelet[738]: E0528 22:25:24.704685 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.267735 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.141913 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.269944 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.718176 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.270294 1540905 logs.go:138] Found kubelet problem: May 28 22:25:36 old-k8s-version-137556 kubelet[738]: E0528 22:25:36.690449 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.270624 1540905 logs.go:138] Found kubelet problem: May 28 22:25:47 old-k8s-version-137556 kubelet[738]: E0528 22:25:47.705446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.271334 1540905 logs.go:138] Found kubelet problem: May 28 22:25:49 old-k8s-version-137556 kubelet[738]: E0528 22:25:49.705980 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.271686 1540905 logs.go:138] Found kubelet problem: May 28 22:26:00 old-k8s-version-137556 kubelet[738]: E0528 22:26:00.704155 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.271875 1540905 logs.go:138] Found kubelet problem: May 28 22:26:01 old-k8s-version-137556 kubelet[738]: E0528 22:26:01.704811 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.272461 1540905 logs.go:138] Found kubelet problem: May 28 22:26:15 old-k8s-version-137556 kubelet[738]: E0528 22:26:15.207095 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.272794 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.690320 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.272980 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.704608 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.273349 1540905 logs.go:138] Found kubelet problem: May 28 22:26:28 old-k8s-version-137556 kubelet[738]: E0528 22:26:28.704143 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.273541 1540905 logs.go:138] Found kubelet problem: May 28 22:26:29 old-k8s-version-137556 kubelet[738]: E0528 22:26:29.704616 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.273874 1540905 logs.go:138] Found kubelet problem: May 28 22:26:39 old-k8s-version-137556 kubelet[738]: E0528 22:26:39.704876 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.274074 1540905 logs.go:138] Found kubelet problem: May 28 22:26:43 old-k8s-version-137556 kubelet[738]: E0528 22:26:43.705110 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.274980 1540905 logs.go:138] Found kubelet problem: May 28 22:26:53 old-k8s-version-137556 kubelet[738]: E0528 22:26:53.704310 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.275170 1540905 logs.go:138] Found kubelet problem: May 28 22:26:54 old-k8s-version-137556 kubelet[738]: E0528 22:26:54.704655 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.275499 1540905 logs.go:138] Found kubelet problem: May 28 22:27:06 old-k8s-version-137556 kubelet[738]: E0528 22:27:06.704128 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.277543 1540905 logs.go:138] Found kubelet problem: May 28 22:27:09 old-k8s-version-137556 kubelet[738]: E0528 22:27:09.715928 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.277870 1540905 logs.go:138] Found kubelet problem: May 28 22:27:17 old-k8s-version-137556 kubelet[738]: E0528 22:27:17.704167 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.278074 1540905 logs.go:138] Found kubelet problem: May 28 22:27:21 old-k8s-version-137556 kubelet[738]: E0528 22:27:21.704885 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.278407 1540905 logs.go:138] Found kubelet problem: May 28 22:27:30 old-k8s-version-137556 kubelet[738]: E0528 22:27:30.704161 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.278671 1540905 logs.go:138] Found kubelet problem: May 28 22:27:33 old-k8s-version-137556 kubelet[738]: E0528 22:27:33.704586 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.279301 1540905 logs.go:138] Found kubelet problem: May 28 22:27:44 old-k8s-version-137556 kubelet[738]: E0528 22:27:44.340886 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.279669 1540905 logs.go:138] Found kubelet problem: May 28 22:27:46 old-k8s-version-137556 kubelet[738]: E0528 22:27:46.690347 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.279882 1540905 logs.go:138] Found kubelet problem: May 28 22:27:48 old-k8s-version-137556 kubelet[738]: E0528 22:27:48.704625 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.280239 1540905 logs.go:138] Found kubelet problem: May 28 22:27:58 old-k8s-version-137556 kubelet[738]: E0528 22:27:58.704805 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.280461 1540905 logs.go:138] Found kubelet problem: May 28 22:28:00 old-k8s-version-137556 kubelet[738]: E0528 22:28:00.704891 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.280810 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705029 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.281039 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705742 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.281253 1540905 logs.go:138] Found kubelet problem: May 28 22:28:24 old-k8s-version-137556 kubelet[738]: E0528 22:28:24.704663 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.281601 1540905 logs.go:138] Found kubelet problem: May 28 22:28:26 old-k8s-version-137556 kubelet[738]: E0528 22:28:26.704162 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.281815 1540905 logs.go:138] Found kubelet problem: May 28 22:28:39 old-k8s-version-137556 kubelet[738]: E0528 22:28:39.704787 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.282338 1540905 logs.go:138] Found kubelet problem: May 28 22:28:40 old-k8s-version-137556 kubelet[738]: E0528 22:28:40.704255 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.282977 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.704205 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.283198 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.283416 1540905 logs.go:138] Found kubelet problem: May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.284178 1540905 logs.go:138] Found kubelet problem: May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.284427 1540905 logs.go:138] Found kubelet problem: May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.284797 1540905 logs.go:138] Found kubelet problem: May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.285031 1540905 logs.go:138] Found kubelet problem: May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0528 22:29:33.285058 1540905 logs.go:123] Gathering logs for kube-apiserver [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3] ...
I0528 22:29:33.285090 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:33.358393 1540905 logs.go:123] Gathering logs for container status ...
I0528 22:29:33.358434 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:33.410155 1540905 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:33.410232 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:33.558604 1540905 logs.go:123] Gathering logs for coredns [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a] ...
I0528 22:29:33.558637 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:33.598686 1540905 logs.go:123] Gathering logs for kube-controller-manager [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29] ...
I0528 22:29:33.598716 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:33.670199 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:33.670230 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:33.670308 1540905 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0528 22:29:33.670324 1540905 out.go:239] May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.670455 1540905 out.go:239] May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.670464 1540905 out.go:239] May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.670488 1540905 out.go:239] May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.670497 1540905 out.go:239] May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0528 22:29:33.670507 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:33.670514 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:43.671573 1540905 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0528 22:29:43.691524 1540905 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0528 22:29:43.693704 1540905 out.go:177]
W0528 22:29:43.696067 1540905 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0528 22:29:43.696109 1540905 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0528 22:29:43.696128 1540905 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0528 22:29:43.696133 1540905 out.go:239] *
*
W0528 22:29:43.697172 1540905 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0528 22:29:43.699282 1540905 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-137556 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=crio --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-137556
helpers_test.go:235: (dbg) docker inspect old-k8s-version-137556:
-- stdout --
[
{
"Id": "34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592",
"Created": "2024-05-28T22:20:30.593486142Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1541142,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-05-28T22:23:35.236400801Z",
"FinishedAt": "2024-05-28T22:23:33.929963613Z"
},
"Image": "sha256:acea75078737755d2f999491dfa245ea1d1040bffc73283b8c9ba9ff1fde89b5",
"ResolvConfPath": "/var/lib/docker/containers/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592/hostname",
"HostsPath": "/var/lib/docker/containers/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592/hosts",
"LogPath": "/var/lib/docker/containers/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592-json.log",
"Name": "/old-k8s-version-137556",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"old-k8s-version-137556:/var",
"/lib/modules:/lib/modules:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-137556",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/dcb7b39eec5c282cf08db0a244cb2bbb481d91d8b22f64c75f2fd4605431dc6c-init/diff:/var/lib/docker/overlay2/41cb90b313a958e97d6c40ed76425369b134e98a770fd8f601707592b588c01d/diff",
"MergedDir": "/var/lib/docker/overlay2/dcb7b39eec5c282cf08db0a244cb2bbb481d91d8b22f64c75f2fd4605431dc6c/merged",
"UpperDir": "/var/lib/docker/overlay2/dcb7b39eec5c282cf08db0a244cb2bbb481d91d8b22f64c75f2fd4605431dc6c/diff",
"WorkDir": "/var/lib/docker/overlay2/dcb7b39eec5c282cf08db0a244cb2bbb481d91d8b22f64c75f2fd4605431dc6c/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-137556",
"Source": "/var/lib/docker/volumes/old-k8s-version-137556/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-137556",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-137556",
"name.minikube.sigs.k8s.io": "old-k8s-version-137556",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "6dcf91602b61d4206e9bbc36d4a140d5195f1b3ee4ffb18922d9a3a883ee499f",
"SandboxKey": "/var/run/docker/netns/6dcf91602b61",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34589"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34588"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34585"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34587"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34586"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-137556": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:55:02",
"NetworkID": "06df03bcb9a730cd78ebc03cfb3323e1208621b10bd8b73f11e4f24de9725707",
"EndpointID": "1ac3e023546291c4b1869fafc48bb4a7a3b4b2df3704ad860b9d8d0feaac0750",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-137556",
"34fd7d98b135"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-137556 -n old-k8s-version-137556
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-137556 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-137556 logs -n 25: (2.090093681s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| ssh | -p cilium-982195 sudo find | cilium-982195 | jenkins | v1.33.1 | 28 May 24 22:18 UTC | |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p cilium-982195 sudo crio | cilium-982195 | jenkins | v1.33.1 | 28 May 24 22:18 UTC | |
| | config | | | | | |
| delete | -p cilium-982195 | cilium-982195 | jenkins | v1.33.1 | 28 May 24 22:18 UTC | 28 May 24 22:18 UTC |
| start | -p force-systemd-env-959636 | force-systemd-env-959636 | jenkins | v1.33.1 | 28 May 24 22:18 UTC | 28 May 24 22:19 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| start | -p kubernetes-upgrade-510763 | kubernetes-upgrade-510763 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | |
| | --memory=2200 | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| start | -p kubernetes-upgrade-510763 | kubernetes-upgrade-510763 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | 28 May 24 22:19 UTC |
| | --memory=2200 | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=1 --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| delete | -p kubernetes-upgrade-510763 | kubernetes-upgrade-510763 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | 28 May 24 22:19 UTC |
| start | -p cert-expiration-511332 | cert-expiration-511332 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | 28 May 24 22:20 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| delete | -p force-systemd-env-959636 | force-systemd-env-959636 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | 28 May 24 22:19 UTC |
| start | -p cert-options-768265 | cert-options-768265 | jenkins | v1.33.1 | 28 May 24 22:19 UTC | 28 May 24 22:20 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| ssh | cert-options-768265 ssh | cert-options-768265 | jenkins | v1.33.1 | 28 May 24 22:20 UTC | 28 May 24 22:20 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-768265 -- sudo | cert-options-768265 | jenkins | v1.33.1 | 28 May 24 22:20 UTC | 28 May 24 22:20 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-768265 | cert-options-768265 | jenkins | v1.33.1 | 28 May 24 22:20 UTC | 28 May 24 22:20 UTC |
| start | -p old-k8s-version-137556 | old-k8s-version-137556 | jenkins | v1.33.1 | 28 May 24 22:20 UTC | 28 May 24 22:23 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-511332 | cert-expiration-511332 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:23 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| addons | enable metrics-server -p old-k8s-version-137556 | old-k8s-version-137556 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:23 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-137556 | old-k8s-version-137556 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:23 UTC |
| | --alsologtostderr -v=3 | | | | | |
| delete | -p cert-expiration-511332 | cert-expiration-511332 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:23 UTC |
| start | -p no-preload-264173 | no-preload-264173 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:24 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
| addons | enable dashboard -p old-k8s-version-137556 | old-k8s-version-137556 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | 28 May 24 22:23 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-137556 | old-k8s-version-137556 | jenkins | v1.33.1 | 28 May 24 22:23 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-264173 | no-preload-264173 | jenkins | v1.33.1 | 28 May 24 22:24 UTC | 28 May 24 22:24 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-264173 | no-preload-264173 | jenkins | v1.33.1 | 28 May 24 22:24 UTC | 28 May 24 22:25 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-264173 | no-preload-264173 | jenkins | v1.33.1 | 28 May 24 22:25 UTC | 28 May 24 22:25 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-264173 | no-preload-264173 | jenkins | v1.33.1 | 28 May 24 22:25 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=crio | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/05/28 22:25:00
Running on machine: ip-172-31-29-130
Binary: Built with gc go1.22.3 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0528 22:25:00.898762 1546162 out.go:291] Setting OutFile to fd 1 ...
I0528 22:25:00.899001 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:25:00.899016 1546162 out.go:304] Setting ErrFile to fd 2...
I0528 22:25:00.899021 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:25:00.899325 1546162 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18966-1349783/.minikube/bin
I0528 22:25:00.899747 1546162 out.go:298] Setting JSON to false
I0528 22:25:00.900812 1546162 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":22049,"bootTime":1716913052,"procs":210,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1062-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0528 22:25:00.900891 1546162 start.go:139] virtualization:
I0528 22:25:00.905505 1546162 out.go:177] * [no-preload-264173] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0528 22:25:00.907719 1546162 out.go:177] - MINIKUBE_LOCATION=18966
I0528 22:25:00.910487 1546162 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0528 22:25:00.907774 1546162 notify.go:220] Checking for updates...
I0528 22:25:00.913241 1546162 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:25:00.915438 1546162 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18966-1349783/.minikube
I0528 22:25:00.917565 1546162 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0528 22:25:00.919862 1546162 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0528 22:25:00.922785 1546162 config.go:182] Loaded profile config "no-preload-264173": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 22:25:00.923331 1546162 driver.go:392] Setting default libvirt URI to qemu:///system
I0528 22:25:00.948567 1546162 docker.go:122] docker version: linux-26.1.3:Docker Engine - Community
I0528 22:25:00.948717 1546162 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0528 22:25:01.021288 1546162 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:37 OomKillDisable:true NGoroutines:52 SystemTime:2024-05-28 22:25:01.011657777 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1062-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214974464 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89 Expected:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0528 22:25:01.021402 1546162 docker.go:295] overlay module found
I0528 22:25:01.024744 1546162 out.go:177] * Using the docker driver based on existing profile
I0528 22:25:01.027147 1546162 start.go:297] selected driver: docker
I0528 22:25:01.027173 1546162 start.go:901] validating driver "docker" against &{Name:no-preload-264173 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:no-preload-264173 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home
/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:25:01.027374 1546162 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0528 22:25:01.028059 1546162 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0528 22:25:01.079767 1546162 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:37 OomKillDisable:true NGoroutines:52 SystemTime:2024-05-28 22:25:01.069750615 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1062-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214974464 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89 Expected:8b3b7ca2e5ce38e8f31a34f35b2b68ceb8470d89} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0528 22:25:01.080157 1546162 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0528 22:25:01.080187 1546162 cni.go:84] Creating CNI manager for ""
I0528 22:25:01.080195 1546162 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0528 22:25:01.080254 1546162 start.go:340] cluster config:
{Name:no-preload-264173 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:no-preload-264173 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMS
ize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:25:01.082911 1546162 out.go:177] * Starting "no-preload-264173" primary control-plane node in "no-preload-264173" cluster
I0528 22:25:01.085222 1546162 cache.go:121] Beginning downloading kic base image for docker with crio
I0528 22:25:01.087522 1546162 out.go:177] * Pulling base image v0.0.44-1716228441-18934 ...
I0528 22:25:01.090440 1546162 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 in local docker daemon
I0528 22:25:01.090629 1546162 preload.go:132] Checking if preload exists for k8s version v1.30.1 and runtime crio
I0528 22:25:01.090768 1546162 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/config.json ...
I0528 22:25:01.091081 1546162 cache.go:107] acquiring lock: {Name:mk657eded7f3162e770147cb0b4b3152fb7e76ad Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091163 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0528 22:25:01.091177 1546162 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 100.732µs
I0528 22:25:01.091194 1546162 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0528 22:25:01.091303 1546162 cache.go:107] acquiring lock: {Name:mkab6f49139af1588edb99533d7df58a239cfcf5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091364 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.1 exists
I0528 22:25:01.091373 1546162 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.30.1" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.1" took 74.944µs
I0528 22:25:01.091381 1546162 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.30.1 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.30.1 succeeded
I0528 22:25:01.091393 1546162 cache.go:107] acquiring lock: {Name:mk2c7d6b1c9426f8d4fbc55d66c0bae5158c4382 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091431 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.1 exists
I0528 22:25:01.091440 1546162 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.30.1" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.1" took 48.835µs
I0528 22:25:01.091446 1546162 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.30.1 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.30.1 succeeded
I0528 22:25:01.091456 1546162 cache.go:107] acquiring lock: {Name:mkb8d69f358bc4da52023bf1814c5a76fe601c4b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091500 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.1 exists
I0528 22:25:01.091510 1546162 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.30.1" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.1" took 55.891µs
I0528 22:25:01.091517 1546162 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.30.1 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.30.1 succeeded
I0528 22:25:01.091526 1546162 cache.go:107] acquiring lock: {Name:mkd1d0631bf4ec5aa92850f985fdd52f6d1b897a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091556 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.1 exists
I0528 22:25:01.091561 1546162 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.30.1" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.1" took 36.167µs
I0528 22:25:01.091571 1546162 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.30.1 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.30.1 succeeded
I0528 22:25:01.091585 1546162 cache.go:107] acquiring lock: {Name:mk8e65c5299968149ba0b83461da6df903fe8422 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091620 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0 exists
I0528 22:25:01.091629 1546162 cache.go:96] cache image "registry.k8s.io/etcd:3.5.12-0" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0" took 47.646µs
I0528 22:25:01.091635 1546162 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.12-0 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.12-0 succeeded
I0528 22:25:01.091647 1546162 cache.go:107] acquiring lock: {Name:mk4c83f437e70c0f7f8995f5f4a8d6feb99324c9 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091680 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9 exists
I0528 22:25:01.091690 1546162 cache.go:96] cache image "registry.k8s.io/pause:3.9" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9" took 43.708µs
I0528 22:25:01.091696 1546162 cache.go:80] save to tar file registry.k8s.io/pause:3.9 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/pause_3.9 succeeded
I0528 22:25:01.091711 1546162 cache.go:107] acquiring lock: {Name:mk52cbc84a8bf1a6423d99bba62b99f08b7b20c5 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.091743 1546162 cache.go:115] /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 exists
I0528 22:25:01.091754 1546162 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.1" -> "/home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1" took 49.574µs
I0528 22:25:01.091763 1546162 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.1 -> /home/jenkins/minikube-integration/18966-1349783/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 succeeded
I0528 22:25:01.091770 1546162 cache.go:87] Successfully saved all images to host disk.
I0528 22:25:01.111086 1546162 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 in local docker daemon, skipping pull
I0528 22:25:01.111117 1546162 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 exists in daemon, skipping load
I0528 22:25:01.111139 1546162 cache.go:194] Successfully downloaded all kic artifacts
I0528 22:25:01.111167 1546162 start.go:360] acquireMachinesLock for no-preload-264173: {Name:mkaefb5e133965f56a195fd090186d52ce52ef7d Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0528 22:25:01.111313 1546162 start.go:364] duration metric: took 124.608µs to acquireMachinesLock for "no-preload-264173"
I0528 22:25:01.111341 1546162 start.go:96] Skipping create...Using existing machine configuration
I0528 22:25:01.111359 1546162 fix.go:54] fixHost starting:
I0528 22:25:01.111650 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:01.127574 1546162 fix.go:112] recreateIfNeeded on no-preload-264173: state=Stopped err=<nil>
W0528 22:25:01.127608 1546162 fix.go:138] unexpected machine state, will restart: <nil>
I0528 22:25:01.130517 1546162 out.go:177] * Restarting existing docker container for "no-preload-264173" ...
I0528 22:25:01.446499 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:03.946841 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:01.132971 1546162 cli_runner.go:164] Run: docker start no-preload-264173
I0528 22:25:01.479923 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:01.505196 1546162 kic.go:430] container "no-preload-264173" state is running.
I0528 22:25:01.505737 1546162 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-264173
I0528 22:25:01.536246 1546162 profile.go:143] Saving config to /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/config.json ...
I0528 22:25:01.536474 1546162 machine.go:94] provisionDockerMachine start ...
I0528 22:25:01.536541 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:01.562236 1546162 main.go:141] libmachine: Using SSH client type: native
I0528 22:25:01.562867 1546162 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34594 <nil> <nil>}
I0528 22:25:01.562886 1546162 main.go:141] libmachine: About to run SSH command:
hostname
I0528 22:25:01.563784 1546162 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0528 22:25:04.685817 1546162 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-264173
I0528 22:25:04.685841 1546162 ubuntu.go:169] provisioning hostname "no-preload-264173"
I0528 22:25:04.685915 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:04.705801 1546162 main.go:141] libmachine: Using SSH client type: native
I0528 22:25:04.706131 1546162 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34594 <nil> <nil>}
I0528 22:25:04.706151 1546162 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-264173 && echo "no-preload-264173" | sudo tee /etc/hostname
I0528 22:25:04.842429 1546162 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-264173
I0528 22:25:04.842510 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:04.859309 1546162 main.go:141] libmachine: Using SSH client type: native
I0528 22:25:04.859575 1546162 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34594 <nil> <nil>}
I0528 22:25:04.859605 1546162 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-264173' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-264173/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-264173' | sudo tee -a /etc/hosts;
fi
fi
I0528 22:25:04.987308 1546162 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0528 22:25:04.987337 1546162 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18966-1349783/.minikube CaCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18966-1349783/.minikube}
I0528 22:25:04.987375 1546162 ubuntu.go:177] setting up certificates
I0528 22:25:04.987387 1546162 provision.go:84] configureAuth start
I0528 22:25:04.987471 1546162 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-264173
I0528 22:25:05.009521 1546162 provision.go:143] copyHostCerts
I0528 22:25:05.009622 1546162 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem, removing ...
I0528 22:25:05.009633 1546162 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem
I0528 22:25:05.009725 1546162 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.pem (1082 bytes)
I0528 22:25:05.009841 1546162 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem, removing ...
I0528 22:25:05.009847 1546162 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem
I0528 22:25:05.009880 1546162 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/cert.pem (1123 bytes)
I0528 22:25:05.009937 1546162 exec_runner.go:144] found /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem, removing ...
I0528 22:25:05.009942 1546162 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem
I0528 22:25:05.009970 1546162 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18966-1349783/.minikube/key.pem (1675 bytes)
I0528 22:25:05.010056 1546162 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem org=jenkins.no-preload-264173 san=[127.0.0.1 192.168.76.2 localhost minikube no-preload-264173]
I0528 22:25:05.347349 1546162 provision.go:177] copyRemoteCerts
I0528 22:25:05.347425 1546162 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0528 22:25:05.347473 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:05.367428 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:05.471164 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0528 22:25:05.498236 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0528 22:25:05.523196 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0528 22:25:05.548839 1546162 provision.go:87] duration metric: took 561.434071ms to configureAuth
I0528 22:25:05.548865 1546162 ubuntu.go:193] setting minikube options for container-runtime
I0528 22:25:05.549096 1546162 config.go:182] Loaded profile config "no-preload-264173": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 22:25:05.549217 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:05.569989 1546162 main.go:141] libmachine: Using SSH client type: native
I0528 22:25:05.570307 1546162 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 34594 <nil> <nil>}
I0528 22:25:05.570333 1546162 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /etc/sysconfig && printf %!s(MISSING) "
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
" | sudo tee /etc/sysconfig/crio.minikube && sudo systemctl restart crio
I0528 22:25:05.937666 1546162 main.go:141] libmachine: SSH cmd err, output: <nil>:
CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
I0528 22:25:05.937745 1546162 machine.go:97] duration metric: took 4.401260705s to provisionDockerMachine
I0528 22:25:05.937770 1546162 start.go:293] postStartSetup for "no-preload-264173" (driver="docker")
I0528 22:25:05.937812 1546162 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0528 22:25:05.937899 1546162 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0528 22:25:05.937984 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:05.962474 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:06.067602 1546162 ssh_runner.go:195] Run: cat /etc/os-release
I0528 22:25:06.071412 1546162 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0528 22:25:06.071453 1546162 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0528 22:25:06.071464 1546162 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0528 22:25:06.071471 1546162 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0528 22:25:06.071483 1546162 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-1349783/.minikube/addons for local assets ...
I0528 22:25:06.071556 1546162 filesync.go:126] Scanning /home/jenkins/minikube-integration/18966-1349783/.minikube/files for local assets ...
I0528 22:25:06.071666 1546162 filesync.go:149] local asset: /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem -> 13551972.pem in /etc/ssl/certs
I0528 22:25:06.071784 1546162 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0528 22:25:06.081082 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem --> /etc/ssl/certs/13551972.pem (1708 bytes)
I0528 22:25:06.110179 1546162 start.go:296] duration metric: took 172.363223ms for postStartSetup
I0528 22:25:06.110269 1546162 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0528 22:25:06.110341 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:06.133986 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:06.223465 1546162 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0528 22:25:06.227993 1546162 fix.go:56] duration metric: took 5.116634799s for fixHost
I0528 22:25:06.228019 1546162 start.go:83] releasing machines lock for "no-preload-264173", held for 5.116690675s
I0528 22:25:06.228111 1546162 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-264173
I0528 22:25:06.248484 1546162 ssh_runner.go:195] Run: cat /version.json
I0528 22:25:06.248505 1546162 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0528 22:25:06.248540 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:06.248563 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:06.270348 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:06.293219 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:06.505358 1546162 ssh_runner.go:195] Run: systemctl --version
I0528 22:25:06.510168 1546162 ssh_runner.go:195] Run: sudo sh -c "podman version >/dev/null"
I0528 22:25:06.652833 1546162 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0528 22:25:06.658624 1546162 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0528 22:25:06.667683 1546162 cni.go:221] loopback cni configuration disabled: "/etc/cni/net.d/*loopback.conf*" found
I0528 22:25:06.667766 1546162 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0528 22:25:06.676858 1546162 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0528 22:25:06.676882 1546162 start.go:494] detecting cgroup driver to use...
I0528 22:25:06.676915 1546162 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0528 22:25:06.676963 1546162 ssh_runner.go:195] Run: sudo systemctl stop -f containerd
I0528 22:25:06.689745 1546162 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service containerd
I0528 22:25:06.708372 1546162 docker.go:217] disabling cri-docker service (if available) ...
I0528 22:25:06.708438 1546162 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0528 22:25:06.724636 1546162 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0528 22:25:06.737641 1546162 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0528 22:25:06.831442 1546162 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0528 22:25:06.924316 1546162 docker.go:233] disabling docker service ...
I0528 22:25:06.924427 1546162 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0528 22:25:06.937913 1546162 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0528 22:25:06.950064 1546162 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0528 22:25:07.046458 1546162 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0528 22:25:07.133831 1546162 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0528 22:25:07.146048 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/crio/crio.sock
" | sudo tee /etc/crictl.yaml"
I0528 22:25:07.162493 1546162 crio.go:59] configure cri-o to use "registry.k8s.io/pause:3.9" pause image...
I0528 22:25:07.162582 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*pause_image = .*$|pause_image = "registry.k8s.io/pause:3.9"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.173269 1546162 crio.go:70] configuring cri-o to use "cgroupfs" as cgroup driver...
I0528 22:25:07.173387 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|^.*cgroup_manager = .*$|cgroup_manager = "cgroupfs"|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.183533 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i '/conmon_cgroup = .*/d' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.195859 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i '/cgroup_manager = .*/a conmon_cgroup = "pod"' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.206858 1546162 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0528 22:25:07.217916 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *"net.ipv4.ip_unprivileged_port_start=.*"/d' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.231541 1546162 ssh_runner.go:195] Run: sh -c "sudo grep -q "^ *default_sysctls" /etc/crio/crio.conf.d/02-crio.conf || sudo sed -i '/conmon_cgroup = .*/a default_sysctls = \[\n\]' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.242577 1546162 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^default_sysctls *= *\[|&\n "net.ipv4.ip_unprivileged_port_start=0",|' /etc/crio/crio.conf.d/02-crio.conf"
I0528 22:25:07.256166 1546162 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0528 22:25:07.267876 1546162 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0528 22:25:07.277133 1546162 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:25:07.359288 1546162 ssh_runner.go:195] Run: sudo systemctl restart crio
I0528 22:25:07.508652 1546162 start.go:541] Will wait 60s for socket path /var/run/crio/crio.sock
I0528 22:25:07.508776 1546162 ssh_runner.go:195] Run: stat /var/run/crio/crio.sock
I0528 22:25:07.514583 1546162 start.go:562] Will wait 60s for crictl version
I0528 22:25:07.514677 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:25:07.519671 1546162 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0528 22:25:07.563535 1546162 start.go:578] Version: 0.1.0
RuntimeName: cri-o
RuntimeVersion: 1.24.6
RuntimeApiVersion: v1
I0528 22:25:07.563646 1546162 ssh_runner.go:195] Run: crio --version
I0528 22:25:07.620626 1546162 ssh_runner.go:195] Run: crio --version
I0528 22:25:07.672109 1546162 out.go:177] * Preparing Kubernetes v1.30.1 on CRI-O 1.24.6 ...
I0528 22:25:07.674105 1546162 cli_runner.go:164] Run: docker network inspect no-preload-264173 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0528 22:25:07.688481 1546162 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0528 22:25:07.692459 1546162 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 22:25:07.703331 1546162 kubeadm.go:877] updating cluster {Name:no-preload-264173 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:no-preload-264173 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikub
e-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0528 22:25:07.703488 1546162 preload.go:132] Checking if preload exists for k8s version v1.30.1 and runtime crio
I0528 22:25:07.703536 1546162 ssh_runner.go:195] Run: sudo crictl images --output json
I0528 22:25:07.751495 1546162 crio.go:514] all images are preloaded for cri-o runtime.
I0528 22:25:07.751521 1546162 cache_images.go:84] Images are preloaded, skipping loading
I0528 22:25:07.751530 1546162 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.30.1 crio true true} ...
I0528 22:25:07.751640 1546162 kubeadm.go:940] kubelet [Unit]
Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.30.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroups-per-qos=false --config=/var/lib/kubelet/config.yaml --enforce-node-allocatable= --hostname-override=no-preload-264173 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.30.1 ClusterName:no-preload-264173 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0528 22:25:07.751735 1546162 ssh_runner.go:195] Run: crio config
I0528 22:25:07.811781 1546162 cni.go:84] Creating CNI manager for ""
I0528 22:25:07.811803 1546162 cni.go:143] "docker" driver + "crio" runtime found, recommending kindnet
I0528 22:25:07.811823 1546162 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0528 22:25:07.811849 1546162 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.30.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-264173 NodeName:no-preload-264173 DNSDomain:cluster.local CRISocket:/var/run/crio/crio.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:/etc
/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/crio/crio.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0528 22:25:07.812016 1546162 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/crio/crio.sock
name: "no-preload-264173"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.30.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/crio/crio.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0528 22:25:07.812080 1546162 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.30.1
I0528 22:25:07.822063 1546162 binaries.go:44] Found k8s binaries, skipping transfer
I0528 22:25:07.822157 1546162 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0528 22:25:07.831349 1546162 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (367 bytes)
I0528 22:25:07.851622 1546162 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0528 22:25:07.871069 1546162 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2155 bytes)
I0528 22:25:07.889985 1546162 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0528 22:25:07.893859 1546162 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0528 22:25:07.905704 1546162 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:25:08.004215 1546162 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0528 22:25:08.021541 1546162 certs.go:68] Setting up /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173 for IP: 192.168.76.2
I0528 22:25:08.021561 1546162 certs.go:194] generating shared ca certs ...
I0528 22:25:08.021577 1546162 certs.go:226] acquiring lock for ca certs: {Name:mk3b01431a293453662fa80a6161920f23c6c736 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:25:08.021720 1546162 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.key
I0528 22:25:08.021769 1546162 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.key
I0528 22:25:08.021777 1546162 certs.go:256] generating profile certs ...
I0528 22:25:08.021864 1546162 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/client.key
I0528 22:25:08.021927 1546162 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/apiserver.key.b07e317e
I0528 22:25:08.021970 1546162 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/proxy-client.key
I0528 22:25:08.022116 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197.pem (1338 bytes)
W0528 22:25:08.022147 1546162 certs.go:480] ignoring /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197_empty.pem, impossibly tiny 0 bytes
I0528 22:25:08.022155 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca-key.pem (1679 bytes)
I0528 22:25:08.022182 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/ca.pem (1082 bytes)
I0528 22:25:08.022208 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/cert.pem (1123 bytes)
I0528 22:25:08.022232 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/key.pem (1675 bytes)
I0528 22:25:08.022277 1546162 certs.go:484] found cert: /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem (1708 bytes)
I0528 22:25:08.022999 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0528 22:25:08.075978 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1675 bytes)
I0528 22:25:08.123437 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0528 22:25:08.160622 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0528 22:25:08.198772 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0528 22:25:08.229741 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0528 22:25:08.257753 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0528 22:25:08.283933 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/profiles/no-preload-264173/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0528 22:25:08.312516 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0528 22:25:08.337680 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/certs/1355197.pem --> /usr/share/ca-certificates/1355197.pem (1338 bytes)
I0528 22:25:08.363237 1546162 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18966-1349783/.minikube/files/etc/ssl/certs/13551972.pem --> /usr/share/ca-certificates/13551972.pem (1708 bytes)
I0528 22:25:08.396228 1546162 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0528 22:25:08.415342 1546162 ssh_runner.go:195] Run: openssl version
I0528 22:25:08.422720 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0528 22:25:08.433235 1546162 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0528 22:25:08.436946 1546162 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 May 28 21:31 /usr/share/ca-certificates/minikubeCA.pem
I0528 22:25:08.437066 1546162 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0528 22:25:08.445114 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0528 22:25:08.454693 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1355197.pem && ln -fs /usr/share/ca-certificates/1355197.pem /etc/ssl/certs/1355197.pem"
I0528 22:25:08.465403 1546162 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1355197.pem
I0528 22:25:08.469049 1546162 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 May 28 21:42 /usr/share/ca-certificates/1355197.pem
I0528 22:25:08.469116 1546162 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1355197.pem
I0528 22:25:08.477653 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1355197.pem /etc/ssl/certs/51391683.0"
I0528 22:25:08.487180 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/13551972.pem && ln -fs /usr/share/ca-certificates/13551972.pem /etc/ssl/certs/13551972.pem"
I0528 22:25:08.497016 1546162 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/13551972.pem
I0528 22:25:08.500661 1546162 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 May 28 21:42 /usr/share/ca-certificates/13551972.pem
I0528 22:25:08.500723 1546162 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/13551972.pem
I0528 22:25:08.507991 1546162 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/13551972.pem /etc/ssl/certs/3ec20f2e.0"
I0528 22:25:08.517692 1546162 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0528 22:25:08.521329 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0528 22:25:08.528660 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0528 22:25:08.535897 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0528 22:25:08.542992 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0528 22:25:08.550139 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0528 22:25:08.557349 1546162 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0528 22:25:08.564735 1546162 kubeadm.go:391] StartCluster: {Name:no-preload-264173 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1716228441-18934@sha256:628b3f20803bc9c4302fd048087dd36cf2ff5dc9ab0ded395ec3288e2f1d0862 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:no-preload-264173 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:crio CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:crio ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-h
ost Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0528 22:25:08.564830 1546162 cri.go:54] listing CRI containers in root : {State:paused Name: Namespaces:[kube-system]}
I0528 22:25:08.564904 1546162 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0528 22:25:08.620481 1546162 cri.go:89] found id: ""
I0528 22:25:08.620600 1546162 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0528 22:25:08.630312 1546162 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0528 22:25:08.630383 1546162 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0528 22:25:08.630400 1546162 kubeadm.go:587] restartPrimaryControlPlane start ...
I0528 22:25:08.630469 1546162 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0528 22:25:08.640220 1546162 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0528 22:25:08.640826 1546162 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-264173" does not appear in /home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:25:08.641086 1546162 kubeconfig.go:62] /home/jenkins/minikube-integration/18966-1349783/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-264173" cluster setting kubeconfig missing "no-preload-264173" context setting]
I0528 22:25:08.641513 1546162 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-1349783/kubeconfig: {Name:mkaf5e1534f034576a412c2bb12acf3530c82fbc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:25:08.643056 1546162 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0528 22:25:08.655555 1546162 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.76.2
I0528 22:25:08.655591 1546162 kubeadm.go:591] duration metric: took 25.174373ms to restartPrimaryControlPlane
I0528 22:25:08.655600 1546162 kubeadm.go:393] duration metric: took 90.877005ms to StartCluster
I0528 22:25:08.655619 1546162 settings.go:142] acquiring lock: {Name:mk3ead4661b05edfaa64061283a93c6a76969cd7 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:25:08.655684 1546162 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18966-1349783/kubeconfig
I0528 22:25:08.656588 1546162 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18966-1349783/kubeconfig: {Name:mkaf5e1534f034576a412c2bb12acf3530c82fbc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0528 22:25:08.656788 1546162 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:crio ControlPlane:true Worker:true}
I0528 22:25:08.660717 1546162 out.go:177] * Verifying Kubernetes components...
I0528 22:25:08.657077 1546162 config.go:182] Loaded profile config "no-preload-264173": Driver=docker, ContainerRuntime=crio, KubernetesVersion=v1.30.1
I0528 22:25:08.657088 1546162 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0528 22:25:08.660939 1546162 addons.go:69] Setting storage-provisioner=true in profile "no-preload-264173"
I0528 22:25:08.663404 1546162 addons.go:234] Setting addon storage-provisioner=true in "no-preload-264173"
W0528 22:25:08.663434 1546162 addons.go:243] addon storage-provisioner should already be in state true
I0528 22:25:08.660947 1546162 addons.go:69] Setting dashboard=true in profile "no-preload-264173"
I0528 22:25:08.663478 1546162 host.go:66] Checking if "no-preload-264173" exists ...
I0528 22:25:08.663510 1546162 addons.go:234] Setting addon dashboard=true in "no-preload-264173"
W0528 22:25:08.663613 1546162 addons.go:243] addon dashboard should already be in state true
I0528 22:25:08.663652 1546162 host.go:66] Checking if "no-preload-264173" exists ...
I0528 22:25:08.664065 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:08.664066 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:08.663330 1546162 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0528 22:25:08.660953 1546162 addons.go:69] Setting default-storageclass=true in profile "no-preload-264173"
I0528 22:25:08.664733 1546162 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-264173"
I0528 22:25:08.660957 1546162 addons.go:69] Setting metrics-server=true in profile "no-preload-264173"
I0528 22:25:08.664847 1546162 addons.go:234] Setting addon metrics-server=true in "no-preload-264173"
W0528 22:25:08.664871 1546162 addons.go:243] addon metrics-server should already be in state true
I0528 22:25:08.664916 1546162 host.go:66] Checking if "no-preload-264173" exists ...
I0528 22:25:08.665015 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:08.665522 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:08.715457 1546162 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0528 22:25:08.717759 1546162 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0528 22:25:08.725329 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0528 22:25:08.726222 1546162 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0528 22:25:08.726289 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:08.726876 1546162 addons.go:234] Setting addon default-storageclass=true in "no-preload-264173"
W0528 22:25:08.726889 1546162 addons.go:243] addon default-storageclass should already be in state true
I0528 22:25:08.726914 1546162 host.go:66] Checking if "no-preload-264173" exists ...
I0528 22:25:08.727313 1546162 cli_runner.go:164] Run: docker container inspect no-preload-264173 --format={{.State.Status}}
I0528 22:25:08.740002 1546162 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0528 22:25:08.742600 1546162 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0528 22:25:08.742625 1546162 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0528 22:25:08.742696 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:08.765130 1546162 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0528 22:25:05.947874 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:08.447084 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:08.770460 1546162 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:25:08.770483 1546162 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0528 22:25:08.770552 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:08.773880 1546162 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0528 22:25:08.773900 1546162 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0528 22:25:08.773972 1546162 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-264173
I0528 22:25:08.792487 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:08.814209 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:08.857464 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:08.859844 1546162 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34594 SSHKeyPath:/home/jenkins/minikube-integration/18966-1349783/.minikube/machines/no-preload-264173/id_rsa Username:docker}
I0528 22:25:09.072358 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:25:09.074707 1546162 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0528 22:25:09.074779 1546162 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0528 22:25:09.082521 1546162 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0528 22:25:09.107035 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0528 22:25:09.146834 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0528 22:25:09.146911 1546162 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0528 22:25:09.162374 1546162 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0528 22:25:09.162398 1546162 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0528 22:25:09.191483 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0528 22:25:09.191512 1546162 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0528 22:25:09.256341 1546162 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:25:09.256370 1546162 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0528 22:25:09.261003 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0528 22:25:09.261077 1546162 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0528 22:25:09.321530 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:25:09.342437 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0528 22:25:09.342509 1546162 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0528 22:25:09.375328 1546162 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.375413 1546162 retry.go:31] will retry after 295.051083ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storage-provisioner.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.375479 1546162 node_ready.go:35] waiting up to 6m0s for node "no-preload-264173" to be "Ready" ...
W0528 22:25:09.422268 1546162 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.422364 1546162 retry.go:31] will retry after 142.820685ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.427608 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0528 22:25:09.427683 1546162 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0528 22:25:09.505069 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0528 22:25:09.505147 1546162 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
W0528 22:25:09.523264 1546162 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.523357 1546162 retry.go:31] will retry after 276.038849ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
error validating "/etc/kubernetes/addons/metrics-apiservice.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-deployment.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-rbac.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
error validating "/etc/kubernetes/addons/metrics-server-service.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0528 22:25:09.537624 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0528 22:25:09.537704 1546162 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0528 22:25:09.566228 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0528 22:25:09.584457 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0528 22:25:09.584538 1546162 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0528 22:25:09.627557 1546162 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:25:09.627637 1546162 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0528 22:25:09.670839 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0528 22:25:09.684946 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0528 22:25:09.800323 1546162 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0528 22:25:10.946031 1540905 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:11.951732 1540905 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:11.951759 1540905 pod_ready.go:81] duration metric: took 1m7.512242141s for pod "kube-controller-manager-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.951771 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-8jz6w" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.974855 1540905 pod_ready.go:92] pod "kube-proxy-8jz6w" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:11.974883 1540905 pod_ready.go:81] duration metric: took 23.104329ms for pod "kube-proxy-8jz6w" in "kube-system" namespace to be "Ready" ...
I0528 22:25:11.974920 1540905 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:13.982194 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:13.672600 1546162 node_ready.go:49] node "no-preload-264173" has status "Ready":"True"
I0528 22:25:13.672623 1546162 node_ready.go:38] duration metric: took 4.297112165s for node "no-preload-264173" to be "Ready" ...
I0528 22:25:13.672632 1546162 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0528 22:25:13.805734 1546162 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (4.239418709s)
I0528 22:25:13.917810 1546162 pod_ready.go:78] waiting up to 6m0s for pod "coredns-7db6d8ff4d-4l429" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.036539 1546162 pod_ready.go:92] pod "coredns-7db6d8ff4d-4l429" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:14.036611 1546162 pod_ready.go:81] duration metric: took 118.700904ms for pod "coredns-7db6d8ff4d-4l429" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.036639 1546162 pod_ready.go:78] waiting up to 6m0s for pod "etcd-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.116019 1546162 pod_ready.go:92] pod "etcd-no-preload-264173" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:14.116089 1546162 pod_ready.go:81] duration metric: took 79.42835ms for pod "etcd-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.116120 1546162 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.215691 1546162 pod_ready.go:92] pod "kube-apiserver-no-preload-264173" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:14.215761 1546162 pod_ready.go:81] duration metric: took 99.619809ms for pod "kube-apiserver-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.215787 1546162 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.288368 1546162 pod_ready.go:92] pod "kube-controller-manager-no-preload-264173" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:14.288394 1546162 pod_ready.go:81] duration metric: took 72.587973ms for pod "kube-controller-manager-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.288443 1546162 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-clscp" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.352894 1546162 pod_ready.go:92] pod "kube-proxy-clscp" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:14.352919 1546162 pod_ready.go:81] duration metric: took 64.462688ms for pod "kube-proxy-clscp" in "kube-system" namespace to be "Ready" ...
I0528 22:25:14.352932 1546162 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:15.197931 1546162 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (5.527004285s)
I0528 22:25:15.542957 1546162 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (5.857915893s)
I0528 22:25:15.545354 1546162 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-264173 addons enable metrics-server
I0528 22:25:15.543240 1546162 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.30.1/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (5.742832998s)
I0528 22:25:15.545472 1546162 addons.go:475] Verifying addon metrics-server=true in "no-preload-264173"
I0528 22:25:15.548025 1546162 out.go:177] * Enabled addons: default-storageclass, storage-provisioner, metrics-server, dashboard
I0528 22:25:15.550702 1546162 addons.go:510] duration metric: took 6.89360502s for enable addons: enabled=[default-storageclass storage-provisioner metrics-server dashboard]
I0528 22:25:16.480792 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:18.482060 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:16.359570 1546162 pod_ready.go:102] pod "kube-scheduler-no-preload-264173" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:18.859476 1546162 pod_ready.go:102] pod "kube-scheduler-no-preload-264173" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:20.482153 1540905 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:20.982171 1540905 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:20.982200 1540905 pod_ready.go:81] duration metric: took 9.007259537s for pod "kube-scheduler-old-k8s-version-137556" in "kube-system" namespace to be "Ready" ...
I0528 22:25:20.982213 1540905 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace to be "Ready" ...
I0528 22:25:22.989102 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:21.360330 1546162 pod_ready.go:102] pod "kube-scheduler-no-preload-264173" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:22.359497 1546162 pod_ready.go:92] pod "kube-scheduler-no-preload-264173" in "kube-system" namespace has status "Ready":"True"
I0528 22:25:22.359522 1546162 pod_ready.go:81] duration metric: took 8.006582294s for pod "kube-scheduler-no-preload-264173" in "kube-system" namespace to be "Ready" ...
I0528 22:25:22.359534 1546162 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace to be "Ready" ...
I0528 22:25:24.366683 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:24.989608 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:27.489518 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:26.865186 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:28.866064 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:30.868441 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:29.988533 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:31.991247 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:34.488194 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:33.366119 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:35.382357 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:36.495271 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:38.990348 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:37.870607 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:40.365901 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:41.495366 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:43.521873 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:42.372928 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:44.865398 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:45.988876 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:47.989265 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:46.866232 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:49.366258 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:50.488844 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:52.488950 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:54.492792 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:51.865030 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:53.867071 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:56.989265 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:59.488620 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:56.365998 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:25:58.865699 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:01.488762 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:03.988913 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:01.369144 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:03.865131 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:05.865426 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:06.488936 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:08.988177 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:07.865511 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:09.866721 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:10.988907 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:12.989263 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:12.366014 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:14.865384 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:15.489223 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:17.988232 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:16.865855 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:19.365485 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:19.988565 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:22.488224 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:24.489557 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:21.365826 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:23.865604 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:26.988463 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:28.990525 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:26.365967 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:28.864896 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:30.866151 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:31.487940 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:33.488677 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:33.366169 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:35.368478 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:35.988481 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:37.988691 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:37.866204 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:40.366949 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:39.989047 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:42.497971 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:42.865337 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:44.866077 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:44.988931 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:46.988984 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:49.488471 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:46.866308 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:48.866342 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:51.988480 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:54.487836 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:51.365534 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:53.866767 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:56.488805 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:58.988626 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:56.365798 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:26:58.369574 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:00.865211 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:00.988931 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:02.989274 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:02.865620 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:05.366191 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:05.487582 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:07.487769 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:09.488409 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:07.865654 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:09.866153 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:11.987932 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:13.993302 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:12.366125 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:14.366164 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:16.488026 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:18.988722 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:16.865446 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:18.865584 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:20.865851 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:20.988794 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:23.488388 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:22.866052 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:25.366082 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:25.489230 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:27.988524 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:27.366540 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:29.865189 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:29.989139 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:31.989250 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:34.488910 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:31.866146 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:34.366215 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:36.988638 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:39.488046 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:36.865667 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:38.865699 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:40.866373 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:41.488536 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:43.488804 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:43.366348 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:45.865183 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:45.988591 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:47.989534 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:47.865467 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:50.366506 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:50.488161 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:52.989006 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:52.866416 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:55.366488 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:55.015860 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:57.488310 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:59.488865 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:57.864930 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:27:59.864963 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:01.493382 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:03.988248 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:01.866864 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:04.365807 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:05.989110 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:08.492906 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:06.366042 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:08.368216 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:10.865738 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:10.988521 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:12.989371 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:12.866255 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:15.365881 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:15.488132 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:17.988756 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:17.366823 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:19.865098 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:20.488836 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:22.988169 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:21.865814 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:23.866249 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:25.492744 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:27.987967 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:26.365468 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:28.365944 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:30.366205 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:29.988864 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:31.989186 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:34.488439 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:32.865797 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:34.866904 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:36.488805 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:38.988909 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:37.365283 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:39.365967 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:41.488703 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:43.988369 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:41.865744 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:44.365783 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:45.988453 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:47.989135 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:46.865235 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:48.866110 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:49.989359 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:52.487694 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:54.488326 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:51.365618 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:53.366723 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:55.866483 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:56.489000 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:58.989339 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:28:58.365462 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:00.366117 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:01.488277 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:03.988703 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:02.866160 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:05.365989 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:05.989324 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:07.990979 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:07.865756 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:10.366219 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:10.489138 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:12.989242 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:12.865709 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:15.365199 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:15.488807 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:17.989551 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:17.366206 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:19.367278 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:20.489092 1540905 pod_ready.go:102] pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:20.988019 1540905 pod_ready.go:81] duration metric: took 4m0.005793638s for pod "metrics-server-9975d5f86-h5vcp" in "kube-system" namespace to be "Ready" ...
E0528 22:29:20.988044 1540905 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0528 22:29:20.988093 1540905 pod_ready.go:38] duration metric: took 5m17.798780687s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0528 22:29:20.988114 1540905 api_server.go:52] waiting for apiserver process to appear ...
I0528 22:29:20.988153 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:20.988223 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:21.032451 1540905 cri.go:89] found id: "9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:21.032474 1540905 cri.go:89] found id: ""
I0528 22:29:21.032489 1540905 logs.go:276] 1 containers: [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3]
I0528 22:29:21.032561 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.036290 1540905 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:21.036360 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:21.078257 1540905 cri.go:89] found id: "c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:21.078279 1540905 cri.go:89] found id: ""
I0528 22:29:21.078288 1540905 logs.go:276] 1 containers: [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8]
I0528 22:29:21.078346 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.081979 1540905 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:21.082082 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:21.128720 1540905 cri.go:89] found id: "cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:21.128744 1540905 cri.go:89] found id: ""
I0528 22:29:21.128752 1540905 logs.go:276] 1 containers: [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a]
I0528 22:29:21.128828 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.132418 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:21.132487 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:21.182402 1540905 cri.go:89] found id: "b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:21.182464 1540905 cri.go:89] found id: ""
I0528 22:29:21.182485 1540905 logs.go:276] 1 containers: [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105]
I0528 22:29:21.182568 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.186351 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:21.186428 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:21.236128 1540905 cri.go:89] found id: "7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:21.236152 1540905 cri.go:89] found id: ""
I0528 22:29:21.236160 1540905 logs.go:276] 1 containers: [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b]
I0528 22:29:21.236250 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.239897 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:21.239973 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:21.276341 1540905 cri.go:89] found id: "41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:21.276373 1540905 cri.go:89] found id: ""
I0528 22:29:21.276382 1540905 logs.go:276] 1 containers: [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29]
I0528 22:29:21.276440 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.280074 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:21.280157 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:21.329000 1540905 cri.go:89] found id: "dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:21.329022 1540905 cri.go:89] found id: ""
I0528 22:29:21.329030 1540905 logs.go:276] 1 containers: [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5]
I0528 22:29:21.329099 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.333459 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:21.333533 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:21.376968 1540905 cri.go:89] found id: "bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:21.377039 1540905 cri.go:89] found id: ""
I0528 22:29:21.377060 1540905 logs.go:276] 1 containers: [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527]
I0528 22:29:21.377147 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.381185 1540905 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:21.381306 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:21.421871 1540905 cri.go:89] found id: "4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:21.421943 1540905 cri.go:89] found id: "05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:21.421962 1540905 cri.go:89] found id: ""
I0528 22:29:21.421984 1540905 logs.go:276] 2 containers: [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998]
I0528 22:29:21.422120 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.425791 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:21.429266 1540905 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:21.429294 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:21.449313 1540905 logs.go:123] Gathering logs for kube-apiserver [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3] ...
I0528 22:29:21.449351 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:21.519486 1540905 logs.go:123] Gathering logs for etcd [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8] ...
I0528 22:29:21.519521 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:21.569520 1540905 logs.go:123] Gathering logs for coredns [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a] ...
I0528 22:29:21.569549 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:21.613509 1540905 logs.go:123] Gathering logs for kube-controller-manager [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29] ...
I0528 22:29:21.613536 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:21.686097 1540905 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:21.686137 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:21.831954 1540905 logs.go:123] Gathering logs for storage-provisioner [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33] ...
I0528 22:29:21.831985 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:21.881237 1540905 logs.go:123] Gathering logs for kube-scheduler [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105] ...
I0528 22:29:21.881266 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:21.927283 1540905 logs.go:123] Gathering logs for kindnet [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5] ...
I0528 22:29:21.927310 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:21.967506 1540905 logs.go:123] Gathering logs for storage-provisioner [05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998] ...
I0528 22:29:21.967537 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:22.013255 1540905 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:22.013284 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:22.069448 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.186601 738 reflector.go:138] object-"kube-system"/"kindnet-token-ztnvd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ztnvd" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.069707 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187153 738 reflector.go:138] object-"kube-system"/"kube-proxy-token-cs2pc": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-cs2pc" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.069920 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187714 738 reflector.go:138] object-"default"/"default-token-k8l4j": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-k8l4j" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070157 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.194216 738 reflector.go:138] object-"kube-system"/"storage-provisioner-token-cswbw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-cswbw" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070359 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199146 738 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070571 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199216 738 reflector.go:138] object-"kube-system"/"coredns-token-q7qdg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-q7qdg" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.070775 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199262 738 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.075454 1540905 logs.go:138] Found kubelet problem: May 28 22:24:04 old-k8s-version-137556 kubelet[738]: E0528 22:24:04.243616 738 reflector.go:138] object-"kube-system"/"metrics-server-token-42d6m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-42d6m" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:22.079777 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.119557 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.079965 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.781597 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.082067 1540905 logs.go:138] Found kubelet problem: May 28 22:24:20 old-k8s-version-137556 kubelet[738]: E0528 22:24:20.203798 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.083824 1540905 logs.go:138] Found kubelet problem: May 28 22:24:33 old-k8s-version-137556 kubelet[738]: E0528 22:24:33.708202 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.084544 1540905 logs.go:138] Found kubelet problem: May 28 22:24:38 old-k8s-version-137556 kubelet[738]: E0528 22:24:38.049446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.084872 1540905 logs.go:138] Found kubelet problem: May 28 22:24:39 old-k8s-version-137556 kubelet[738]: E0528 22:24:39.048542 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.085200 1540905 logs.go:138] Found kubelet problem: May 28 22:24:46 old-k8s-version-137556 kubelet[738]: E0528 22:24:46.690430 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.087272 1540905 logs.go:138] Found kubelet problem: May 28 22:24:48 old-k8s-version-137556 kubelet[738]: E0528 22:24:48.729167 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.087463 1540905 logs.go:138] Found kubelet problem: May 28 22:25:00 old-k8s-version-137556 kubelet[738]: E0528 22:25:00.705605 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.088052 1540905 logs.go:138] Found kubelet problem: May 28 22:25:02 old-k8s-version-137556 kubelet[738]: E0528 22:25:02.084189 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.088378 1540905 logs.go:138] Found kubelet problem: May 28 22:25:06 old-k8s-version-137556 kubelet[738]: E0528 22:25:06.690524 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.088562 1540905 logs.go:138] Found kubelet problem: May 28 22:25:13 old-k8s-version-137556 kubelet[738]: E0528 22:25:13.705095 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.088886 1540905 logs.go:138] Found kubelet problem: May 28 22:25:19 old-k8s-version-137556 kubelet[738]: E0528 22:25:19.704584 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.089072 1540905 logs.go:138] Found kubelet problem: May 28 22:25:24 old-k8s-version-137556 kubelet[738]: E0528 22:25:24.704685 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.089655 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.141913 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.091744 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.718176 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.092073 1540905 logs.go:138] Found kubelet problem: May 28 22:25:36 old-k8s-version-137556 kubelet[738]: E0528 22:25:36.690449 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.092399 1540905 logs.go:138] Found kubelet problem: May 28 22:25:47 old-k8s-version-137556 kubelet[738]: E0528 22:25:47.705446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.092585 1540905 logs.go:138] Found kubelet problem: May 28 22:25:49 old-k8s-version-137556 kubelet[738]: E0528 22:25:49.705980 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.092918 1540905 logs.go:138] Found kubelet problem: May 28 22:26:00 old-k8s-version-137556 kubelet[738]: E0528 22:26:00.704155 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.093104 1540905 logs.go:138] Found kubelet problem: May 28 22:26:01 old-k8s-version-137556 kubelet[738]: E0528 22:26:01.704811 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.093692 1540905 logs.go:138] Found kubelet problem: May 28 22:26:15 old-k8s-version-137556 kubelet[738]: E0528 22:26:15.207095 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094030 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.690320 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094212 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.704608 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.094541 1540905 logs.go:138] Found kubelet problem: May 28 22:26:28 old-k8s-version-137556 kubelet[738]: E0528 22:26:28.704143 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.094726 1540905 logs.go:138] Found kubelet problem: May 28 22:26:29 old-k8s-version-137556 kubelet[738]: E0528 22:26:29.704616 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.095052 1540905 logs.go:138] Found kubelet problem: May 28 22:26:39 old-k8s-version-137556 kubelet[738]: E0528 22:26:39.704876 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.095237 1540905 logs.go:138] Found kubelet problem: May 28 22:26:43 old-k8s-version-137556 kubelet[738]: E0528 22:26:43.705110 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.096084 1540905 logs.go:138] Found kubelet problem: May 28 22:26:53 old-k8s-version-137556 kubelet[738]: E0528 22:26:53.704310 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.096269 1540905 logs.go:138] Found kubelet problem: May 28 22:26:54 old-k8s-version-137556 kubelet[738]: E0528 22:26:54.704655 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.096595 1540905 logs.go:138] Found kubelet problem: May 28 22:27:06 old-k8s-version-137556 kubelet[738]: E0528 22:27:06.704128 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.098660 1540905 logs.go:138] Found kubelet problem: May 28 22:27:09 old-k8s-version-137556 kubelet[738]: E0528 22:27:09.715928 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:22.098988 1540905 logs.go:138] Found kubelet problem: May 28 22:27:17 old-k8s-version-137556 kubelet[738]: E0528 22:27:17.704167 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.099172 1540905 logs.go:138] Found kubelet problem: May 28 22:27:21 old-k8s-version-137556 kubelet[738]: E0528 22:27:21.704885 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.099502 1540905 logs.go:138] Found kubelet problem: May 28 22:27:30 old-k8s-version-137556 kubelet[738]: E0528 22:27:30.704161 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.099687 1540905 logs.go:138] Found kubelet problem: May 28 22:27:33 old-k8s-version-137556 kubelet[738]: E0528 22:27:33.704586 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.100291 1540905 logs.go:138] Found kubelet problem: May 28 22:27:44 old-k8s-version-137556 kubelet[738]: E0528 22:27:44.340886 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.100622 1540905 logs.go:138] Found kubelet problem: May 28 22:27:46 old-k8s-version-137556 kubelet[738]: E0528 22:27:46.690347 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.100807 1540905 logs.go:138] Found kubelet problem: May 28 22:27:48 old-k8s-version-137556 kubelet[738]: E0528 22:27:48.704625 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101135 1540905 logs.go:138] Found kubelet problem: May 28 22:27:58 old-k8s-version-137556 kubelet[738]: E0528 22:27:58.704805 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.101320 1540905 logs.go:138] Found kubelet problem: May 28 22:28:00 old-k8s-version-137556 kubelet[738]: E0528 22:28:00.704891 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101634 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705029 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.101831 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705742 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.102015 1540905 logs.go:138] Found kubelet problem: May 28 22:28:24 old-k8s-version-137556 kubelet[738]: E0528 22:28:24.704663 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.102358 1540905 logs.go:138] Found kubelet problem: May 28 22:28:26 old-k8s-version-137556 kubelet[738]: E0528 22:28:26.704162 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.102544 1540905 logs.go:138] Found kubelet problem: May 28 22:28:39 old-k8s-version-137556 kubelet[738]: E0528 22:28:39.704787 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.102873 1540905 logs.go:138] Found kubelet problem: May 28 22:28:40 old-k8s-version-137556 kubelet[738]: E0528 22:28:40.704255 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.103452 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.704205 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.103640 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.103825 1540905 logs.go:138] Found kubelet problem: May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.104151 1540905 logs.go:138] Found kubelet problem: May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.104340 1540905 logs.go:138] Found kubelet problem: May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.104665 1540905 logs.go:138] Found kubelet problem: May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
I0528 22:29:22.104674 1540905 logs.go:123] Gathering logs for kube-proxy [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b] ...
I0528 22:29:22.104688 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:22.141117 1540905 logs.go:123] Gathering logs for kubernetes-dashboard [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527] ...
I0528 22:29:22.141148 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:22.180279 1540905 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:22.180303 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:22.275290 1540905 logs.go:123] Gathering logs for container status ...
I0528 22:29:22.275341 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:22.320717 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:22.320742 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:22.320829 1540905 out.go:239] X Problems detected in kubelet:
W0528 22:29:22.320847 1540905 out.go:239] May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.320988 1540905 out.go:239] May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.321016 1540905 out.go:239] May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:22.321024 1540905 out.go:239] May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:22.321042 1540905 out.go:239] May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
I0528 22:29:22.321050 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:22.321057 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:21.865988 1546162 pod_ready.go:102] pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace has status "Ready":"False"
I0528 22:29:22.364912 1546162 pod_ready.go:81] duration metric: took 4m0.005364553s for pod "metrics-server-569cc877fc-62rwk" in "kube-system" namespace to be "Ready" ...
E0528 22:29:22.364941 1546162 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0528 22:29:22.364952 1546162 pod_ready.go:38] duration metric: took 4m8.692307712s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0528 22:29:22.364968 1546162 api_server.go:52] waiting for apiserver process to appear ...
I0528 22:29:22.364995 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:22.365056 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:22.411170 1546162 cri.go:89] found id: "dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2"
I0528 22:29:22.411193 1546162 cri.go:89] found id: ""
I0528 22:29:22.411200 1546162 logs.go:276] 1 containers: [dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2]
I0528 22:29:22.411259 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.414631 1546162 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:22.414705 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:22.456030 1546162 cri.go:89] found id: "aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8"
I0528 22:29:22.456050 1546162 cri.go:89] found id: ""
I0528 22:29:22.456058 1546162 logs.go:276] 1 containers: [aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8]
I0528 22:29:22.456116 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.459827 1546162 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:22.459948 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:22.499320 1546162 cri.go:89] found id: "61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78"
I0528 22:29:22.499353 1546162 cri.go:89] found id: ""
I0528 22:29:22.499362 1546162 logs.go:276] 1 containers: [61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78]
I0528 22:29:22.499457 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.503219 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:22.503338 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:22.550148 1546162 cri.go:89] found id: "737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b"
I0528 22:29:22.550210 1546162 cri.go:89] found id: ""
I0528 22:29:22.550231 1546162 logs.go:276] 1 containers: [737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b]
I0528 22:29:22.550313 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.554142 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:22.554215 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:22.591936 1546162 cri.go:89] found id: "e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66"
I0528 22:29:22.591957 1546162 cri.go:89] found id: ""
I0528 22:29:22.591965 1546162 logs.go:276] 1 containers: [e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66]
I0528 22:29:22.592020 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.595658 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:22.595747 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:22.635760 1546162 cri.go:89] found id: "92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734"
I0528 22:29:22.635780 1546162 cri.go:89] found id: ""
I0528 22:29:22.635787 1546162 logs.go:276] 1 containers: [92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734]
I0528 22:29:22.635842 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.639503 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:22.639604 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:22.674552 1546162 cri.go:89] found id: "c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a"
I0528 22:29:22.674623 1546162 cri.go:89] found id: ""
I0528 22:29:22.674643 1546162 logs.go:276] 1 containers: [c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a]
I0528 22:29:22.674722 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.678189 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:22.678254 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:22.713766 1546162 cri.go:89] found id: "639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a"
I0528 22:29:22.713785 1546162 cri.go:89] found id: ""
I0528 22:29:22.713793 1546162 logs.go:276] 1 containers: [639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a]
I0528 22:29:22.713870 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.717256 1546162 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:22.717327 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:22.757710 1546162 cri.go:89] found id: "db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9"
I0528 22:29:22.757734 1546162 cri.go:89] found id: "07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129"
I0528 22:29:22.757739 1546162 cri.go:89] found id: ""
I0528 22:29:22.757746 1546162 logs.go:276] 2 containers: [db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9 07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129]
I0528 22:29:22.757809 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.761367 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:22.764656 1546162 logs.go:123] Gathering logs for kubernetes-dashboard [639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a] ...
I0528 22:29:22.764727 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a"
I0528 22:29:22.813399 1546162 logs.go:123] Gathering logs for storage-provisioner [db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9] ...
I0528 22:29:22.813426 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9"
I0528 22:29:22.864851 1546162 logs.go:123] Gathering logs for storage-provisioner [07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129] ...
I0528 22:29:22.864879 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129"
I0528 22:29:22.905620 1546162 logs.go:123] Gathering logs for etcd [aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8] ...
I0528 22:29:22.905654 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8"
I0528 22:29:22.954101 1546162 logs.go:123] Gathering logs for coredns [61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78] ...
I0528 22:29:22.954132 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78"
I0528 22:29:22.994855 1546162 logs.go:123] Gathering logs for kindnet [c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a] ...
I0528 22:29:22.994899 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a"
I0528 22:29:23.048998 1546162 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:23.049028 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:23.129334 1546162 logs.go:123] Gathering logs for container status ...
I0528 22:29:23.129369 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:23.205581 1546162 logs.go:123] Gathering logs for kube-apiserver [dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2] ...
I0528 22:29:23.205611 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2"
I0528 22:29:23.258182 1546162 logs.go:123] Gathering logs for kube-proxy [e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66] ...
I0528 22:29:23.258218 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66"
I0528 22:29:23.301675 1546162 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:23.301704 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.30.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:23.447810 1546162 logs.go:123] Gathering logs for kube-scheduler [737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b] ...
I0528 22:29:23.447847 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b"
I0528 22:29:23.496843 1546162 logs.go:123] Gathering logs for kube-controller-manager [92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734] ...
I0528 22:29:23.496871 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734"
I0528 22:29:23.572239 1546162 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:23.572273 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:23.618623 1546162 logs.go:138] Found kubelet problem: May 28 22:25:28 no-preload-264173 kubelet[747]: W0528 22:25:28.245329 747 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
W0528 22:29:23.618856 1546162 logs.go:138] Found kubelet problem: May 28 22:25:28 no-preload-264173 kubelet[747]: E0528 22:25:28.245366 747 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
I0528 22:29:23.641158 1546162 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:23.641188 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:23.660927 1546162 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:23.660955 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:23.661013 1546162 out.go:239] X Problems detected in kubelet:
W0528 22:29:23.661022 1546162 out.go:239] May 28 22:25:28 no-preload-264173 kubelet[747]: W0528 22:25:28.245329 747 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
W0528 22:29:23.661029 1546162 out.go:239] May 28 22:25:28 no-preload-264173 kubelet[747]: E0528 22:25:28.245366 747 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
I0528 22:29:23.661042 1546162 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:23.661048 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:32.323043 1540905 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0528 22:29:32.335698 1540905 api_server.go:72] duration metric: took 5m48.896786162s to wait for apiserver process to appear ...
I0528 22:29:32.335723 1540905 api_server.go:88] waiting for apiserver healthz status ...
I0528 22:29:32.335762 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:32.335824 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:32.374464 1540905 cri.go:89] found id: "9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:32.374485 1540905 cri.go:89] found id: ""
I0528 22:29:32.374494 1540905 logs.go:276] 1 containers: [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3]
I0528 22:29:32.374556 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.378665 1540905 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:32.378735 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:32.428280 1540905 cri.go:89] found id: "c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:32.428310 1540905 cri.go:89] found id: ""
I0528 22:29:32.428319 1540905 logs.go:276] 1 containers: [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8]
I0528 22:29:32.428377 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.432088 1540905 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:32.432153 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:32.474292 1540905 cri.go:89] found id: "cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:32.474312 1540905 cri.go:89] found id: ""
I0528 22:29:32.474319 1540905 logs.go:276] 1 containers: [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a]
I0528 22:29:32.474376 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.477891 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:32.477960 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:32.521895 1540905 cri.go:89] found id: "b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:32.521917 1540905 cri.go:89] found id: ""
I0528 22:29:32.521925 1540905 logs.go:276] 1 containers: [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105]
I0528 22:29:32.521982 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.526385 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:32.526465 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:32.568221 1540905 cri.go:89] found id: "7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:32.568241 1540905 cri.go:89] found id: ""
I0528 22:29:32.568249 1540905 logs.go:276] 1 containers: [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b]
I0528 22:29:32.568304 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.571975 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:32.572079 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:32.609486 1540905 cri.go:89] found id: "41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:32.609510 1540905 cri.go:89] found id: ""
I0528 22:29:32.609517 1540905 logs.go:276] 1 containers: [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29]
I0528 22:29:32.609574 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.612947 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:32.613016 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:32.652655 1540905 cri.go:89] found id: "dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:32.652680 1540905 cri.go:89] found id: ""
I0528 22:29:32.652689 1540905 logs.go:276] 1 containers: [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5]
I0528 22:29:32.652748 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.656315 1540905 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:32.656442 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:32.693909 1540905 cri.go:89] found id: "bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:32.693941 1540905 cri.go:89] found id: ""
I0528 22:29:32.693949 1540905 logs.go:276] 1 containers: [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527]
I0528 22:29:32.694072 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.697734 1540905 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:32.697815 1540905 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:32.736027 1540905 cri.go:89] found id: "4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:32.736053 1540905 cri.go:89] found id: "05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:32.736058 1540905 cri.go:89] found id: ""
I0528 22:29:32.736065 1540905 logs.go:276] 2 containers: [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998]
I0528 22:29:32.736150 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.742328 1540905 ssh_runner.go:195] Run: which crictl
I0528 22:29:32.746335 1540905 logs.go:123] Gathering logs for storage-provisioner [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33] ...
I0528 22:29:32.746363 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33"
I0528 22:29:32.783928 1540905 logs.go:123] Gathering logs for storage-provisioner [05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998] ...
I0528 22:29:32.783955 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998"
I0528 22:29:32.821570 1540905 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:32.821598 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:32.907550 1540905 logs.go:123] Gathering logs for kubernetes-dashboard [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527] ...
I0528 22:29:32.907629 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527"
I0528 22:29:32.952689 1540905 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:32.952720 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:32.971568 1540905 logs.go:123] Gathering logs for etcd [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8] ...
I0528 22:29:32.971598 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8"
I0528 22:29:33.039002 1540905 logs.go:123] Gathering logs for kube-scheduler [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105] ...
I0528 22:29:33.039034 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105"
I0528 22:29:33.090654 1540905 logs.go:123] Gathering logs for kube-proxy [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b] ...
I0528 22:29:33.090687 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b"
I0528 22:29:33.136556 1540905 logs.go:123] Gathering logs for kindnet [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5] ...
I0528 22:29:33.136585 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5"
I0528 22:29:33.192155 1540905 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:33.192228 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:33.246432 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.186601 738 reflector.go:138] object-"kube-system"/"kindnet-token-ztnvd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-ztnvd" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.246672 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187153 738 reflector.go:138] object-"kube-system"/"kube-proxy-token-cs2pc": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-cs2pc" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.246887 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.187714 738 reflector.go:138] object-"default"/"default-token-k8l4j": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-k8l4j" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247119 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.194216 738 reflector.go:138] object-"kube-system"/"storage-provisioner-token-cswbw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-cswbw" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247320 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199146 738 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247555 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199216 738 reflector.go:138] object-"kube-system"/"coredns-token-q7qdg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-q7qdg" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.247760 1540905 logs.go:138] Found kubelet problem: May 28 22:24:03 old-k8s-version-137556 kubelet[738]: E0528 22:24:03.199262 738 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.252483 1540905 logs.go:138] Found kubelet problem: May 28 22:24:04 old-k8s-version-137556 kubelet[738]: E0528 22:24:04.243616 738 reflector.go:138] object-"kube-system"/"metrics-server-token-42d6m": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-42d6m" is forbidden: User "system:node:old-k8s-version-137556" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-137556' and this object
W0528 22:29:33.256917 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.119557 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.257113 1540905 logs.go:138] Found kubelet problem: May 28 22:24:06 old-k8s-version-137556 kubelet[738]: E0528 22:24:06.781597 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.259198 1540905 logs.go:138] Found kubelet problem: May 28 22:24:20 old-k8s-version-137556 kubelet[738]: E0528 22:24:20.203798 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.260974 1540905 logs.go:138] Found kubelet problem: May 28 22:24:33 old-k8s-version-137556 kubelet[738]: E0528 22:24:33.708202 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.261692 1540905 logs.go:138] Found kubelet problem: May 28 22:24:38 old-k8s-version-137556 kubelet[738]: E0528 22:24:38.049446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.262026 1540905 logs.go:138] Found kubelet problem: May 28 22:24:39 old-k8s-version-137556 kubelet[738]: E0528 22:24:39.048542 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.262358 1540905 logs.go:138] Found kubelet problem: May 28 22:24:46 old-k8s-version-137556 kubelet[738]: E0528 22:24:46.690430 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.265262 1540905 logs.go:138] Found kubelet problem: May 28 22:24:48 old-k8s-version-137556 kubelet[738]: E0528 22:24:48.729167 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.265472 1540905 logs.go:138] Found kubelet problem: May 28 22:25:00 old-k8s-version-137556 kubelet[738]: E0528 22:25:00.705605 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.266085 1540905 logs.go:138] Found kubelet problem: May 28 22:25:02 old-k8s-version-137556 kubelet[738]: E0528 22:25:02.084189 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.266416 1540905 logs.go:138] Found kubelet problem: May 28 22:25:06 old-k8s-version-137556 kubelet[738]: E0528 22:25:06.690524 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.266603 1540905 logs.go:138] Found kubelet problem: May 28 22:25:13 old-k8s-version-137556 kubelet[738]: E0528 22:25:13.705095 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.266933 1540905 logs.go:138] Found kubelet problem: May 28 22:25:19 old-k8s-version-137556 kubelet[738]: E0528 22:25:19.704584 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.267118 1540905 logs.go:138] Found kubelet problem: May 28 22:25:24 old-k8s-version-137556 kubelet[738]: E0528 22:25:24.704685 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.267735 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.141913 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.269944 1540905 logs.go:138] Found kubelet problem: May 28 22:25:35 old-k8s-version-137556 kubelet[738]: E0528 22:25:35.718176 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.270294 1540905 logs.go:138] Found kubelet problem: May 28 22:25:36 old-k8s-version-137556 kubelet[738]: E0528 22:25:36.690449 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.270624 1540905 logs.go:138] Found kubelet problem: May 28 22:25:47 old-k8s-version-137556 kubelet[738]: E0528 22:25:47.705446 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.271334 1540905 logs.go:138] Found kubelet problem: May 28 22:25:49 old-k8s-version-137556 kubelet[738]: E0528 22:25:49.705980 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.271686 1540905 logs.go:138] Found kubelet problem: May 28 22:26:00 old-k8s-version-137556 kubelet[738]: E0528 22:26:00.704155 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.271875 1540905 logs.go:138] Found kubelet problem: May 28 22:26:01 old-k8s-version-137556 kubelet[738]: E0528 22:26:01.704811 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.272461 1540905 logs.go:138] Found kubelet problem: May 28 22:26:15 old-k8s-version-137556 kubelet[738]: E0528 22:26:15.207095 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.272794 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.690320 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.272980 1540905 logs.go:138] Found kubelet problem: May 28 22:26:16 old-k8s-version-137556 kubelet[738]: E0528 22:26:16.704608 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.273349 1540905 logs.go:138] Found kubelet problem: May 28 22:26:28 old-k8s-version-137556 kubelet[738]: E0528 22:26:28.704143 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.273541 1540905 logs.go:138] Found kubelet problem: May 28 22:26:29 old-k8s-version-137556 kubelet[738]: E0528 22:26:29.704616 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.273874 1540905 logs.go:138] Found kubelet problem: May 28 22:26:39 old-k8s-version-137556 kubelet[738]: E0528 22:26:39.704876 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.274074 1540905 logs.go:138] Found kubelet problem: May 28 22:26:43 old-k8s-version-137556 kubelet[738]: E0528 22:26:43.705110 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.274980 1540905 logs.go:138] Found kubelet problem: May 28 22:26:53 old-k8s-version-137556 kubelet[738]: E0528 22:26:53.704310 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.275170 1540905 logs.go:138] Found kubelet problem: May 28 22:26:54 old-k8s-version-137556 kubelet[738]: E0528 22:26:54.704655 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.275499 1540905 logs.go:138] Found kubelet problem: May 28 22:27:06 old-k8s-version-137556 kubelet[738]: E0528 22:27:06.704128 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.277543 1540905 logs.go:138] Found kubelet problem: May 28 22:27:09 old-k8s-version-137556 kubelet[738]: E0528 22:27:09.715928 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = pinging container registry fake.domain: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain: no such host"
W0528 22:29:33.277870 1540905 logs.go:138] Found kubelet problem: May 28 22:27:17 old-k8s-version-137556 kubelet[738]: E0528 22:27:17.704167 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.278074 1540905 logs.go:138] Found kubelet problem: May 28 22:27:21 old-k8s-version-137556 kubelet[738]: E0528 22:27:21.704885 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.278407 1540905 logs.go:138] Found kubelet problem: May 28 22:27:30 old-k8s-version-137556 kubelet[738]: E0528 22:27:30.704161 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.278671 1540905 logs.go:138] Found kubelet problem: May 28 22:27:33 old-k8s-version-137556 kubelet[738]: E0528 22:27:33.704586 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.279301 1540905 logs.go:138] Found kubelet problem: May 28 22:27:44 old-k8s-version-137556 kubelet[738]: E0528 22:27:44.340886 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.279669 1540905 logs.go:138] Found kubelet problem: May 28 22:27:46 old-k8s-version-137556 kubelet[738]: E0528 22:27:46.690347 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.279882 1540905 logs.go:138] Found kubelet problem: May 28 22:27:48 old-k8s-version-137556 kubelet[738]: E0528 22:27:48.704625 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.280239 1540905 logs.go:138] Found kubelet problem: May 28 22:27:58 old-k8s-version-137556 kubelet[738]: E0528 22:27:58.704805 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.280461 1540905 logs.go:138] Found kubelet problem: May 28 22:28:00 old-k8s-version-137556 kubelet[738]: E0528 22:28:00.704891 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.280810 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705029 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.281039 1540905 logs.go:138] Found kubelet problem: May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705742 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.281253 1540905 logs.go:138] Found kubelet problem: May 28 22:28:24 old-k8s-version-137556 kubelet[738]: E0528 22:28:24.704663 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.281601 1540905 logs.go:138] Found kubelet problem: May 28 22:28:26 old-k8s-version-137556 kubelet[738]: E0528 22:28:26.704162 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.281815 1540905 logs.go:138] Found kubelet problem: May 28 22:28:39 old-k8s-version-137556 kubelet[738]: E0528 22:28:39.704787 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.282338 1540905 logs.go:138] Found kubelet problem: May 28 22:28:40 old-k8s-version-137556 kubelet[738]: E0528 22:28:40.704255 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.282977 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.704205 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.283198 1540905 logs.go:138] Found kubelet problem: May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.283416 1540905 logs.go:138] Found kubelet problem: May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.284178 1540905 logs.go:138] Found kubelet problem: May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.284427 1540905 logs.go:138] Found kubelet problem: May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.284797 1540905 logs.go:138] Found kubelet problem: May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.285031 1540905 logs.go:138] Found kubelet problem: May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0528 22:29:33.285058 1540905 logs.go:123] Gathering logs for kube-apiserver [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3] ...
I0528 22:29:33.285090 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3"
I0528 22:29:33.358393 1540905 logs.go:123] Gathering logs for container status ...
I0528 22:29:33.358434 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:33.410155 1540905 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:33.410232 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:33.558604 1540905 logs.go:123] Gathering logs for coredns [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a] ...
I0528 22:29:33.558637 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a"
I0528 22:29:33.598686 1540905 logs.go:123] Gathering logs for kube-controller-manager [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29] ...
I0528 22:29:33.598716 1540905 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29"
I0528 22:29:33.670199 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:33.670230 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:33.670308 1540905 out.go:239] X Problems detected in kubelet:
W0528 22:29:33.670324 1540905 out.go:239] May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.670455 1540905 out.go:239] May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.670464 1540905 out.go:239] May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0528 22:29:33.670488 1540905 out.go:239] May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
W0528 22:29:33.670497 1540905 out.go:239] May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0528 22:29:33.670507 1540905 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:33.670514 1540905 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:33.662036 1546162 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0528 22:29:33.675487 1546162 api_server.go:72] duration metric: took 4m25.018661609s to wait for apiserver process to appear ...
I0528 22:29:33.675517 1546162 api_server.go:88] waiting for apiserver healthz status ...
I0528 22:29:33.675562 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-apiserver Namespaces:[]}
I0528 22:29:33.675622 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0528 22:29:33.727122 1546162 cri.go:89] found id: "dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2"
I0528 22:29:33.727144 1546162 cri.go:89] found id: ""
I0528 22:29:33.727152 1546162 logs.go:276] 1 containers: [dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2]
I0528 22:29:33.727206 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.730731 1546162 cri.go:54] listing CRI containers in root : {State:all Name:etcd Namespaces:[]}
I0528 22:29:33.730813 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0528 22:29:33.771261 1546162 cri.go:89] found id: "aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8"
I0528 22:29:33.771285 1546162 cri.go:89] found id: ""
I0528 22:29:33.771292 1546162 logs.go:276] 1 containers: [aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8]
I0528 22:29:33.771345 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.775371 1546162 cri.go:54] listing CRI containers in root : {State:all Name:coredns Namespaces:[]}
I0528 22:29:33.775448 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0528 22:29:33.811510 1546162 cri.go:89] found id: "61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78"
I0528 22:29:33.811529 1546162 cri.go:89] found id: ""
I0528 22:29:33.811537 1546162 logs.go:276] 1 containers: [61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78]
I0528 22:29:33.811591 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.815319 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-scheduler Namespaces:[]}
I0528 22:29:33.815395 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0528 22:29:33.852867 1546162 cri.go:89] found id: "737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b"
I0528 22:29:33.852889 1546162 cri.go:89] found id: ""
I0528 22:29:33.852897 1546162 logs.go:276] 1 containers: [737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b]
I0528 22:29:33.852979 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.856417 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-proxy Namespaces:[]}
I0528 22:29:33.856487 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0528 22:29:33.892204 1546162 cri.go:89] found id: "e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66"
I0528 22:29:33.892226 1546162 cri.go:89] found id: ""
I0528 22:29:33.892235 1546162 logs.go:276] 1 containers: [e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66]
I0528 22:29:33.892312 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.896455 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kube-controller-manager Namespaces:[]}
I0528 22:29:33.896534 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0528 22:29:33.937747 1546162 cri.go:89] found id: "92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734"
I0528 22:29:33.937770 1546162 cri.go:89] found id: ""
I0528 22:29:33.937777 1546162 logs.go:276] 1 containers: [92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734]
I0528 22:29:33.937866 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.941524 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kindnet Namespaces:[]}
I0528 22:29:33.941648 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0528 22:29:33.979806 1546162 cri.go:89] found id: "c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a"
I0528 22:29:33.979826 1546162 cri.go:89] found id: ""
I0528 22:29:33.979834 1546162 logs.go:276] 1 containers: [c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a]
I0528 22:29:33.979906 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:33.983431 1546162 cri.go:54] listing CRI containers in root : {State:all Name:storage-provisioner Namespaces:[]}
I0528 22:29:33.983502 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0528 22:29:34.029580 1546162 cri.go:89] found id: "db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9"
I0528 22:29:34.029603 1546162 cri.go:89] found id: "07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129"
I0528 22:29:34.029609 1546162 cri.go:89] found id: ""
I0528 22:29:34.029615 1546162 logs.go:276] 2 containers: [db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9 07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129]
I0528 22:29:34.029691 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:34.033489 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:34.037059 1546162 cri.go:54] listing CRI containers in root : {State:all Name:kubernetes-dashboard Namespaces:[]}
I0528 22:29:34.037179 1546162 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0528 22:29:34.080855 1546162 cri.go:89] found id: "639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a"
I0528 22:29:34.080876 1546162 cri.go:89] found id: ""
I0528 22:29:34.080892 1546162 logs.go:276] 1 containers: [639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a]
I0528 22:29:34.080948 1546162 ssh_runner.go:195] Run: which crictl
I0528 22:29:34.085211 1546162 logs.go:123] Gathering logs for kindnet [c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a] ...
I0528 22:29:34.085238 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c436922d5e590ed71af77a5e0fb81ec0ce7e3930afceda784aae103ba74b428a"
I0528 22:29:34.127620 1546162 logs.go:123] Gathering logs for kubernetes-dashboard [639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a] ...
I0528 22:29:34.127656 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 639c3df42842cbc3cacb6b0b6a54672c1485dbd20d7ef4bd83fbfe76d112e30a"
I0528 22:29:34.171103 1546162 logs.go:123] Gathering logs for CRI-O ...
I0528 22:29:34.171134 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u crio -n 400"
I0528 22:29:34.247445 1546162 logs.go:123] Gathering logs for etcd [aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8] ...
I0528 22:29:34.247481 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa0c8e266cfe479bc662b630eb59e99e499e651a685ee88e02399bdbcc467fd8"
I0528 22:29:34.298824 1546162 logs.go:123] Gathering logs for coredns [61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78] ...
I0528 22:29:34.298856 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 61c8c399180a585b6c4aaf6bb7e4f6bcb2bbcc6d0e8136d26cf4e034870ded78"
I0528 22:29:34.349659 1546162 logs.go:123] Gathering logs for storage-provisioner [db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9] ...
I0528 22:29:34.349688 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 db41507f25a5d10cf2c344b0be6d2933ce5190a98b22888de50a60e7699195a9"
I0528 22:29:34.399282 1546162 logs.go:123] Gathering logs for storage-provisioner [07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129] ...
I0528 22:29:34.399307 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 07a58c3fcc756c7e5878f9e675b373c250348ebf565e9c0acb5c499b9aac9129"
I0528 22:29:34.439298 1546162 logs.go:123] Gathering logs for container status ...
I0528 22:29:34.439328 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0528 22:29:34.488734 1546162 logs.go:123] Gathering logs for kubelet ...
I0528 22:29:34.488764 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0528 22:29:34.537884 1546162 logs.go:138] Found kubelet problem: May 28 22:25:28 no-preload-264173 kubelet[747]: W0528 22:25:28.245329 747 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
W0528 22:29:34.538138 1546162 logs.go:138] Found kubelet problem: May 28 22:25:28 no-preload-264173 kubelet[747]: E0528 22:25:28.245366 747 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
I0528 22:29:34.561367 1546162 logs.go:123] Gathering logs for dmesg ...
I0528 22:29:34.561399 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0528 22:29:34.581705 1546162 logs.go:123] Gathering logs for describe nodes ...
I0528 22:29:34.581735 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.30.1/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0528 22:29:34.709346 1546162 logs.go:123] Gathering logs for kube-apiserver [dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2] ...
I0528 22:29:34.709377 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 dfbd292c53c8b0af8de131291c95af8a9bc3f49d144fa7ee77b0e513547eeea2"
I0528 22:29:34.779780 1546162 logs.go:123] Gathering logs for kube-scheduler [737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b] ...
I0528 22:29:34.779859 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 737b93290762f5f2b3818501d896453a1d5caa87b47dfd5d396910388c6eb87b"
I0528 22:29:34.832321 1546162 logs.go:123] Gathering logs for kube-proxy [e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66] ...
I0528 22:29:34.832351 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 e2194e3ac3ff1b30b68520e9f14a14e44e4dd92c56de3c2795db55e43ab28b66"
I0528 22:29:34.869988 1546162 logs.go:123] Gathering logs for kube-controller-manager [92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734] ...
I0528 22:29:34.870040 1546162 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 92648f040c3706797042c24570746c0268b8f17ee46beafa539de155e0258734"
I0528 22:29:34.928666 1546162 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:34.928700 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0528 22:29:34.928771 1546162 out.go:239] X Problems detected in kubelet:
W0528 22:29:34.928787 1546162 out.go:239] May 28 22:25:28 no-preload-264173 kubelet[747]: W0528 22:25:28.245329 747 reflector.go:547] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
W0528 22:29:34.928795 1546162 out.go:239] May 28 22:25:28 no-preload-264173 kubelet[747]: E0528 22:25:28.245366 747 reflector.go:150] object-"kubernetes-dashboard"/"kube-root-ca.crt": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-264173" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-264173' and this object
I0528 22:29:34.928916 1546162 out.go:304] Setting ErrFile to fd 2...
I0528 22:29:34.928923 1546162 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0528 22:29:43.671573 1540905 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0528 22:29:43.691524 1540905 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0528 22:29:43.693704 1540905 out.go:177]
W0528 22:29:43.696067 1540905 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0528 22:29:43.696109 1540905 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0528 22:29:43.696128 1540905 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0528 22:29:43.696133 1540905 out.go:239] *
W0528 22:29:43.697172 1540905 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0528 22:29:43.699282 1540905 out.go:177]
==> CRI-O <==
May 28 22:27:43 old-k8s-version-137556 conmon[2351]: conmon 9c6d3e2adbe78524504e <ninfo>: container 2362 exited with status 1
May 28 22:27:44 old-k8s-version-137556 crio[625]: time="2024-05-28 22:27:44.341951600Z" level=info msg="Removing container: ebdef6f57e7753d65d025f754096305c549444aee4586118d0891d48d3df3fef" id=8509da57-d478-4835-868e-c68ac1ded682 name=/runtime.v1alpha2.RuntimeService/RemoveContainer
May 28 22:27:44 old-k8s-version-137556 crio[625]: time="2024-05-28 22:27:44.369436594Z" level=info msg="Removed container ebdef6f57e7753d65d025f754096305c549444aee4586118d0891d48d3df3fef: kubernetes-dashboard/dashboard-metrics-scraper-8d5bb5db8-p2r9s/dashboard-metrics-scraper" id=8509da57-d478-4835-868e-c68ac1ded682 name=/runtime.v1alpha2.RuntimeService/RemoveContainer
May 28 22:27:48 old-k8s-version-137556 crio[625]: time="2024-05-28 22:27:48.704171596Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=37da0746-bc4c-40c6-b2c2-a96723cea7d3 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:27:48 old-k8s-version-137556 crio[625]: time="2024-05-28 22:27:48.704400316Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=37da0746-bc4c-40c6-b2c2-a96723cea7d3 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:00 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:00.704222025Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=775aa0f8-713d-46db-9a8b-2731a6cb2acc name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:00 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:00.704508738Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=775aa0f8-713d-46db-9a8b-2731a6cb2acc name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:12 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:12.704578924Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=fd07f485-0a85-4632-ad47-d133671002ee name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:12 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:12.704810139Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=fd07f485-0a85-4632-ad47-d133671002ee name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:24 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:24.704202073Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=4ea1a5aa-9006-4b79-bdd6-8561eae39a19 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:24 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:24.704437538Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=4ea1a5aa-9006-4b79-bdd6-8561eae39a19 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:39 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:39.704175705Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=3fa0de35-2f17-4525-97ae-4773efde85aa name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:39 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:39.704413746Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=3fa0de35-2f17-4525-97ae-4773efde85aa name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:51 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:51.441679326Z" level=info msg="Checking image status: k8s.gcr.io/pause:3.2" id=2acf67e4-bdca-4e71-88a0-2b4a33185d70 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:51 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:51.441924489Z" level=info msg="Image status: &ImageStatusResponse{Image:&Image{Id:2a060e2e7101d419352bf82c613158587400be743482d9a537ec4a9d1b4eb93c,RepoTags:[k8s.gcr.io/pause:3.2 registry.k8s.io/pause:3.2],RepoDigests:[k8s.gcr.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f registry.k8s.io/pause@sha256:31d3efd12022ffeffb3146bc10ae8beb890c80ed2f07363515580add7ed47636 registry.k8s.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f],Size_:489397,Uid:nil,Username:,Spec:nil,},Info:map[string]string{},}" id=2acf67e4-bdca-4e71-88a0-2b4a33185d70 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:52 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:52.704308698Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=22d55952-e529-4206-8d10-33c45dbe03d7 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:28:52 old-k8s-version-137556 crio[625]: time="2024-05-28 22:28:52.704527745Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=22d55952-e529-4206-8d10-33c45dbe03d7 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:04 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:04.704359437Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=87956884-873e-4a58-9790-a8a2db8c6b8f name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:04 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:04.704609588Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=87956884-873e-4a58-9790-a8a2db8c6b8f name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:18 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:18.704219952Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=39abebf7-5550-444d-b3e1-8b790d0ee0a9 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:18 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:18.704461579Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=39abebf7-5550-444d-b3e1-8b790d0ee0a9 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:29 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:29.704233490Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=be79a416-203f-4ec5-bd0e-f71c448ce9e9 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:29 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:29.704468060Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=be79a416-203f-4ec5-bd0e-f71c448ce9e9 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:42 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:42.704241825Z" level=info msg="Checking image status: fake.domain/registry.k8s.io/echoserver:1.4" id=f9abf525-53f6-4e56-beb1-124861fb66d3 name=/runtime.v1alpha2.ImageService/ImageStatus
May 28 22:29:42 old-k8s-version-137556 crio[625]: time="2024-05-28 22:29:42.704499164Z" level=info msg="Image fake.domain/registry.k8s.io/echoserver:1.4 not found" id=f9abf525-53f6-4e56-beb1-124861fb66d3 name=/runtime.v1alpha2.ImageService/ImageStatus
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
9c6d3e2adbe78 a90209bb39e3d7b5fc9daf60c17044ea969aaca0333d672d8c7a34c7446e7ff7 2 minutes ago Exited dashboard-metrics-scraper 5 bf431ccc20423 dashboard-metrics-scraper-8d5bb5db8-p2r9s
4c549161f1119 ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Running storage-provisioner 1 e76f6651fd7f8 storage-provisioner
bd823ba62bc36 docker.io/kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93 5 minutes ago Running kubernetes-dashboard 0 7240af405d932 kubernetes-dashboard-cd95d586-8fm5h
d0d1f2d0a469d 1611cd07b61d57dbbfebe6db242513fd51e1c02d20ba08af17a45837d86a8a8c 5 minutes ago Running busybox 0 942d1ccda4bb1 busybox
cfa4cf6ca058c db91994f4ee8f894a1e8a6c1a76f615da8fc3c019300a3686291ce6fcbc57895 5 minutes ago Running coredns 0 6310403df32fa coredns-74ff55c5b-rkctv
05623e44fa29d ba04bb24b95753201135cbc420b233c1b0b9fa2e1fd21d28319c348c33fbcde6 5 minutes ago Exited storage-provisioner 0 e76f6651fd7f8 storage-provisioner
7b6e2d0457829 25a5233254979d0678a2db1d15b76b73dc380d81bc5eed93916ba5638b3cd894 5 minutes ago Running kube-proxy 0 14802998d0f83 kube-proxy-8jz6w
dd01bfbb0358d 89d73d416b992e8f9602b67b4614d9e7f0655aebb3696e18efec695e0b654c40 5 minutes ago Running kindnet-cni 0 51500ce31d489 kindnet-p94zm
9ac42df22be82 2c08bbbc02d3aa5dfbf4e79f15c0a61424049288917aa10364464ca1f7de7157 5 minutes ago Running kube-apiserver 0 5e814ea5b33b5 kube-apiserver-old-k8s-version-137556
c6104b8b4482d 05b738aa1bc6355db8a2ee8639f3631b908286e43f584a3d2ee0c472de033c28 5 minutes ago Running etcd 0 7c44d1765ed5b etcd-old-k8s-version-137556
41bbe9eb76e24 1df8a2b116bd16f7070fd383a6769c8d644b365575e8ffa3e492b84e4f05fc74 5 minutes ago Running kube-controller-manager 0 81d9e66b117bd kube-controller-manager-old-k8s-version-137556
b431299048367 e7605f88f17d6a4c3f083ef9c6f5f19b39f87e4d4406a05a8612b54a6ea57051 5 minutes ago Running kube-scheduler 0 f22d367b8975b kube-scheduler-old-k8s-version-137556
==> coredns [cfa4cf6ca058c1565fcdfd433992ad14f91256b5d1fce4f4d2a0e7f68b1a683a] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:43030 - 41591 "HINFO IN 2352742964943992761.5656879409533965859. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.013668915s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:42507 - 57517 "HINFO IN 229562044839275671.5753682441334951666. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.02204632s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0528 22:24:36.006932 1 trace.go:116] Trace[1427131847]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-28 22:24:05.999342715 +0000 UTC m=+0.099054401) (total time: 30.007210072s):
Trace[1427131847]: [30.007210072s] [30.007210072s] END
E0528 22:24:36.006976 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0528 22:24:36.007119 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-28 22:24:05.998812562 +0000 UTC m=+0.098524256) (total time: 30.007616092s):
Trace[2019727887]: [30.007616092s] [30.007616092s] END
E0528 22:24:36.007129 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0528 22:24:36.007210 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-28 22:24:05.999603582 +0000 UTC m=+0.099315268) (total time: 30.007115551s):
Trace[939984059]: [30.007115551s] [30.007115551s] END
E0528 22:24:36.007214 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> describe nodes <==
Name: old-k8s-version-137556
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-137556
kubernetes.io/os=linux
minikube.k8s.io/commit=c95b4fdda455689199e2a93674568b261e34dc82
minikube.k8s.io/name=old-k8s-version-137556
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_05_28T22_21_20_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/crio/crio.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Tue, 28 May 2024 22:21:16 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-137556
AcquireTime: <unset>
RenewTime: Tue, 28 May 2024 22:29:35 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Tue, 28 May 2024 22:24:54 +0000 Tue, 28 May 2024 22:21:09 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Tue, 28 May 2024 22:24:54 +0000 Tue, 28 May 2024 22:21:09 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Tue, 28 May 2024 22:24:54 +0000 Tue, 28 May 2024 22:21:09 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 28 May 2024 22:24:54 +0000 Tue, 28 May 2024 22:21:48 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-137556
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022436Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022436Ki
pods: 110
System Info:
Machine ID: 14aafd46ac8547e9800fd0b239a4e2b6
System UUID: 6556a3cc-e8a0-4e10-bbe8-bc825fb4ce55
Boot ID: 2882d43f-5a85-456c-aec3-876199af1cc0
Kernel Version: 5.15.0-1062-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: cri-o://1.24.6
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m35s
kube-system coredns-74ff55c5b-rkctv 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m10s
kube-system etcd-old-k8s-version-137556 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m17s
kube-system kindnet-p94zm 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m10s
kube-system kube-apiserver-old-k8s-version-137556 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m17s
kube-system kube-controller-manager-old-k8s-version-137556 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m17s
kube-system kube-proxy-8jz6w 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m10s
kube-system kube-scheduler-old-k8s-version-137556 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m17s
kube-system metrics-server-9975d5f86-h5vcp 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m23s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m9s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-p2r9s 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m22s
kubernetes-dashboard kubernetes-dashboard-cd95d586-8fm5h 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m22s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m37s (x5 over 8m37s) kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m37s (x4 over 8m37s) kubelet Node old-k8s-version-137556 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m37s (x4 over 8m37s) kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientPID
Normal Starting 8m18s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m17s kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m17s kubelet Node old-k8s-version-137556 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m17s kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientPID
Normal Starting 8m8s kube-proxy Starting kube-proxy.
Normal NodeReady 7m57s kubelet Node old-k8s-version-137556 status is now: NodeReady
Normal Starting 5m54s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 5m54s (x8 over 5m54s) kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 5m54s (x8 over 5m54s) kubelet Node old-k8s-version-137556 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 5m54s (x8 over 5m54s) kubelet Node old-k8s-version-137556 status is now: NodeHasSufficientPID
Normal Starting 5m39s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.000809] FS-Cache: N-cookie c=00000054 [p=0000004b fl=2 nc=0 na=1]
[ +0.001129] FS-Cache: N-cookie d=000000000847e55d{9p.inode} n=000000003d37d697
[ +0.001063] FS-Cache: N-key=[8] '02d8c90000000000'
[ +0.004281] FS-Cache: Duplicate cookie detected
[ +0.000738] FS-Cache: O-cookie c=0000004e [p=0000004b fl=226 nc=0 na=1]
[ +0.001015] FS-Cache: O-cookie d=000000000847e55d{9p.inode} n=00000000bc6509f8
[ +0.001137] FS-Cache: O-key=[8] '02d8c90000000000'
[ +0.000764] FS-Cache: N-cookie c=00000055 [p=0000004b fl=2 nc=0 na=1]
[ +0.000912] FS-Cache: N-cookie d=000000000847e55d{9p.inode} n=000000000017120e
[ +0.001082] FS-Cache: N-key=[8] '02d8c90000000000'
[ +2.267064] FS-Cache: Duplicate cookie detected
[ +0.001056] FS-Cache: O-cookie c=0000004c [p=0000004b fl=226 nc=0 na=1]
[ +0.001170] FS-Cache: O-cookie d=000000000847e55d{9p.inode} n=00000000aa2a29a4
[ +0.001168] FS-Cache: O-key=[8] '01d8c90000000000'
[ +0.000769] FS-Cache: N-cookie c=00000057 [p=0000004b fl=2 nc=0 na=1]
[ +0.001030] FS-Cache: N-cookie d=000000000847e55d{9p.inode} n=000000003d37d697
[ +0.001300] FS-Cache: N-key=[8] '01d8c90000000000'
[ +0.392949] FS-Cache: Duplicate cookie detected
[ +0.000791] FS-Cache: O-cookie c=00000051 [p=0000004b fl=226 nc=0 na=1]
[ +0.000981] FS-Cache: O-cookie d=000000000847e55d{9p.inode} n=00000000dae7ba57
[ +0.001205] FS-Cache: O-key=[8] '07d8c90000000000'
[ +0.000689] FS-Cache: N-cookie c=00000058 [p=0000004b fl=2 nc=0 na=1]
[ +0.000911] FS-Cache: N-cookie d=000000000847e55d{9p.inode} n=0000000086e4d09a
[ +0.001030] FS-Cache: N-key=[8] '07d8c90000000000'
[May28 22:16] overlayfs: '/var/lib/containers/storage/overlay/l/Q2QJNMTVZL6GMULS36RA5ZJGSA' not a directory
==> etcd [c6104b8b4482d109a66493c71b372abfe5cfac7f2c526736c072a385dc7798f8] <==
2024-05-28 22:25:43.028527 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:25:53.028348 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:03.028513 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:13.028464 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:23.028415 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:33.028659 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:43.028496 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:26:53.028506 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:03.028592 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:13.028411 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:23.028531 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:33.028462 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:43.028436 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:27:53.028558 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:03.028443 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:13.028570 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:23.028453 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:33.028428 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:43.028456 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:28:53.028587 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:29:03.028456 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:29:13.028473 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:29:23.028675 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:29:33.029276 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-28 22:29:43.028529 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
22:29:45 up 6:12, 0 users, load average: 0.50, 1.63, 2.27
Linux old-k8s-version-137556 5.15.0-1062-aws #68~20.04.1-Ubuntu SMP Tue May 7 11:50:35 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [dd01bfbb0358d6cf9694d13afcf99a9691a35a5e806521ed56465425c701b2e5] <==
I0528 22:27:35.586771 1 main.go:227] handling current node
I0528 22:27:45.600921 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:27:45.600952 1 main.go:227] handling current node
I0528 22:27:55.606703 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:27:55.606732 1 main.go:227] handling current node
I0528 22:28:05.612232 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:05.612261 1 main.go:227] handling current node
I0528 22:28:15.625686 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:15.625713 1 main.go:227] handling current node
I0528 22:28:25.640436 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:25.640467 1 main.go:227] handling current node
I0528 22:28:35.645672 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:35.645705 1 main.go:227] handling current node
I0528 22:28:45.658407 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:45.658435 1 main.go:227] handling current node
I0528 22:28:55.672137 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:28:55.672165 1 main.go:227] handling current node
I0528 22:29:05.764618 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:29:05.766117 1 main.go:227] handling current node
I0528 22:29:15.848444 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:29:15.848472 1 main.go:227] handling current node
I0528 22:29:25.950911 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:29:25.950942 1 main.go:227] handling current node
I0528 22:29:36.024814 1 main.go:223] Handling node with IPs: map[192.168.85.2:{}]
I0528 22:29:36.024847 1 main.go:227] handling current node
==> kube-apiserver [9ac42df22be82ca66d9f1b421375548c94bd4989289312d40efe43edfffb28c3] <==
I0528 22:26:18.244564 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:26:18.244572 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0528 22:26:59.406999 1 client.go:360] parsed scheme: "passthrough"
I0528 22:26:59.407044 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:26:59.407052 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0528 22:27:07.579516 1 handler_proxy.go:102] no RequestInfo found in the context
E0528 22:27:07.579683 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0528 22:27:07.579735 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0528 22:27:41.490722 1 client.go:360] parsed scheme: "passthrough"
I0528 22:27:41.490766 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:27:41.490774 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0528 22:28:19.555374 1 client.go:360] parsed scheme: "passthrough"
I0528 22:28:19.555420 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:28:19.555430 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0528 22:28:50.981475 1 client.go:360] parsed scheme: "passthrough"
I0528 22:28:50.981533 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:28:50.981541 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0528 22:29:04.395639 1 handler_proxy.go:102] no RequestInfo found in the context
E0528 22:29:04.395713 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0528 22:29:04.395723 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0528 22:29:25.536438 1 client.go:360] parsed scheme: "passthrough"
I0528 22:29:25.536482 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0528 22:29:25.536491 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [41bbe9eb76e24222f16e08bedb27572c1a1f23de7c322144f5edad4a3849ca29] <==
E0528 22:25:24.366218 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:25:27.986224 1 request.go:655] Throttling request took 1.048444155s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0528 22:25:28.837739 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:25:54.868469 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:26:00.488147 1 request.go:655] Throttling request took 1.048446419s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1?timeout=32s
W0528 22:26:01.339744 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:26:25.370770 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:26:32.990139 1 request.go:655] Throttling request took 1.048400455s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0528 22:26:33.841533 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:26:55.872792 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:27:05.491912 1 request.go:655] Throttling request took 1.048342159s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0528 22:27:06.343465 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:27:26.380824 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:27:37.993817 1 request.go:655] Throttling request took 1.048402416s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0528 22:27:38.845302 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:27:56.883455 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:28:10.495689 1 request.go:655] Throttling request took 1.048377021s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0528 22:28:11.347146 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:28:27.385352 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:28:42.997681 1 request.go:655] Throttling request took 1.048235486s, request: GET:https://192.168.85.2:8443/apis/authorization.k8s.io/v1beta1?timeout=32s
W0528 22:28:43.849096 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:28:57.887153 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0528 22:29:15.499666 1 request.go:655] Throttling request took 1.048299206s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0528 22:29:16.351149 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0528 22:29:28.388829 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
==> kube-proxy [7b6e2d0457829e44adcacbd8351453a6bae452df959fbf6234bf0632a810909b] <==
I0528 22:21:36.987864 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0528 22:21:36.988177 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0528 22:21:37.011026 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0528 22:21:37.011261 1 server_others.go:185] Using iptables Proxier.
I0528 22:21:37.011554 1 server.go:650] Version: v1.20.0
I0528 22:21:37.012359 1 config.go:315] Starting service config controller
I0528 22:21:37.012430 1 shared_informer.go:240] Waiting for caches to sync for service config
I0528 22:21:37.012489 1 config.go:224] Starting endpoint slice config controller
I0528 22:21:37.012532 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0528 22:21:37.112582 1 shared_informer.go:247] Caches are synced for service config
I0528 22:21:37.112719 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0528 22:24:06.737367 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0528 22:24:06.737610 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0528 22:24:06.808607 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0528 22:24:06.808821 1 server_others.go:185] Using iptables Proxier.
I0528 22:24:06.809108 1 server.go:650] Version: v1.20.0
I0528 22:24:06.824713 1 config.go:315] Starting service config controller
I0528 22:24:06.824814 1 shared_informer.go:240] Waiting for caches to sync for service config
I0528 22:24:06.824908 1 config.go:224] Starting endpoint slice config controller
I0528 22:24:06.833337 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0528 22:24:06.924962 1 shared_informer.go:247] Caches are synced for service config
I0528 22:24:06.933548 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-scheduler [b43129904836783c320b181b2b3d5e9b5752b3373a3d3cdfe053173d601b7105] <==
E0528 22:21:16.498638 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0528 22:21:16.498729 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0528 22:21:16.498804 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0528 22:21:16.501910 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0528 22:21:16.502178 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0528 22:21:16.502549 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0528 22:21:16.502643 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0528 22:21:17.356700 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0528 22:21:17.412974 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0528 22:21:17.483579 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0528 22:21:17.484784 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0528 22:21:17.498248 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0528 22:21:17.508923 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0528 22:21:17.534460 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
I0528 22:21:19.956101 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0528 22:23:57.458752 1 serving.go:331] Generated self-signed cert in-memory
W0528 22:24:03.178274 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0528 22:24:03.178413 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0528 22:24:03.178428 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0528 22:24:03.178435 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0528 22:24:03.755299 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0528 22:24:03.764400 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0528 22:24:03.764431 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0528 22:24:03.764456 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0528 22:24:04.079468 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
May 28 22:28:12 old-k8s-version-137556 kubelet[738]: I0528 22:28:12.703884 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705029 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:28:12 old-k8s-version-137556 kubelet[738]: E0528 22:28:12.705742 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:28:24 old-k8s-version-137556 kubelet[738]: E0528 22:28:24.704663 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:28:26 old-k8s-version-137556 kubelet[738]: I0528 22:28:26.703823 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:28:26 old-k8s-version-137556 kubelet[738]: E0528 22:28:26.704162 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:28:39 old-k8s-version-137556 kubelet[738]: E0528 22:28:39.704787 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:28:40 old-k8s-version-137556 kubelet[738]: I0528 22:28:40.703884 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:28:40 old-k8s-version-137556 kubelet[738]: E0528 22:28:40.704255 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:28:51 old-k8s-version-137556 kubelet[738]: E0528 22:28:51.616331 738 container_manager_linux.go:533] failed to find cgroups of kubelet - cpu and memory cgroup hierarchy not unified. cpu: /docker/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592, memory: /docker/34fd7d98b135c1134fea5ddc99ba08b42735c00cc3f7217c47fa37b68a70a592/system.slice/kubelet.service
May 28 22:28:52 old-k8s-version-137556 kubelet[738]: I0528 22:28:52.703870 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.704205 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:28:52 old-k8s-version-137556 kubelet[738]: E0528 22:28:52.705255 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:04 old-k8s-version-137556 kubelet[738]: E0528 22:29:04.704842 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:07 old-k8s-version-137556 kubelet[738]: I0528 22:29:07.703951 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:29:07 old-k8s-version-137556 kubelet[738]: E0528 22:29:07.704319 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:18 old-k8s-version-137556 kubelet[738]: E0528 22:29:18.704693 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:20 old-k8s-version-137556 kubelet[738]: I0528 22:29:20.703841 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:29:20 old-k8s-version-137556 kubelet[738]: E0528 22:29:20.704192 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:29 old-k8s-version-137556 kubelet[738]: E0528 22:29:29.704635 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:34 old-k8s-version-137556 kubelet[738]: I0528 22:29:34.703808 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:29:34 old-k8s-version-137556 kubelet[738]: E0528 22:29:34.704147 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
May 28 22:29:42 old-k8s-version-137556 kubelet[738]: E0528 22:29:42.704958 738 pod_workers.go:191] Error syncing pod 7bd3caf6-43e5-49dc-a226-fcdaf2249bff ("metrics-server-9975d5f86-h5vcp_kube-system(7bd3caf6-43e5-49dc-a226-fcdaf2249bff)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 28 22:29:45 old-k8s-version-137556 kubelet[738]: I0528 22:29:45.704985 738 scope.go:95] [topologymanager] RemoveContainer - Container ID: 9c6d3e2adbe78524504e484ae9691bb7b6d52f34a4ca64dd2e3f201d3598ea79
May 28 22:29:45 old-k8s-version-137556 kubelet[738]: E0528 22:29:45.705976 738 pod_workers.go:191] Error syncing pod 52ebbcd6-6bc2-4840-81e6-3a68800bf50f ("dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-p2r9s_kubernetes-dashboard(52ebbcd6-6bc2-4840-81e6-3a68800bf50f)"
==> kubernetes-dashboard [bd823ba62bc36515af73815b13f0095eafdd2fc56df1f3d22fcb9af68a5fd527] <==
2024/05/28 22:24:30 Using namespace: kubernetes-dashboard
2024/05/28 22:24:30 Using in-cluster config to connect to apiserver
2024/05/28 22:24:30 Using secret token for csrf signing
2024/05/28 22:24:30 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/05/28 22:24:30 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/05/28 22:24:30 Successful initial request to the apiserver, version: v1.20.0
2024/05/28 22:24:30 Generating JWE encryption key
2024/05/28 22:24:30 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/05/28 22:24:30 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/05/28 22:24:32 Initializing JWE encryption key from synchronized object
2024/05/28 22:24:32 Creating in-cluster Sidecar client
2024/05/28 22:24:32 Serving insecurely on HTTP port: 9090
2024/05/28 22:24:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:25:02 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:25:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:26:02 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:26:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:27:02 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:27:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:28:02 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:28:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:29:02 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:29:32 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/28 22:24:30 Starting overwatch
==> storage-provisioner [05623e44fa29d497e3027c10fedf174172e73932779e0845b16168bfb8528998] <==
I0528 22:21:53.158252 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0528 22:21:53.197182 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0528 22:21:53.197439 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0528 22:21:53.242070 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0528 22:21:53.242764 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-137556_868d864f-3eab-4c41-b18d-bb027c72ee63!
I0528 22:21:53.242614 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"fb0c04ff-3f43-492c-b399-fb03b1ba22b1", APIVersion:"v1", ResourceVersion:"489", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-137556_868d864f-3eab-4c41-b18d-bb027c72ee63 became leader
I0528 22:21:53.343442 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-137556_868d864f-3eab-4c41-b18d-bb027c72ee63!
I0528 22:24:06.585477 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0528 22:24:36.590194 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [4c549161f111967cb616fe3ed8c91bce54b8269bcb2565d06539abb210ddfc33] <==
I0528 22:24:37.259057 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0528 22:24:37.275049 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0528 22:24:37.275109 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0528 22:24:54.734279 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0528 22:24:54.734675 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-137556_12fd02f4-7b11-45e7-801c-ad3f1194016f!
I0528 22:24:54.735304 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"fb0c04ff-3f43-492c-b399-fb03b1ba22b1", APIVersion:"v1", ResourceVersion:"819", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-137556_12fd02f4-7b11-45e7-801c-ad3f1194016f became leader
I0528 22:24:54.835782 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-137556_12fd02f4-7b11-45e7-801c-ad3f1194016f!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-137556 -n old-k8s-version-137556
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-137556 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-h5vcp
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-137556 describe pod metrics-server-9975d5f86-h5vcp
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-137556 describe pod metrics-server-9975d5f86-h5vcp: exit status 1 (94.222143ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-h5vcp" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-137556 describe pod metrics-server-9975d5f86-h5vcp: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (372.69s)