=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-145466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
E0815 01:27:05.963939 592660 cert_rotation.go:171] "Unhandled Error" err="key failed with : open /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/functional-369279/client.crt: no such file or directory" logger="UnhandledError"
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-145466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m18.345860387s)
-- stdout --
* [old-k8s-version-145466] minikube v1.33.1 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19443
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19443-587265/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19443-587265/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
* Using the docker driver based on existing profile
* Starting "old-k8s-version-145466" primary control-plane node in "old-k8s-version-145466" cluster
* Pulling base image v0.0.44-1723650208-19443 ...
* Restarting existing docker container for "old-k8s-version-145466" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.20 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image fake.domain/registry.k8s.io/echoserver:1.4
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-145466 addons enable metrics-server
* Enabled addons: metrics-server, storage-provisioner, dashboard, default-storageclass
-- /stdout --
** stderr **
I0815 01:25:59.573951 798414 out.go:291] Setting OutFile to fd 1 ...
I0815 01:25:59.574084 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:25:59.574095 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:25:59.574100 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:25:59.574334 798414 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19443-587265/.minikube/bin
I0815 01:25:59.574709 798414 out.go:298] Setting JSON to false
I0815 01:25:59.575653 798414 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":18482,"bootTime":1723666678,"procs":202,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1067-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0815 01:25:59.575726 798414 start.go:139] virtualization:
I0815 01:25:59.578147 798414 out.go:177] * [old-k8s-version-145466] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0815 01:25:59.580624 798414 out.go:177] - MINIKUBE_LOCATION=19443
I0815 01:25:59.580809 798414 notify.go:220] Checking for updates...
I0815 01:25:59.584160 798414 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0815 01:25:59.585694 798414 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:25:59.587773 798414 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19443-587265/.minikube
I0815 01:25:59.589455 798414 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0815 01:25:59.591443 798414 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0815 01:25:59.593889 798414 config.go:182] Loaded profile config "old-k8s-version-145466": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0815 01:25:59.596194 798414 out.go:177] * Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
I0815 01:25:59.597841 798414 driver.go:392] Setting default libvirt URI to qemu:///system
I0815 01:25:59.621518 798414 docker.go:123] docker version: linux-27.1.2:Docker Engine - Community
I0815 01:25:59.621657 798414 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0815 01:25:59.690023 798414 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:53 OomKillDisable:true NGoroutines:68 SystemTime:2024-08-15 01:25:59.67772726 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aarc
h64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerError
s:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0815 01:25:59.690140 798414 docker.go:307] overlay module found
I0815 01:25:59.692335 798414 out.go:177] * Using the docker driver based on existing profile
I0815 01:25:59.694336 798414 start.go:297] selected driver: docker
I0815 01:25:59.694353 798414 start.go:901] validating driver "docker" against &{Name:old-k8s-version-145466 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-145466 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:25:59.694546 798414 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0815 01:25:59.695179 798414 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0815 01:25:59.751775 798414 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:53 OomKillDisable:true NGoroutines:68 SystemTime:2024-08-15 01:25:59.742078871 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0815 01:25:59.752151 798414 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0815 01:25:59.752188 798414 cni.go:84] Creating CNI manager for ""
I0815 01:25:59.752197 798414 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0815 01:25:59.752246 798414 start.go:340] cluster config:
{Name:old-k8s-version-145466 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-145466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:25:59.754616 798414 out.go:177] * Starting "old-k8s-version-145466" primary control-plane node in "old-k8s-version-145466" cluster
I0815 01:25:59.756482 798414 cache.go:121] Beginning downloading kic base image for docker with containerd
I0815 01:25:59.758412 798414 out.go:177] * Pulling base image v0.0.44-1723650208-19443 ...
I0815 01:25:59.760008 798414 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0815 01:25:59.760073 798414 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19443-587265/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0815 01:25:59.760096 798414 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local docker daemon
I0815 01:25:59.760101 798414 cache.go:56] Caching tarball of preloaded images
I0815 01:25:59.760190 798414 preload.go:172] Found /home/jenkins/minikube-integration/19443-587265/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0815 01:25:59.760201 798414 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0815 01:25:59.760318 798414 profile.go:143] Saving config to /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/config.json ...
W0815 01:25:59.787246 798414 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 is of wrong architecture
I0815 01:25:59.787268 798414 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 to local cache
I0815 01:25:59.787353 798414 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local cache directory
I0815 01:25:59.787378 798414 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local cache directory, skipping pull
I0815 01:25:59.787383 798414 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 exists in cache, skipping pull
I0815 01:25:59.787397 798414 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 as a tarball
I0815 01:25:59.787409 798414 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 from local cache
I0815 01:25:59.922097 798414 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 from cached tarball
I0815 01:25:59.922134 798414 cache.go:194] Successfully downloaded all kic artifacts
I0815 01:25:59.922173 798414 start.go:360] acquireMachinesLock for old-k8s-version-145466: {Name:mk4e9cba23400dab2e3a7a2919cb59bfc861e072 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:25:59.922243 798414 start.go:364] duration metric: took 45.063µs to acquireMachinesLock for "old-k8s-version-145466"
I0815 01:25:59.922271 798414 start.go:96] Skipping create...Using existing machine configuration
I0815 01:25:59.922277 798414 fix.go:54] fixHost starting:
I0815 01:25:59.922587 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:25:59.938278 798414 fix.go:112] recreateIfNeeded on old-k8s-version-145466: state=Stopped err=<nil>
W0815 01:25:59.938306 798414 fix.go:138] unexpected machine state, will restart: <nil>
I0815 01:25:59.940644 798414 out.go:177] * Restarting existing docker container for "old-k8s-version-145466" ...
I0815 01:25:59.942493 798414 cli_runner.go:164] Run: docker start old-k8s-version-145466
I0815 01:26:00.522118 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:00.552318 798414 kic.go:430] container "old-k8s-version-145466" state is running.
I0815 01:26:00.552782 798414 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-145466
I0815 01:26:00.585700 798414 profile.go:143] Saving config to /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/config.json ...
I0815 01:26:00.586155 798414 machine.go:94] provisionDockerMachine start ...
I0815 01:26:00.586229 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:00.606533 798414 main.go:141] libmachine: Using SSH client type: native
I0815 01:26:00.606834 798414 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33805 <nil> <nil>}
I0815 01:26:00.606846 798414 main.go:141] libmachine: About to run SSH command:
hostname
I0815 01:26:00.608488 798414 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: EOF
I0815 01:26:03.747543 798414 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-145466
I0815 01:26:03.747568 798414 ubuntu.go:169] provisioning hostname "old-k8s-version-145466"
I0815 01:26:03.747648 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:03.766016 798414 main.go:141] libmachine: Using SSH client type: native
I0815 01:26:03.766303 798414 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33805 <nil> <nil>}
I0815 01:26:03.766319 798414 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-145466 && echo "old-k8s-version-145466" | sudo tee /etc/hostname
I0815 01:26:03.915942 798414 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-145466
I0815 01:26:03.916027 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:03.936903 798414 main.go:141] libmachine: Using SSH client type: native
I0815 01:26:03.937222 798414 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33805 <nil> <nil>}
I0815 01:26:03.937246 798414 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-145466' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-145466/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-145466' | sudo tee -a /etc/hosts;
fi
fi
I0815 01:26:04.076172 798414 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0815 01:26:04.076201 798414 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19443-587265/.minikube CaCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19443-587265/.minikube}
I0815 01:26:04.076241 798414 ubuntu.go:177] setting up certificates
I0815 01:26:04.076252 798414 provision.go:84] configureAuth start
I0815 01:26:04.076323 798414 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-145466
I0815 01:26:04.095332 798414 provision.go:143] copyHostCerts
I0815 01:26:04.095406 798414 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem, removing ...
I0815 01:26:04.095423 798414 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem
I0815 01:26:04.095515 798414 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem (1675 bytes)
I0815 01:26:04.095619 798414 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem, removing ...
I0815 01:26:04.095629 798414 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem
I0815 01:26:04.095659 798414 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem (1082 bytes)
I0815 01:26:04.095723 798414 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem, removing ...
I0815 01:26:04.095733 798414 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem
I0815 01:26:04.095760 798414 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem (1123 bytes)
I0815 01:26:04.095817 798414 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-145466 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-145466]
I0815 01:26:04.816454 798414 provision.go:177] copyRemoteCerts
I0815 01:26:04.816566 798414 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0815 01:26:04.816627 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:04.865610 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:04.961731 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0815 01:26:04.986962 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0815 01:26:05.022151 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0815 01:26:05.048703 798414 provision.go:87] duration metric: took 972.433008ms to configureAuth
I0815 01:26:05.048731 798414 ubuntu.go:193] setting minikube options for container-runtime
I0815 01:26:05.048934 798414 config.go:182] Loaded profile config "old-k8s-version-145466": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0815 01:26:05.048950 798414 machine.go:97] duration metric: took 4.462769778s to provisionDockerMachine
I0815 01:26:05.048958 798414 start.go:293] postStartSetup for "old-k8s-version-145466" (driver="docker")
I0815 01:26:05.048968 798414 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0815 01:26:05.049021 798414 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0815 01:26:05.049063 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:05.072013 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:05.170225 798414 ssh_runner.go:195] Run: cat /etc/os-release
I0815 01:26:05.173544 798414 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0815 01:26:05.173583 798414 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0815 01:26:05.173593 798414 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0815 01:26:05.173601 798414 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0815 01:26:05.173612 798414 filesync.go:126] Scanning /home/jenkins/minikube-integration/19443-587265/.minikube/addons for local assets ...
I0815 01:26:05.173671 798414 filesync.go:126] Scanning /home/jenkins/minikube-integration/19443-587265/.minikube/files for local assets ...
I0815 01:26:05.173762 798414 filesync.go:149] local asset: /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem -> 5926602.pem in /etc/ssl/certs
I0815 01:26:05.173877 798414 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0815 01:26:05.182899 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem --> /etc/ssl/certs/5926602.pem (1708 bytes)
I0815 01:26:05.209284 798414 start.go:296] duration metric: took 160.311879ms for postStartSetup
I0815 01:26:05.209368 798414 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0815 01:26:05.209415 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:05.226804 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:05.320876 798414 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0815 01:26:05.325268 798414 fix.go:56] duration metric: took 5.40298248s for fixHost
I0815 01:26:05.325293 798414 start.go:83] releasing machines lock for "old-k8s-version-145466", held for 5.403033705s
I0815 01:26:05.325362 798414 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-145466
I0815 01:26:05.341534 798414 ssh_runner.go:195] Run: cat /version.json
I0815 01:26:05.341595 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:05.341594 798414 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0815 01:26:05.341750 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:05.358372 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:05.373543 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:05.585439 798414 ssh_runner.go:195] Run: systemctl --version
I0815 01:26:05.589817 798414 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0815 01:26:05.594226 798414 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0815 01:26:05.611706 798414 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0815 01:26:05.611784 798414 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0815 01:26:05.620827 798414 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0815 01:26:05.620867 798414 start.go:495] detecting cgroup driver to use...
I0815 01:26:05.620904 798414 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0815 01:26:05.620970 798414 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0815 01:26:05.634481 798414 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0815 01:26:05.647095 798414 docker.go:217] disabling cri-docker service (if available) ...
I0815 01:26:05.647166 798414 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0815 01:26:05.660789 798414 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0815 01:26:05.672598 798414 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0815 01:26:05.761732 798414 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0815 01:26:05.846989 798414 docker.go:233] disabling docker service ...
I0815 01:26:05.847057 798414 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0815 01:26:05.860790 798414 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0815 01:26:05.872605 798414 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0815 01:26:05.968597 798414 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0815 01:26:06.077959 798414 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0815 01:26:06.090706 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0815 01:26:06.108435 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0815 01:26:06.119881 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0815 01:26:06.132368 798414 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0815 01:26:06.132441 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0815 01:26:06.142917 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0815 01:26:06.153383 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0815 01:26:06.163568 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0815 01:26:06.173539 798414 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0815 01:26:06.184905 798414 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0815 01:26:06.194797 798414 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0815 01:26:06.203755 798414 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0815 01:26:06.212910 798414 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:26:06.328154 798414 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0815 01:26:06.757971 798414 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0815 01:26:06.758050 798414 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0815 01:26:06.764487 798414 start.go:563] Will wait 60s for crictl version
I0815 01:26:06.764550 798414 ssh_runner.go:195] Run: which crictl
I0815 01:26:06.768112 798414 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0815 01:26:06.826837 798414 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.20
RuntimeApiVersion: v1
I0815 01:26:06.826906 798414 ssh_runner.go:195] Run: containerd --version
I0815 01:26:06.853824 798414 ssh_runner.go:195] Run: containerd --version
I0815 01:26:06.890136 798414 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.20 ...
I0815 01:26:06.892071 798414 cli_runner.go:164] Run: docker network inspect old-k8s-version-145466 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0815 01:26:06.920269 798414 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0815 01:26:06.924190 798414 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 01:26:06.938830 798414 kubeadm.go:883] updating cluster {Name:old-k8s-version-145466 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-145466 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0815 01:26:06.938954 798414 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0815 01:26:06.939012 798414 ssh_runner.go:195] Run: sudo crictl images --output json
I0815 01:26:06.986729 798414 containerd.go:627] all images are preloaded for containerd runtime.
I0815 01:26:06.986752 798414 containerd.go:534] Images already preloaded, skipping extraction
I0815 01:26:06.986815 798414 ssh_runner.go:195] Run: sudo crictl images --output json
I0815 01:26:07.083046 798414 containerd.go:627] all images are preloaded for containerd runtime.
I0815 01:26:07.083072 798414 cache_images.go:84] Images are preloaded, skipping loading
I0815 01:26:07.083080 798414 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.20.0 containerd true true} ...
I0815 01:26:07.083202 798414 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-145466 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-145466 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0815 01:26:07.083277 798414 ssh_runner.go:195] Run: sudo crictl info
I0815 01:26:07.166482 798414 cni.go:84] Creating CNI manager for ""
I0815 01:26:07.166511 798414 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0815 01:26:07.166522 798414 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0815 01:26:07.166574 798414 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-145466 NodeName:old-k8s-version-145466 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0815 01:26:07.166755 798414 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-145466"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0815 01:26:07.166840 798414 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0815 01:26:07.177168 798414 binaries.go:44] Found k8s binaries, skipping transfer
I0815 01:26:07.177242 798414 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0815 01:26:07.186863 798414 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0815 01:26:07.206656 798414 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0815 01:26:07.226261 798414 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0815 01:26:07.246084 798414 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0815 01:26:07.249707 798414 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 01:26:07.261141 798414 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:26:07.376798 798414 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0815 01:26:07.400340 798414 certs.go:68] Setting up /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466 for IP: 192.168.85.2
I0815 01:26:07.400360 798414 certs.go:194] generating shared ca certs ...
I0815 01:26:07.400382 798414 certs.go:226] acquiring lock for ca certs: {Name:mkd44da6bd4b219dfe871c9c58d5756252de3a14 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:26:07.400542 798414 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19443-587265/.minikube/ca.key
I0815 01:26:07.400613 798414 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.key
I0815 01:26:07.400627 798414 certs.go:256] generating profile certs ...
I0815 01:26:07.400741 798414 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/client.key
I0815 01:26:07.400828 798414 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/apiserver.key.ae927303
I0815 01:26:07.400905 798414 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/proxy-client.key
I0815 01:26:07.401049 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660.pem (1338 bytes)
W0815 01:26:07.401101 798414 certs.go:480] ignoring /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660_empty.pem, impossibly tiny 0 bytes
I0815 01:26:07.401115 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem (1675 bytes)
I0815 01:26:07.401158 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem (1082 bytes)
I0815 01:26:07.401205 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem (1123 bytes)
I0815 01:26:07.401236 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem (1675 bytes)
I0815 01:26:07.401297 798414 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem (1708 bytes)
I0815 01:26:07.402037 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0815 01:26:07.490895 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0815 01:26:07.561737 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0815 01:26:07.629284 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0815 01:26:07.695983 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0815 01:26:07.732866 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0815 01:26:07.770035 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0815 01:26:07.807374 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/old-k8s-version-145466/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0815 01:26:07.844306 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem --> /usr/share/ca-certificates/5926602.pem (1708 bytes)
I0815 01:26:07.880814 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0815 01:26:07.918363 798414 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660.pem --> /usr/share/ca-certificates/592660.pem (1338 bytes)
I0815 01:26:07.973325 798414 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0815 01:26:08.021984 798414 ssh_runner.go:195] Run: openssl version
I0815 01:26:08.031704 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/5926602.pem && ln -fs /usr/share/ca-certificates/5926602.pem /etc/ssl/certs/5926602.pem"
I0815 01:26:08.051374 798414 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/5926602.pem
I0815 01:26:08.058833 798414 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 15 00:47 /usr/share/ca-certificates/5926602.pem
I0815 01:26:08.058939 798414 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/5926602.pem
I0815 01:26:08.074680 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/5926602.pem /etc/ssl/certs/3ec20f2e.0"
I0815 01:26:08.087311 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0815 01:26:08.099152 798414 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0815 01:26:08.103350 798414 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 15 00:36 /usr/share/ca-certificates/minikubeCA.pem
I0815 01:26:08.103416 798414 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0815 01:26:08.110812 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0815 01:26:08.126568 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/592660.pem && ln -fs /usr/share/ca-certificates/592660.pem /etc/ssl/certs/592660.pem"
I0815 01:26:08.137368 798414 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/592660.pem
I0815 01:26:08.141357 798414 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 15 00:47 /usr/share/ca-certificates/592660.pem
I0815 01:26:08.141496 798414 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/592660.pem
I0815 01:26:08.149087 798414 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/592660.pem /etc/ssl/certs/51391683.0"
I0815 01:26:08.158887 798414 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0815 01:26:08.162750 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0815 01:26:08.169988 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0815 01:26:08.177414 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0815 01:26:08.184743 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0815 01:26:08.192504 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0815 01:26:08.199918 798414 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0815 01:26:08.207300 798414 kubeadm.go:392] StartCluster: {Name:old-k8s-version-145466 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-145466 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:26:08.207397 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0815 01:26:08.207453 798414 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0815 01:26:08.260835 798414 cri.go:89] found id: "56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:26:08.260862 798414 cri.go:89] found id: "8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:26:08.260867 798414 cri.go:89] found id: "76c1204f9e705ed1d3d61aa06410f99c08c09543c18fb107bec9e21b0ffe3046"
I0815 01:26:08.260870 798414 cri.go:89] found id: "8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:26:08.260873 798414 cri.go:89] found id: "d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:26:08.260877 798414 cri.go:89] found id: "f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:26:08.260881 798414 cri.go:89] found id: "149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:26:08.260884 798414 cri.go:89] found id: "d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:26:08.260887 798414 cri.go:89] found id: ""
I0815 01:26:08.260940 798414 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0815 01:26:08.274037 798414 cri.go:116] JSON = null
W0815 01:26:08.274103 798414 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0815 01:26:08.274162 798414 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0815 01:26:08.284122 798414 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0815 01:26:08.284139 798414 kubeadm.go:593] restartPrimaryControlPlane start ...
I0815 01:26:08.284190 798414 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0815 01:26:08.293156 798414 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0815 01:26:08.293581 798414 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-145466" does not appear in /home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:26:08.293673 798414 kubeconfig.go:62] /home/jenkins/minikube-integration/19443-587265/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-145466" cluster setting kubeconfig missing "old-k8s-version-145466" context setting]
I0815 01:26:08.293931 798414 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19443-587265/kubeconfig: {Name:mka65351b6674d2edd84b4cf38d527ec03739af8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:26:08.295085 798414 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0815 01:26:08.304747 798414 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0815 01:26:08.304777 798414 kubeadm.go:597] duration metric: took 20.632645ms to restartPrimaryControlPlane
I0815 01:26:08.304786 798414 kubeadm.go:394] duration metric: took 97.501283ms to StartCluster
I0815 01:26:08.304801 798414 settings.go:142] acquiring lock: {Name:mkf353d296e2684cbdd29a016c10a0eb45e9f213 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:26:08.304853 798414 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:26:08.305495 798414 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19443-587265/kubeconfig: {Name:mka65351b6674d2edd84b4cf38d527ec03739af8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:26:08.305682 798414 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0815 01:26:08.306095 798414 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0815 01:26:08.306170 798414 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-145466"
I0815 01:26:08.306193 798414 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-145466"
W0815 01:26:08.306199 798414 addons.go:243] addon storage-provisioner should already be in state true
I0815 01:26:08.306221 798414 host.go:66] Checking if "old-k8s-version-145466" exists ...
I0815 01:26:08.306682 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:08.307032 798414 config.go:182] Loaded profile config "old-k8s-version-145466": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0815 01:26:08.307096 798414 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-145466"
I0815 01:26:08.307141 798414 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-145466"
I0815 01:26:08.307425 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:08.307746 798414 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-145466"
I0815 01:26:08.307771 798414 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-145466"
W0815 01:26:08.307778 798414 addons.go:243] addon metrics-server should already be in state true
I0815 01:26:08.307799 798414 host.go:66] Checking if "old-k8s-version-145466" exists ...
I0815 01:26:08.308205 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:08.312167 798414 addons.go:69] Setting dashboard=true in profile "old-k8s-version-145466"
I0815 01:26:08.312208 798414 addons.go:234] Setting addon dashboard=true in "old-k8s-version-145466"
W0815 01:26:08.312215 798414 addons.go:243] addon dashboard should already be in state true
I0815 01:26:08.312262 798414 host.go:66] Checking if "old-k8s-version-145466" exists ...
I0815 01:26:08.312700 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:08.313136 798414 out.go:177] * Verifying Kubernetes components...
I0815 01:26:08.314916 798414 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:26:08.383448 798414 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0815 01:26:08.383453 798414 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0815 01:26:08.386410 798414 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0815 01:26:08.386445 798414 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0815 01:26:08.386523 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:08.386647 798414 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:08.386661 798414 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0815 01:26:08.386708 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:08.387000 798414 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0815 01:26:08.388872 798414 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0815 01:26:08.390315 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0815 01:26:08.390333 798414 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0815 01:26:08.390396 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:08.424897 798414 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-145466"
W0815 01:26:08.424919 798414 addons.go:243] addon default-storageclass should already be in state true
I0815 01:26:08.424944 798414 host.go:66] Checking if "old-k8s-version-145466" exists ...
I0815 01:26:08.425377 798414 cli_runner.go:164] Run: docker container inspect old-k8s-version-145466 --format={{.State.Status}}
I0815 01:26:08.459227 798414 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0815 01:26:08.469264 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:08.476319 798414 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-145466" to be "Ready" ...
I0815 01:26:08.481878 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:08.495313 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:08.496789 798414 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0815 01:26:08.496808 798414 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0815 01:26:08.496869 798414 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-145466
I0815 01:26:08.565241 798414 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33805 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/old-k8s-version-145466/id_rsa Username:docker}
I0815 01:26:08.643350 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0815 01:26:08.643447 798414 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0815 01:26:08.681989 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:08.706315 798414 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0815 01:26:08.706401 798414 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0815 01:26:08.754136 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0815 01:26:08.768309 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0815 01:26:08.768333 798414 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0815 01:26:08.820894 798414 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0815 01:26:08.820920 798414 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0815 01:26:08.845892 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0815 01:26:08.845992 798414 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0815 01:26:08.918811 798414 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:26:08.918895 798414 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0815 01:26:08.925269 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0815 01:26:08.925338 798414 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0815 01:26:09.014629 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:26:09.024017 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0815 01:26:09.024051 798414 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0815 01:26:09.091927 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.091965 798414 retry.go:31] will retry after 129.132835ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:09.100042 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.100072 798414 retry.go:31] will retry after 312.027235ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.110841 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0815 01:26:09.110865 798414 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0815 01:26:09.153114 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0815 01:26:09.153138 798414 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0815 01:26:09.206342 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0815 01:26:09.206362 798414 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0815 01:26:09.221594 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0815 01:26:09.251106 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.251196 798414 retry.go:31] will retry after 316.401978ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.251701 798414 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:09.251718 798414 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0815 01:26:09.318942 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0815 01:26:09.370585 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.370622 798414 retry.go:31] will retry after 431.069804ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.412926 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:09.457810 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.457843 798414 retry.go:31] will retry after 308.596157ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:09.528982 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.529026 798414 retry.go:31] will retry after 344.892212ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.568485 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0815 01:26:09.675741 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.675777 798414 retry.go:31] will retry after 520.552668ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.767028 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:09.802796 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:09.874117 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:09.897072 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:09.897108 798414 retry.go:31] will retry after 230.591454ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:10.022743 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.022783 798414 retry.go:31] will retry after 816.130672ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:10.073281 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.073317 798414 retry.go:31] will retry after 533.635614ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.128687 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:10.197190 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0815 01:26:10.234725 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.234761 798414 retry.go:31] will retry after 320.710466ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:10.328544 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.328581 798414 retry.go:31] will retry after 834.039661ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.477225 798414 node_ready.go:53] error getting node "old-k8s-version-145466": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-145466": dial tcp 192.168.85.2:8443: connect: connection refused
I0815 01:26:10.556633 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:10.607167 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:10.707780 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.707879 798414 retry.go:31] will retry after 687.49878ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:10.773917 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.773999 798414 retry.go:31] will retry after 968.333544ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.839108 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0815 01:26:10.940978 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:10.941069 798414 retry.go:31] will retry after 528.236798ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.162840 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0815 01:26:11.286411 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.286443 798414 retry.go:31] will retry after 658.949406ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.395918 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:11.470312 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0815 01:26:11.503442 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.503473 798414 retry.go:31] will retry after 925.477913ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:11.585707 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.585737 798414 retry.go:31] will retry after 856.324307ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.743233 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:11.852010 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.852048 798414 retry.go:31] will retry after 1.661820104s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:11.946312 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0815 01:26:12.058959 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:12.058992 798414 retry.go:31] will retry after 1.138577647s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:12.429690 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:12.443030 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:12.477590 798414 node_ready.go:53] error getting node "old-k8s-version-145466": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-145466": dial tcp 192.168.85.2:8443: connect: connection refused
W0815 01:26:12.573657 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:12.573692 798414 retry.go:31] will retry after 2.333911479s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:12.632843 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:12.632886 798414 retry.go:31] will retry after 1.631093453s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:13.198272 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0815 01:26:13.312854 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:13.312884 798414 retry.go:31] will retry after 1.512618801s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:13.514843 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:13.609863 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:13.609891 798414 retry.go:31] will retry after 2.074716504s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:14.264244 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0815 01:26:14.353886 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:14.353922 798414 retry.go:31] will retry after 2.26243475s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:14.477643 798414 node_ready.go:53] error getting node "old-k8s-version-145466": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-145466": dial tcp 192.168.85.2:8443: connect: connection refused
I0815 01:26:14.826520 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:26:14.908050 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0815 01:26:14.921327 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:14.921357 798414 retry.go:31] will retry after 4.245821324s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:15.013007 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:15.013047 798414 retry.go:31] will retry after 1.791379232s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:15.685026 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0815 01:26:15.799855 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:15.799895 798414 retry.go:31] will retry after 3.485169679s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:16.477700 798414 node_ready.go:53] error getting node "old-k8s-version-145466": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-145466": dial tcp 192.168.85.2:8443: connect: connection refused
I0815 01:26:16.617055 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:16.805661 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0815 01:26:16.821914 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:16.821949 798414 retry.go:31] will retry after 3.027880504s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0815 01:26:16.937177 798414 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:16.937214 798414 retry.go:31] will retry after 5.349298615s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0815 01:26:19.168288 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:26:19.285652 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0815 01:26:19.850891 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:26:22.287074 798414 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:26:28.217889 798414 node_ready.go:49] node "old-k8s-version-145466" has status "Ready":"True"
I0815 01:26:28.217914 798414 node_ready.go:38] duration metric: took 19.741560308s for node "old-k8s-version-145466" to be "Ready" ...
I0815 01:26:28.217924 798414 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0815 01:26:28.672034 798414 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-sc7dc" in "kube-system" namespace to be "Ready" ...
I0815 01:26:29.200249 798414 pod_ready.go:92] pod "coredns-74ff55c5b-sc7dc" in "kube-system" namespace has status "Ready":"True"
I0815 01:26:29.200326 798414 pod_ready.go:81] duration metric: took 528.215279ms for pod "coredns-74ff55c5b-sc7dc" in "kube-system" namespace to be "Ready" ...
I0815 01:26:29.200354 798414 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:26:29.309603 798414 pod_ready.go:92] pod "etcd-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:26:29.309677 798414 pod_ready.go:81] duration metric: took 109.299774ms for pod "etcd-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:26:29.309707 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:26:31.339217 798414 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:32.058244 798414 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (12.772541642s)
I0815 01:26:32.058489 798414 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (12.890159568s)
I0815 01:26:32.058511 798414 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-145466"
I0815 01:26:32.058680 798414 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (12.207758269s)
I0815 01:26:32.058764 798414 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (9.771660472s)
I0815 01:26:32.061346 798414 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-145466 addons enable metrics-server
I0815 01:26:32.069407 798414 out.go:177] * Enabled addons: metrics-server, storage-provisioner, dashboard, default-storageclass
I0815 01:26:32.071322 798414 addons.go:510] duration metric: took 23.765221635s for enable addons: enabled=[metrics-server storage-provisioner dashboard default-storageclass]
I0815 01:26:33.816154 798414 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:35.816278 798414 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:37.817001 798414 pod_ready.go:102] pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:39.316924 798414 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:26:39.317000 798414 pod_ready.go:81] duration metric: took 10.007268856s for pod "kube-apiserver-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:26:39.317028 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:26:41.325804 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:43.823966 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:46.323804 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:48.823753 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:51.329608 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:53.824163 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:55.825107 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:26:58.325731 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:00.337533 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:02.828916 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:05.323897 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:07.325034 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:09.824268 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:12.325043 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:14.824140 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:16.824555 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:19.336792 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:21.822762 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:23.823506 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:25.824875 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:28.323355 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:30.326045 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:32.326505 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:34.824704 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:37.324127 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:39.325365 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:41.839092 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:44.325718 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:46.824159 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:48.323803 798414 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:48.323828 798414 pod_ready.go:81] duration metric: took 1m9.006777258s for pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.323840 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-hdj25" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.328838 798414 pod_ready.go:92] pod "kube-proxy-hdj25" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:48.328866 798414 pod_ready.go:81] duration metric: took 4.98266ms for pod "kube-proxy-hdj25" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.328878 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:50.337069 798414 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:52.836311 798414 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:54.334906 798414 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:54.334933 798414 pod_ready.go:81] duration metric: took 6.006014766s for pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:54.334945 798414 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace to be "Ready" ...
I0815 01:27:56.341453 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:58.841137 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:01.341468 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:03.342277 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:05.841177 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:07.841420 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:09.841515 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:11.841903 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:14.341251 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:16.341817 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:18.905274 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:21.340938 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:23.343888 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:25.840886 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:27.841307 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:29.842382 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:31.845262 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:34.340679 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:36.340926 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:38.348235 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:40.841788 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:43.341955 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:45.343754 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:47.841262 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:50.341583 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:52.342360 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:54.842055 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:57.341767 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:59.842110 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:02.341631 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:04.841905 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:07.341412 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:09.361881 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:11.840761 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:13.845821 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:16.341158 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:18.341582 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:20.342140 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:22.842042 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:24.842165 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:27.341302 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:29.844818 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:32.340995 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:34.341661 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:36.841766 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:39.342526 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:41.841158 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:43.842484 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:46.341306 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:48.840994 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:50.842218 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:53.341747 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:55.841257 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:58.341465 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:00.380117 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:02.840765 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:05.341086 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:07.341145 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:09.341333 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:11.342252 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:13.841939 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:16.345255 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:18.843266 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:21.341071 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:23.341252 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:25.341574 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:27.842623 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:30.341285 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:32.342057 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:34.841107 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:36.841164 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:38.841855 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:41.340950 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:43.341511 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:45.342593 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:47.840983 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:49.842312 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:52.341179 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:54.841042 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:56.841647 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:59.340948 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:01.840886 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:03.841711 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:06.341111 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:08.845348 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:11.340908 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:13.340956 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:15.341002 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:17.342150 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:19.841230 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:21.841934 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:24.341985 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:26.841205 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:28.841630 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:31.341383 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:33.841991 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:36.341927 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:38.840908 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:41.340767 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:43.341898 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:45.342464 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:47.842026 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:50.341226 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:52.841594 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:54.341770 798414 pod_ready.go:81] duration metric: took 4m0.006810385s for pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace to be "Ready" ...
E0815 01:31:54.341793 798414 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0815 01:31:54.341801 798414 pod_ready.go:38] duration metric: took 5m26.123866447s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0815 01:31:54.341815 798414 api_server.go:52] waiting for apiserver process to appear ...
I0815 01:31:54.341844 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:31:54.341906 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:31:54.389104 798414 cri.go:89] found id: "5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:31:54.389129 798414 cri.go:89] found id: "d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:31:54.389135 798414 cri.go:89] found id: ""
I0815 01:31:54.389142 798414 logs.go:276] 2 containers: [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f]
I0815 01:31:54.389202 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.392818 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.396470 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:31:54.396573 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:31:54.440719 798414 cri.go:89] found id: "1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:31:54.440738 798414 cri.go:89] found id: "149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:31:54.440744 798414 cri.go:89] found id: ""
I0815 01:31:54.440751 798414 logs.go:276] 2 containers: [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d]
I0815 01:31:54.440852 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.444415 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.447805 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:31:54.447916 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:31:54.487638 798414 cri.go:89] found id: "08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:31:54.487661 798414 cri.go:89] found id: "56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:31:54.487666 798414 cri.go:89] found id: ""
I0815 01:31:54.487673 798414 logs.go:276] 2 containers: [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4]
I0815 01:31:54.487731 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.491365 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.494774 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:31:54.494846 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:31:54.556835 798414 cri.go:89] found id: "ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:31:54.556860 798414 cri.go:89] found id: "f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:31:54.556865 798414 cri.go:89] found id: ""
I0815 01:31:54.556873 798414 logs.go:276] 2 containers: [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d]
I0815 01:31:54.556935 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.560493 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.563824 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:31:54.563938 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:31:54.604088 798414 cri.go:89] found id: "65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:31:54.604161 798414 cri.go:89] found id: "8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:31:54.604173 798414 cri.go:89] found id: ""
I0815 01:31:54.604181 798414 logs.go:276] 2 containers: [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5]
I0815 01:31:54.604282 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.607763 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.611024 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:31:54.611091 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:31:54.649541 798414 cri.go:89] found id: "30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:31:54.649565 798414 cri.go:89] found id: "d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:31:54.649570 798414 cri.go:89] found id: ""
I0815 01:31:54.649577 798414 logs.go:276] 2 containers: [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f]
I0815 01:31:54.649635 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.653540 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.656990 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:31:54.657061 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:31:54.705048 798414 cri.go:89] found id: "6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:31:54.705073 798414 cri.go:89] found id: "8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:31:54.705078 798414 cri.go:89] found id: ""
I0815 01:31:54.705086 798414 logs.go:276] 2 containers: [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1]
I0815 01:31:54.705142 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.708794 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.712050 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:31:54.712119 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:31:54.761867 798414 cri.go:89] found id: "666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:31:54.761892 798414 cri.go:89] found id: ""
I0815 01:31:54.761901 798414 logs.go:276] 1 containers: [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6]
I0815 01:31:54.761956 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.765515 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:31:54.765592 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:31:54.814259 798414 cri.go:89] found id: "fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:31:54.814287 798414 cri.go:89] found id: "6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:31:54.814292 798414 cri.go:89] found id: ""
I0815 01:31:54.814299 798414 logs.go:276] 2 containers: [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414]
I0815 01:31:54.814357 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.818624 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.822284 798414 logs.go:123] Gathering logs for kube-scheduler [f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d] ...
I0815 01:31:54.822315 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:31:54.866675 798414 logs.go:123] Gathering logs for kindnet [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b] ...
I0815 01:31:54.866707 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:31:54.927529 798414 logs.go:123] Gathering logs for etcd [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43] ...
I0815 01:31:54.927578 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:31:54.977111 798414 logs.go:123] Gathering logs for coredns [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9] ...
I0815 01:31:54.977140 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:31:55.029599 798414 logs.go:123] Gathering logs for kube-proxy [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418] ...
I0815 01:31:55.029630 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:31:55.072848 798414 logs.go:123] Gathering logs for kube-controller-manager [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd] ...
I0815 01:31:55.072876 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:31:55.141657 798414 logs.go:123] Gathering logs for kube-controller-manager [d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f] ...
I0815 01:31:55.141692 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:31:55.207613 798414 logs.go:123] Gathering logs for kubelet ...
I0815 01:31:55.207649 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:31:55.272355 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206496 664 reflector.go:138] object-"kube-system"/"coredns-token-hqq2w": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-hqq2w" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.272573 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206838 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.272786 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207185 664 reflector.go:138] object-"kube-system"/"kindnet-token-pflml": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pflml" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273001 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207498 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-rl52n": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-rl52n" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273206 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207678 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273517 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.264938 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k6pcd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k6pcd" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273725 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.114283 664 reflector.go:138] object-"default"/"default-token-j4wgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-j4wgn" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.276976 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.390721 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.278325 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.415771 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.281690 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:42 old-k8s-version-145466 kubelet[664]: E0815 01:26:42.146871 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.282438 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:49 old-k8s-version-145466 kubelet[664]: E0815 01:26:49.166497 664 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-fksp6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-fksp6" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.283970 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:57 old-k8s-version-145466 kubelet[664]: E0815 01:26:57.141841 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.284430 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:00 old-k8s-version-145466 kubelet[664]: E0815 01:27:00.751687 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.284885 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:01 old-k8s-version-145466 kubelet[664]: E0815 01:27:01.756479 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.285325 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:02 old-k8s-version-145466 kubelet[664]: E0815 01:27:02.761739 664 pod_workers.go:191] Error syncing pod d7e84c38-c90e-427c-bfc5-45adf788d6fe ("storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"
W0815 01:31:55.285983 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:08 old-k8s-version-145466 kubelet[664]: E0815 01:27:08.528394 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.288438 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:11 old-k8s-version-145466 kubelet[664]: E0815 01:27:11.140512 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.289156 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:20 old-k8s-version-145466 kubelet[664]: E0815 01:27:20.809846 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.289339 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:24 old-k8s-version-145466 kubelet[664]: E0815 01:27:24.133897 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.289665 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:28 old-k8s-version-145466 kubelet[664]: E0815 01:27:28.528406 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.289850 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:38 old-k8s-version-145466 kubelet[664]: E0815 01:27:38.186907 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.290435 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:42 old-k8s-version-145466 kubelet[664]: E0815 01:27:42.877822 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.290766 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:48 old-k8s-version-145466 kubelet[664]: E0815 01:27:48.529042 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.293220 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:53 old-k8s-version-145466 kubelet[664]: E0815 01:27:53.141943 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.293549 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:02 old-k8s-version-145466 kubelet[664]: E0815 01:28:02.132198 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.293733 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:04 old-k8s-version-145466 kubelet[664]: E0815 01:28:04.133346 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.293928 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:16 old-k8s-version-145466 kubelet[664]: E0815 01:28:16.144812 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.294252 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:17 old-k8s-version-145466 kubelet[664]: E0815 01:28:17.132160 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.294839 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:29 old-k8s-version-145466 kubelet[664]: E0815 01:28:29.028736 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.295025 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:31 old-k8s-version-145466 kubelet[664]: E0815 01:28:31.132458 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.295352 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:38 old-k8s-version-145466 kubelet[664]: E0815 01:28:38.528400 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.295538 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:44 old-k8s-version-145466 kubelet[664]: E0815 01:28:44.132479 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.295872 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:51 old-k8s-version-145466 kubelet[664]: E0815 01:28:51.133407 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.296062 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:56 old-k8s-version-145466 kubelet[664]: E0815 01:28:56.135291 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.296387 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:04 old-k8s-version-145466 kubelet[664]: E0815 01:29:04.132344 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.296569 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:10 old-k8s-version-145466 kubelet[664]: E0815 01:29:10.132536 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.296893 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:17 old-k8s-version-145466 kubelet[664]: E0815 01:29:17.132162 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.299321 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:23 old-k8s-version-145466 kubelet[664]: E0815 01:29:23.139752 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.299673 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:28 old-k8s-version-145466 kubelet[664]: E0815 01:29:28.139064 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.299863 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:34 old-k8s-version-145466 kubelet[664]: E0815 01:29:34.132664 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.300190 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:40 old-k8s-version-145466 kubelet[664]: E0815 01:29:40.133096 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.300379 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:49 old-k8s-version-145466 kubelet[664]: E0815 01:29:49.133068 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.300968 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:51 old-k8s-version-145466 kubelet[664]: E0815 01:29:51.281771 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301293 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:58 old-k8s-version-145466 kubelet[664]: E0815 01:29:58.528848 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301485 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:00 old-k8s-version-145466 kubelet[664]: E0815 01:30:00.160446 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.301814 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:10 old-k8s-version-145466 kubelet[664]: E0815 01:30:10.135730 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301998 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:11 old-k8s-version-145466 kubelet[664]: E0815 01:30:11.132453 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.302337 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:22 old-k8s-version-145466 kubelet[664]: E0815 01:30:22.132670 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.302530 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:26 old-k8s-version-145466 kubelet[664]: E0815 01:30:26.135386 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.302859 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:33 old-k8s-version-145466 kubelet[664]: E0815 01:30:33.132520 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.303041 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:41 old-k8s-version-145466 kubelet[664]: E0815 01:30:41.132604 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.303369 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: E0815 01:30:45.132399 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.303551 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:54 old-k8s-version-145466 kubelet[664]: E0815 01:30:54.132932 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.303895 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: E0815 01:30:56.132938 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.304080 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:06 old-k8s-version-145466 kubelet[664]: E0815 01:31:06.132954 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.304404 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: E0815 01:31:11.132193 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.304589 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:20 old-k8s-version-145466 kubelet[664]: E0815 01:31:20.132734 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.304913 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: E0815 01:31:23.132188 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.305096 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.305420 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.305603 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.305928 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.306109 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0815 01:31:55.306119 798414 logs.go:123] Gathering logs for kube-apiserver [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645] ...
I0815 01:31:55.306135 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:31:55.368137 798414 logs.go:123] Gathering logs for kindnet [8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1] ...
I0815 01:31:55.368175 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:31:55.418716 798414 logs.go:123] Gathering logs for kubernetes-dashboard [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6] ...
I0815 01:31:55.418749 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:31:55.463235 798414 logs.go:123] Gathering logs for storage-provisioner [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3] ...
I0815 01:31:55.463266 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:31:55.524586 798414 logs.go:123] Gathering logs for container status ...
I0815 01:31:55.524614 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:31:55.586139 798414 logs.go:123] Gathering logs for etcd [149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d] ...
I0815 01:31:55.586167 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:31:55.628448 798414 logs.go:123] Gathering logs for kube-scheduler [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8] ...
I0815 01:31:55.628476 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:31:55.667804 798414 logs.go:123] Gathering logs for kube-apiserver [d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f] ...
I0815 01:31:55.667828 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:31:55.730471 798414 logs.go:123] Gathering logs for coredns [56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4] ...
I0815 01:31:55.730513 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:31:55.779800 798414 logs.go:123] Gathering logs for kube-proxy [8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5] ...
I0815 01:31:55.779833 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:31:55.823656 798414 logs.go:123] Gathering logs for storage-provisioner [6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414] ...
I0815 01:31:55.823733 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:31:55.861500 798414 logs.go:123] Gathering logs for containerd ...
I0815 01:31:55.861528 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:31:55.921791 798414 logs.go:123] Gathering logs for dmesg ...
I0815 01:31:55.921824 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:31:55.943979 798414 logs.go:123] Gathering logs for describe nodes ...
I0815 01:31:55.944021 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:31:56.104386 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:56.104411 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:31:56.104602 798414 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0815 01:31:56.104620 798414 out.go:239] Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:56.104640 798414 out.go:239] Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:56.104655 798414 out.go:239] Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:56.104675 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:56.104690 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0815 01:31:56.104697 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:56.104709 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:32:06.105579 798414 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0815 01:32:06.118752 798414 api_server.go:72] duration metric: took 5m57.813041637s to wait for apiserver process to appear ...
I0815 01:32:06.118774 798414 api_server.go:88] waiting for apiserver healthz status ...
I0815 01:32:06.118810 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:32:06.118865 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:32:06.165617 798414 cri.go:89] found id: "5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:32:06.165640 798414 cri.go:89] found id: "d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:32:06.165646 798414 cri.go:89] found id: ""
I0815 01:32:06.165653 798414 logs.go:276] 2 containers: [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f]
I0815 01:32:06.165708 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.169565 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.173032 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:32:06.173102 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:32:06.214710 798414 cri.go:89] found id: "1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:32:06.214731 798414 cri.go:89] found id: "149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:32:06.214735 798414 cri.go:89] found id: ""
I0815 01:32:06.214743 798414 logs.go:276] 2 containers: [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d]
I0815 01:32:06.214801 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.219522 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.223511 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:32:06.223588 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:32:06.275179 798414 cri.go:89] found id: "08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:32:06.275200 798414 cri.go:89] found id: "56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:32:06.275205 798414 cri.go:89] found id: ""
I0815 01:32:06.275212 798414 logs.go:276] 2 containers: [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4]
I0815 01:32:06.275271 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.279316 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.283547 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:32:06.283635 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:32:06.325027 798414 cri.go:89] found id: "ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:32:06.325047 798414 cri.go:89] found id: "f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:32:06.325051 798414 cri.go:89] found id: ""
I0815 01:32:06.325059 798414 logs.go:276] 2 containers: [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d]
I0815 01:32:06.325114 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.328921 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.332566 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:32:06.332658 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:32:06.374132 798414 cri.go:89] found id: "65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:32:06.374165 798414 cri.go:89] found id: "8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:32:06.374173 798414 cri.go:89] found id: ""
I0815 01:32:06.374184 798414 logs.go:276] 2 containers: [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5]
I0815 01:32:06.374246 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.379778 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.383769 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:32:06.383887 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:32:06.424513 798414 cri.go:89] found id: "30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:32:06.424585 798414 cri.go:89] found id: "d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:32:06.424598 798414 cri.go:89] found id: ""
I0815 01:32:06.424607 798414 logs.go:276] 2 containers: [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f]
I0815 01:32:06.424671 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.428875 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.433119 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:32:06.433275 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:32:06.478733 798414 cri.go:89] found id: "6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:32:06.478760 798414 cri.go:89] found id: "8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:32:06.478767 798414 cri.go:89] found id: ""
I0815 01:32:06.478775 798414 logs.go:276] 2 containers: [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1]
I0815 01:32:06.478845 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.482927 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.486698 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:32:06.486788 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:32:06.536723 798414 cri.go:89] found id: "666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:32:06.536748 798414 cri.go:89] found id: ""
I0815 01:32:06.536757 798414 logs.go:276] 1 containers: [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6]
I0815 01:32:06.536832 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.540620 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:32:06.540726 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:32:06.577792 798414 cri.go:89] found id: "fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:32:06.577814 798414 cri.go:89] found id: "6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:32:06.577820 798414 cri.go:89] found id: ""
I0815 01:32:06.577827 798414 logs.go:276] 2 containers: [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414]
I0815 01:32:06.577881 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.581525 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.585154 798414 logs.go:123] Gathering logs for kube-proxy [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418] ...
I0815 01:32:06.585180 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:32:06.629579 798414 logs.go:123] Gathering logs for kubernetes-dashboard [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6] ...
I0815 01:32:06.629604 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:32:06.671944 798414 logs.go:123] Gathering logs for storage-provisioner [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3] ...
I0815 01:32:06.671982 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:32:06.713621 798414 logs.go:123] Gathering logs for container status ...
I0815 01:32:06.713664 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:32:06.771192 798414 logs.go:123] Gathering logs for describe nodes ...
I0815 01:32:06.771223 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:32:06.916807 798414 logs.go:123] Gathering logs for coredns [56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4] ...
I0815 01:32:06.916881 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:32:06.967152 798414 logs.go:123] Gathering logs for kube-scheduler [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8] ...
I0815 01:32:06.967233 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:32:07.009506 798414 logs.go:123] Gathering logs for kube-proxy [8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5] ...
I0815 01:32:07.009537 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:32:07.049321 798414 logs.go:123] Gathering logs for containerd ...
I0815 01:32:07.049360 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:32:07.113454 798414 logs.go:123] Gathering logs for kubelet ...
I0815 01:32:07.113494 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:32:07.177310 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206496 664 reflector.go:138] object-"kube-system"/"coredns-token-hqq2w": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-hqq2w" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177533 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206838 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177748 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207185 664 reflector.go:138] object-"kube-system"/"kindnet-token-pflml": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pflml" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177967 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207498 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-rl52n": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-rl52n" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178175 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207678 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178488 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.264938 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k6pcd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k6pcd" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178700 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.114283 664 reflector.go:138] object-"default"/"default-token-j4wgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-j4wgn" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.181924 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.390721 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.183276 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.415771 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.186634 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:42 old-k8s-version-145466 kubelet[664]: E0815 01:26:42.146871 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.187378 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:49 old-k8s-version-145466 kubelet[664]: E0815 01:26:49.166497 664 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-fksp6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-fksp6" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.188913 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:57 old-k8s-version-145466 kubelet[664]: E0815 01:26:57.141841 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.189372 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:00 old-k8s-version-145466 kubelet[664]: E0815 01:27:00.751687 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.189828 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:01 old-k8s-version-145466 kubelet[664]: E0815 01:27:01.756479 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.190265 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:02 old-k8s-version-145466 kubelet[664]: E0815 01:27:02.761739 664 pod_workers.go:191] Error syncing pod d7e84c38-c90e-427c-bfc5-45adf788d6fe ("storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"
W0815 01:32:07.190931 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:08 old-k8s-version-145466 kubelet[664]: E0815 01:27:08.528394 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.193356 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:11 old-k8s-version-145466 kubelet[664]: E0815 01:27:11.140512 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.194070 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:20 old-k8s-version-145466 kubelet[664]: E0815 01:27:20.809846 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.194257 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:24 old-k8s-version-145466 kubelet[664]: E0815 01:27:24.133897 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.194585 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:28 old-k8s-version-145466 kubelet[664]: E0815 01:27:28.528406 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.194768 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:38 old-k8s-version-145466 kubelet[664]: E0815 01:27:38.186907 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.195356 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:42 old-k8s-version-145466 kubelet[664]: E0815 01:27:42.877822 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.195681 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:48 old-k8s-version-145466 kubelet[664]: E0815 01:27:48.529042 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.198128 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:53 old-k8s-version-145466 kubelet[664]: E0815 01:27:53.141943 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.198456 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:02 old-k8s-version-145466 kubelet[664]: E0815 01:28:02.132198 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.198668 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:04 old-k8s-version-145466 kubelet[664]: E0815 01:28:04.133346 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.198852 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:16 old-k8s-version-145466 kubelet[664]: E0815 01:28:16.144812 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.199177 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:17 old-k8s-version-145466 kubelet[664]: E0815 01:28:17.132160 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.199766 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:29 old-k8s-version-145466 kubelet[664]: E0815 01:28:29.028736 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.199975 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:31 old-k8s-version-145466 kubelet[664]: E0815 01:28:31.132458 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.200308 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:38 old-k8s-version-145466 kubelet[664]: E0815 01:28:38.528400 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.200493 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:44 old-k8s-version-145466 kubelet[664]: E0815 01:28:44.132479 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.200817 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:51 old-k8s-version-145466 kubelet[664]: E0815 01:28:51.133407 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.201005 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:56 old-k8s-version-145466 kubelet[664]: E0815 01:28:56.135291 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.201329 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:04 old-k8s-version-145466 kubelet[664]: E0815 01:29:04.132344 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.201512 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:10 old-k8s-version-145466 kubelet[664]: E0815 01:29:10.132536 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.201837 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:17 old-k8s-version-145466 kubelet[664]: E0815 01:29:17.132162 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.204300 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:23 old-k8s-version-145466 kubelet[664]: E0815 01:29:23.139752 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.204628 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:28 old-k8s-version-145466 kubelet[664]: E0815 01:29:28.139064 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.204812 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:34 old-k8s-version-145466 kubelet[664]: E0815 01:29:34.132664 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.205154 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:40 old-k8s-version-145466 kubelet[664]: E0815 01:29:40.133096 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.205337 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:49 old-k8s-version-145466 kubelet[664]: E0815 01:29:49.133068 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.205925 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:51 old-k8s-version-145466 kubelet[664]: E0815 01:29:51.281771 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206255 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:58 old-k8s-version-145466 kubelet[664]: E0815 01:29:58.528848 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206438 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:00 old-k8s-version-145466 kubelet[664]: E0815 01:30:00.160446 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.206767 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:10 old-k8s-version-145466 kubelet[664]: E0815 01:30:10.135730 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206949 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:11 old-k8s-version-145466 kubelet[664]: E0815 01:30:11.132453 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.207272 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:22 old-k8s-version-145466 kubelet[664]: E0815 01:30:22.132670 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.207454 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:26 old-k8s-version-145466 kubelet[664]: E0815 01:30:26.135386 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.207781 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:33 old-k8s-version-145466 kubelet[664]: E0815 01:30:33.132520 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.207971 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:41 old-k8s-version-145466 kubelet[664]: E0815 01:30:41.132604 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.208297 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: E0815 01:30:45.132399 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.208479 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:54 old-k8s-version-145466 kubelet[664]: E0815 01:30:54.132932 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.208803 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: E0815 01:30:56.132938 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.208985 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:06 old-k8s-version-145466 kubelet[664]: E0815 01:31:06.132954 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.209308 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: E0815 01:31:11.132193 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.209490 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:20 old-k8s-version-145466 kubelet[664]: E0815 01:31:20.132734 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.209814 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: E0815 01:31:23.132188 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.209997 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.210321 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.210506 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.210833 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.211019 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.213451 798414 logs.go:138] Found kubelet problem: Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.213780 798414 logs.go:138] Found kubelet problem: Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
I0815 01:32:07.213790 798414 logs.go:123] Gathering logs for etcd [149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d] ...
I0815 01:32:07.213804 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:32:07.268209 798414 logs.go:123] Gathering logs for coredns [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9] ...
I0815 01:32:07.268240 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:32:07.312267 798414 logs.go:123] Gathering logs for kube-scheduler [f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d] ...
I0815 01:32:07.312297 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:32:07.353856 798414 logs.go:123] Gathering logs for kube-controller-manager [d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f] ...
I0815 01:32:07.353886 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:32:07.412945 798414 logs.go:123] Gathering logs for kindnet [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b] ...
I0815 01:32:07.412980 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:32:07.480680 798414 logs.go:123] Gathering logs for etcd [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43] ...
I0815 01:32:07.480758 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:32:07.557927 798414 logs.go:123] Gathering logs for kube-controller-manager [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd] ...
I0815 01:32:07.557960 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:32:07.614762 798414 logs.go:123] Gathering logs for kindnet [8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1] ...
I0815 01:32:07.614799 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:32:07.664776 798414 logs.go:123] Gathering logs for storage-provisioner [6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414] ...
I0815 01:32:07.664822 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:32:07.703095 798414 logs.go:123] Gathering logs for dmesg ...
I0815 01:32:07.703130 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:32:07.722145 798414 logs.go:123] Gathering logs for kube-apiserver [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645] ...
I0815 01:32:07.722176 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:32:07.778182 798414 logs.go:123] Gathering logs for kube-apiserver [d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f] ...
I0815 01:32:07.778222 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:32:07.835453 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:07.835484 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:32:07.835549 798414 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0815 01:32:07.835563 798414 out.go:239] Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.835572 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.835580 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.835587 798414 out.go:239] Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.835601 798414 out.go:239] Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
I0815 01:32:07.835607 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:07.835621 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:32:17.836740 798414 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0815 01:32:17.849082 798414 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0815 01:32:17.851627 798414 out.go:177]
W0815 01:32:17.854581 798414 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0815 01:32:17.854634 798414 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0815 01:32:17.854660 798414 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0815 01:32:17.854666 798414 out.go:239] *
*
W0815 01:32:17.856250 798414 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0815 01:32:17.863608 798414 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-145466 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-145466
helpers_test.go:235: (dbg) docker inspect old-k8s-version-145466:
-- stdout --
[
{
"Id": "5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278",
"Created": "2024-08-15T01:23:08.661953003Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 798615,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-08-15T01:26:00.297919427Z",
"FinishedAt": "2024-08-15T01:25:59.061047456Z"
},
"Image": "sha256:2b339a1cac4376103734d3066f7ccdf0ac7377a2f8f8d5eb9e81c29f3abcec50",
"ResolvConfPath": "/var/lib/docker/containers/5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278/hostname",
"HostsPath": "/var/lib/docker/containers/5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278/hosts",
"LogPath": "/var/lib/docker/containers/5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278/5855b1f20999a15761f9b4ac6911a8fc19051ec20e47fbd1c365f58751ca8278-json.log",
"Name": "/old-k8s-version-145466",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-145466:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-145466",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/8fa07c694dbba46c257c73caaea090e7f9f192a3c9dd4e4f0bad3bd9474b4a81-init/diff:/var/lib/docker/overlay2/724d641fa67867c1f8a89bb3b136ff9997d84663650d206cbef2b533f5f97838/diff",
"MergedDir": "/var/lib/docker/overlay2/8fa07c694dbba46c257c73caaea090e7f9f192a3c9dd4e4f0bad3bd9474b4a81/merged",
"UpperDir": "/var/lib/docker/overlay2/8fa07c694dbba46c257c73caaea090e7f9f192a3c9dd4e4f0bad3bd9474b4a81/diff",
"WorkDir": "/var/lib/docker/overlay2/8fa07c694dbba46c257c73caaea090e7f9f192a3c9dd4e4f0bad3bd9474b4a81/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-145466",
"Source": "/var/lib/docker/volumes/old-k8s-version-145466/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-145466",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-145466",
"name.minikube.sigs.k8s.io": "old-k8s-version-145466",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "7877f4e547154d3239ffa605a29802b604e1c5b2477aa2ae911f68aa14b9b494",
"SandboxKey": "/var/run/docker/netns/7877f4e54715",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33805"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33806"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33809"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33807"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "33808"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-145466": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:55:02",
"DriverOpts": null,
"NetworkID": "4910ff7e81efe82fbc19afffd0db73a9ac7793c2a634229a5e83dd7ddb6a42c4",
"EndpointID": "c0970a44525f4d3927e0189bac16e1fd17777f2b40c205c605203064c591de0f",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-145466",
"5855b1f20999"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-145466 -n old-k8s-version-145466
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-145466 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-145466 logs -n 25: (2.030709434s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
| ssh | -p cilium-404506 sudo find | cilium-404506 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | |
| | /etc/crio -type f -exec sh -c | | | | | |
| | 'echo {}; cat {}' \; | | | | | |
| ssh | -p cilium-404506 sudo crio | cilium-404506 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | |
| | config | | | | | |
| delete | -p cilium-404506 | cilium-404506 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | 15 Aug 24 01:21 UTC |
| start | -p force-systemd-env-673385 | force-systemd-env-673385 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | 15 Aug 24 01:22 UTC |
| | --memory=2048 | | | | | |
| | --alsologtostderr | | | | | |
| | -v=5 --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-flag-246341 | force-systemd-flag-246341 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | 15 Aug 24 01:21 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-flag-246341 | force-systemd-flag-246341 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | 15 Aug 24 01:21 UTC |
| start | -p cert-expiration-480110 | cert-expiration-480110 | jenkins | v1.33.1 | 15 Aug 24 01:21 UTC | 15 Aug 24 01:22 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-673385 | force-systemd-env-673385 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:22 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-673385 | force-systemd-env-673385 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:22 UTC |
| start | -p cert-options-476187 | cert-options-476187 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:22 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-476187 ssh | cert-options-476187 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:22 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-476187 -- sudo | cert-options-476187 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:22 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-476187 | cert-options-476187 | jenkins | v1.33.1 | 15 Aug 24 01:22 UTC | 15 Aug 24 01:23 UTC |
| start | -p old-k8s-version-145466 | old-k8s-version-145466 | jenkins | v1.33.1 | 15 Aug 24 01:23 UTC | 15 Aug 24 01:25 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-480110 | cert-expiration-480110 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:25 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-480110 | cert-expiration-480110 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:25 UTC |
| start | -p no-preload-891255 | no-preload-891255 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:27 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
| addons | enable metrics-server -p old-k8s-version-145466 | old-k8s-version-145466 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:25 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-145466 | old-k8s-version-145466 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:25 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-145466 | old-k8s-version-145466 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | 15 Aug 24 01:25 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-145466 | old-k8s-version-145466 | jenkins | v1.33.1 | 15 Aug 24 01:25 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-891255 | no-preload-891255 | jenkins | v1.33.1 | 15 Aug 24 01:27 UTC | 15 Aug 24 01:27 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-891255 | no-preload-891255 | jenkins | v1.33.1 | 15 Aug 24 01:27 UTC | 15 Aug 24 01:27 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-891255 | no-preload-891255 | jenkins | v1.33.1 | 15 Aug 24 01:27 UTC | 15 Aug 24 01:27 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-891255 | no-preload-891255 | jenkins | v1.33.1 | 15 Aug 24 01:27 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
|---------|--------------------------------------------------------|---------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/08/15 01:27:30
Running on machine: ip-172-31-31-251
Binary: Built with gc go1.22.5 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0815 01:27:30.217561 803761 out.go:291] Setting OutFile to fd 1 ...
I0815 01:27:30.217701 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:27:30.217713 803761 out.go:304] Setting ErrFile to fd 2...
I0815 01:27:30.217720 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:27:30.217982 803761 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19443-587265/.minikube/bin
I0815 01:27:30.218444 803761 out.go:298] Setting JSON to false
I0815 01:27:30.219594 803761 start.go:129] hostinfo: {"hostname":"ip-172-31-31-251","uptime":18573,"bootTime":1723666678,"procs":246,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1067-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"982e3628-3742-4b3e-bb63-ac1b07660ec7"}
I0815 01:27:30.219680 803761 start.go:139] virtualization:
I0815 01:27:30.223461 803761 out.go:177] * [no-preload-891255] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0815 01:27:30.225728 803761 out.go:177] - MINIKUBE_LOCATION=19443
I0815 01:27:30.225828 803761 notify.go:220] Checking for updates...
I0815 01:27:30.228839 803761 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0815 01:27:30.230616 803761 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:27:30.233073 803761 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19443-587265/.minikube
I0815 01:27:30.235130 803761 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0815 01:27:30.236762 803761 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0815 01:27:30.238942 803761 config.go:182] Loaded profile config "no-preload-891255": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.0
I0815 01:27:30.239507 803761 driver.go:392] Setting default libvirt URI to qemu:///system
I0815 01:27:30.262308 803761 docker.go:123] docker version: linux-27.1.2:Docker Engine - Community
I0815 01:27:30.262449 803761 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0815 01:27:30.352647 803761 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-15 01:27:30.342788552 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0815 01:27:30.352759 803761 docker.go:307] overlay module found
I0815 01:27:30.355397 803761 out.go:177] * Using the docker driver based on existing profile
I0815 01:27:30.358421 803761 start.go:297] selected driver: docker
I0815 01:27:30.358441 803761 start.go:901] validating driver "docker" against &{Name:no-preload-891255 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-891255 Namespace:default APIServerHAVIP: APIServerNa
me:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false Mount
String:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:27:30.358536 803761 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0815 01:27:30.359361 803761 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0815 01:27:30.427324 803761 info.go:266] docker info: {ID:EOU5:DNGX:XN6V:L2FZ:UXRM:5TWK:EVUR:KC2F:GT7Z:Y4O4:GB77:5PD3 Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:41 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-15 01:27:30.41837525 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aarc
h64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-31-251 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerError
s:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0815 01:27:30.427717 803761 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0815 01:27:30.427775 803761 cni.go:84] Creating CNI manager for ""
I0815 01:27:30.427791 803761 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0815 01:27:30.427896 803761 start.go:340] cluster config:
{Name:no-preload-891255 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-891255 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Containe
rRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker Moun
tIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:27:30.430593 803761 out.go:177] * Starting "no-preload-891255" primary control-plane node in "no-preload-891255" cluster
I0815 01:27:30.432721 803761 cache.go:121] Beginning downloading kic base image for docker with containerd
I0815 01:27:30.434488 803761 out.go:177] * Pulling base image v0.0.44-1723650208-19443 ...
I0815 01:27:30.436306 803761 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime containerd
I0815 01:27:30.436385 803761 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local docker daemon
I0815 01:27:30.436456 803761 profile.go:143] Saving config to /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/config.json ...
I0815 01:27:30.436939 803761 cache.go:107] acquiring lock: {Name:mk377bae187156e25e03e74dc26a8fc6b664c12f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437025 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 exists
I0815 01:27:30.437039 803761 cache.go:96] cache image "gcr.io/k8s-minikube/storage-provisioner:v5" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5" took 108.956µs
I0815 01:27:30.437048 803761 cache.go:80] save to tar file gcr.io/k8s-minikube/storage-provisioner:v5 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/gcr.io/k8s-minikube/storage-provisioner_v5 succeeded
I0815 01:27:30.437058 803761 cache.go:107] acquiring lock: {Name:mkfa33fd52f81b72095b4a245252d467804de46a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437094 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0 exists
I0815 01:27:30.437104 803761 cache.go:96] cache image "registry.k8s.io/kube-apiserver:v1.31.0" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0" took 47.212µs
I0815 01:27:30.437111 803761 cache.go:80] save to tar file registry.k8s.io/kube-apiserver:v1.31.0 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-apiserver_v1.31.0 succeeded
I0815 01:27:30.437121 803761 cache.go:107] acquiring lock: {Name:mkc20c405e053840f9847268b96de1815177ab97 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437152 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0 exists
I0815 01:27:30.437162 803761 cache.go:96] cache image "registry.k8s.io/kube-controller-manager:v1.31.0" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0" took 42.626µs
I0815 01:27:30.437220 803761 cache.go:80] save to tar file registry.k8s.io/kube-controller-manager:v1.31.0 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-controller-manager_v1.31.0 succeeded
I0815 01:27:30.437232 803761 cache.go:107] acquiring lock: {Name:mk516ca1588a985afdddcbddef9d2e8e029d52ba Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437338 803761 cache.go:107] acquiring lock: {Name:mk571f6e7ebb7534db4e04b364c3f2c216aedb9b Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437388 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 exists
I0815 01:27:30.437400 803761 cache.go:96] cache image "registry.k8s.io/pause:3.10" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10" took 65.231µs
I0815 01:27:30.437406 803761 cache.go:80] save to tar file registry.k8s.io/pause:3.10 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/pause_3.10 succeeded
I0815 01:27:30.437416 803761 cache.go:107] acquiring lock: {Name:mk1e6bd7ba33ff5f26626ccd6b8c25445619a36a Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437448 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0 exists
I0815 01:27:30.437457 803761 cache.go:96] cache image "registry.k8s.io/kube-proxy:v1.31.0" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0" took 42.305µs
I0815 01:27:30.437464 803761 cache.go:80] save to tar file registry.k8s.io/kube-proxy:v1.31.0 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-proxy_v1.31.0 succeeded
I0815 01:27:30.437475 803761 cache.go:107] acquiring lock: {Name:mkc84118c26dca31c17232db577c88bba77012fe Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437505 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 exists
I0815 01:27:30.437515 803761 cache.go:96] cache image "registry.k8s.io/etcd:3.5.15-0" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0" took 41.378µs
I0815 01:27:30.437521 803761 cache.go:80] save to tar file registry.k8s.io/etcd:3.5.15-0 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/etcd_3.5.15-0 succeeded
I0815 01:27:30.437531 803761 cache.go:107] acquiring lock: {Name:mk72ede4086bb42d67cd2dd687eaa1c3b51091cd Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.437594 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 exists
I0815 01:27:30.437608 803761 cache.go:96] cache image "registry.k8s.io/coredns/coredns:v1.11.1" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1" took 78.179µs
I0815 01:27:30.437616 803761 cache.go:80] save to tar file registry.k8s.io/coredns/coredns:v1.11.1 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/coredns/coredns_v1.11.1 succeeded
I0815 01:27:30.437639 803761 cache.go:115] /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0 exists
I0815 01:27:30.437649 803761 cache.go:96] cache image "registry.k8s.io/kube-scheduler:v1.31.0" -> "/home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0" took 418.904µs
I0815 01:27:30.437655 803761 cache.go:80] save to tar file registry.k8s.io/kube-scheduler:v1.31.0 -> /home/jenkins/minikube-integration/19443-587265/.minikube/cache/images/arm64/registry.k8s.io/kube-scheduler_v1.31.0 succeeded
I0815 01:27:30.437691 803761 cache.go:87] Successfully saved all images to host disk.
W0815 01:27:30.455772 803761 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 is of wrong architecture
I0815 01:27:30.455794 803761 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 to local cache
I0815 01:27:30.455975 803761 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local cache directory
I0815 01:27:30.456590 803761 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 in local cache directory, skipping pull
I0815 01:27:30.456601 803761 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 exists in cache, skipping pull
I0815 01:27:30.456613 803761 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 as a tarball
I0815 01:27:30.456619 803761 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 from local cache
I0815 01:27:30.588228 803761 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 from cached tarball
I0815 01:27:30.588267 803761 cache.go:194] Successfully downloaded all kic artifacts
I0815 01:27:30.588311 803761 start.go:360] acquireMachinesLock for no-preload-891255: {Name:mkbcde4c98f1feca8d388937567a29bcafc800e7 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0815 01:27:30.588410 803761 start.go:364] duration metric: took 74.839µs to acquireMachinesLock for "no-preload-891255"
I0815 01:27:30.588434 803761 start.go:96] Skipping create...Using existing machine configuration
I0815 01:27:30.588445 803761 fix.go:54] fixHost starting:
I0815 01:27:30.588725 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:30.604885 803761 fix.go:112] recreateIfNeeded on no-preload-891255: state=Stopped err=<nil>
W0815 01:27:30.604918 803761 fix.go:138] unexpected machine state, will restart: <nil>
I0815 01:27:30.607211 803761 out.go:177] * Restarting existing docker container for "no-preload-891255" ...
I0815 01:27:30.326045 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:32.326505 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:30.609064 803761 cli_runner.go:164] Run: docker start no-preload-891255
I0815 01:27:30.946277 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:30.973780 803761 kic.go:430] container "no-preload-891255" state is running.
I0815 01:27:30.975532 803761 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-891255
I0815 01:27:30.997578 803761 profile.go:143] Saving config to /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/config.json ...
I0815 01:27:30.997806 803761 machine.go:94] provisionDockerMachine start ...
I0815 01:27:30.997872 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:31.020856 803761 main.go:141] libmachine: Using SSH client type: native
I0815 01:27:31.021125 803761 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33810 <nil> <nil>}
I0815 01:27:31.021140 803761 main.go:141] libmachine: About to run SSH command:
hostname
I0815 01:27:31.021719 803761 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:45724->127.0.0.1:33810: read: connection reset by peer
I0815 01:27:34.159503 803761 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-891255
I0815 01:27:34.159591 803761 ubuntu.go:169] provisioning hostname "no-preload-891255"
I0815 01:27:34.159697 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:34.176612 803761 main.go:141] libmachine: Using SSH client type: native
I0815 01:27:34.176871 803761 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33810 <nil> <nil>}
I0815 01:27:34.176890 803761 main.go:141] libmachine: About to run SSH command:
sudo hostname no-preload-891255 && echo "no-preload-891255" | sudo tee /etc/hostname
I0815 01:27:34.331324 803761 main.go:141] libmachine: SSH cmd err, output: <nil>: no-preload-891255
I0815 01:27:34.331489 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:34.350487 803761 main.go:141] libmachine: Using SSH client type: native
I0815 01:27:34.350750 803761 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 33810 <nil> <nil>}
I0815 01:27:34.350767 803761 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sno-preload-891255' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 no-preload-891255/g' /etc/hosts;
else
echo '127.0.1.1 no-preload-891255' | sudo tee -a /etc/hosts;
fi
fi
I0815 01:27:34.488067 803761 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0815 01:27:34.488103 803761 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19443-587265/.minikube CaCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19443-587265/.minikube}
I0815 01:27:34.488127 803761 ubuntu.go:177] setting up certificates
I0815 01:27:34.488137 803761 provision.go:84] configureAuth start
I0815 01:27:34.488196 803761 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-891255
I0815 01:27:34.505440 803761 provision.go:143] copyHostCerts
I0815 01:27:34.505507 803761 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem, removing ...
I0815 01:27:34.505523 803761 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem
I0815 01:27:34.505607 803761 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/ca.pem (1082 bytes)
I0815 01:27:34.505727 803761 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem, removing ...
I0815 01:27:34.505740 803761 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem
I0815 01:27:34.505770 803761 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/cert.pem (1123 bytes)
I0815 01:27:34.505839 803761 exec_runner.go:144] found /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem, removing ...
I0815 01:27:34.505849 803761 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem
I0815 01:27:34.505875 803761 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19443-587265/.minikube/key.pem (1675 bytes)
I0815 01:27:34.505937 803761 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem org=jenkins.no-preload-891255 san=[127.0.0.1 192.168.76.2 localhost minikube no-preload-891255]
I0815 01:27:34.765073 803761 provision.go:177] copyRemoteCerts
I0815 01:27:34.765164 803761 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0815 01:27:34.765236 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:34.782604 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:34.882448 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0815 01:27:34.911055 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0815 01:27:34.938186 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0815 01:27:34.970893 803761 provision.go:87] duration metric: took 482.741715ms to configureAuth
I0815 01:27:34.970922 803761 ubuntu.go:193] setting minikube options for container-runtime
I0815 01:27:34.971125 803761 config.go:182] Loaded profile config "no-preload-891255": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.0
I0815 01:27:34.971137 803761 machine.go:97] duration metric: took 3.973319383s to provisionDockerMachine
I0815 01:27:34.971145 803761 start.go:293] postStartSetup for "no-preload-891255" (driver="docker")
I0815 01:27:34.971159 803761 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0815 01:27:34.971220 803761 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0815 01:27:34.971264 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:34.988187 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:35.094103 803761 ssh_runner.go:195] Run: cat /etc/os-release
I0815 01:27:35.098107 803761 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0815 01:27:35.098147 803761 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0815 01:27:35.098160 803761 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0815 01:27:35.098173 803761 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0815 01:27:35.098208 803761 filesync.go:126] Scanning /home/jenkins/minikube-integration/19443-587265/.minikube/addons for local assets ...
I0815 01:27:35.098292 803761 filesync.go:126] Scanning /home/jenkins/minikube-integration/19443-587265/.minikube/files for local assets ...
I0815 01:27:35.098392 803761 filesync.go:149] local asset: /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem -> 5926602.pem in /etc/ssl/certs
I0815 01:27:35.098516 803761 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0815 01:27:35.108070 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem --> /etc/ssl/certs/5926602.pem (1708 bytes)
I0815 01:27:35.134614 803761 start.go:296] duration metric: took 163.450011ms for postStartSetup
I0815 01:27:35.134743 803761 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0815 01:27:35.134837 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:35.153119 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:35.249320 803761 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0815 01:27:35.253770 803761 fix.go:56] duration metric: took 4.665319499s for fixHost
I0815 01:27:35.253797 803761 start.go:83] releasing machines lock for "no-preload-891255", held for 4.66537485s
I0815 01:27:35.253884 803761 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" no-preload-891255
I0815 01:27:35.271001 803761 ssh_runner.go:195] Run: cat /version.json
I0815 01:27:35.271066 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:35.271092 803761 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0815 01:27:35.271152 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:35.287966 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:35.303109 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:35.533499 803761 ssh_runner.go:195] Run: systemctl --version
I0815 01:27:35.537985 803761 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0815 01:27:35.542537 803761 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0815 01:27:35.560110 803761 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0815 01:27:35.560217 803761 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0815 01:27:35.569395 803761 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0815 01:27:35.569475 803761 start.go:495] detecting cgroup driver to use...
I0815 01:27:35.569521 803761 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0815 01:27:35.569611 803761 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0815 01:27:35.583051 803761 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0815 01:27:35.595039 803761 docker.go:217] disabling cri-docker service (if available) ...
I0815 01:27:35.595149 803761 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0815 01:27:35.609596 803761 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0815 01:27:35.623379 803761 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0815 01:27:35.707493 803761 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0815 01:27:35.795262 803761 docker.go:233] disabling docker service ...
I0815 01:27:35.795328 803761 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0815 01:27:35.808774 803761 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0815 01:27:35.824144 803761 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0815 01:27:35.908852 803761 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0815 01:27:36.018182 803761 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0815 01:27:36.031666 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0815 01:27:36.051746 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0815 01:27:36.063557 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0815 01:27:36.075101 803761 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0815 01:27:36.075219 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0815 01:27:36.087706 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0815 01:27:36.098900 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0815 01:27:36.109941 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0815 01:27:36.121226 803761 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0815 01:27:36.137079 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0815 01:27:36.149018 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0815 01:27:36.159749 803761 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0815 01:27:36.170739 803761 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0815 01:27:36.180275 803761 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0815 01:27:36.189120 803761 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:27:36.273204 803761 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0815 01:27:36.432963 803761 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0815 01:27:36.433038 803761 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0815 01:27:36.437282 803761 start.go:563] Will wait 60s for crictl version
I0815 01:27:36.437357 803761 ssh_runner.go:195] Run: which crictl
I0815 01:27:36.440693 803761 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0815 01:27:36.484837 803761 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.20
RuntimeApiVersion: v1
I0815 01:27:36.484911 803761 ssh_runner.go:195] Run: containerd --version
I0815 01:27:36.511673 803761 ssh_runner.go:195] Run: containerd --version
I0815 01:27:36.541876 803761 out.go:177] * Preparing Kubernetes v1.31.0 on containerd 1.7.20 ...
I0815 01:27:36.543510 803761 cli_runner.go:164] Run: docker network inspect no-preload-891255 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0815 01:27:36.579131 803761 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0815 01:27:36.588569 803761 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 01:27:36.600920 803761 kubeadm.go:883] updating cluster {Name:no-preload-891255 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-891255 Namespace:default APIServerHAVIP: APIServerName:minikubeCA API
ServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenk
ins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0815 01:27:36.601051 803761 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime containerd
I0815 01:27:36.601099 803761 ssh_runner.go:195] Run: sudo crictl images --output json
I0815 01:27:36.645978 803761 containerd.go:627] all images are preloaded for containerd runtime.
I0815 01:27:36.645999 803761 cache_images.go:84] Images are preloaded, skipping loading
I0815 01:27:36.646008 803761 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.31.0 containerd true true} ...
I0815 01:27:36.646118 803761 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=no-preload-891255 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.31.0 ClusterName:no-preload-891255 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0815 01:27:36.646181 803761 ssh_runner.go:195] Run: sudo crictl info
I0815 01:27:36.688709 803761 cni.go:84] Creating CNI manager for ""
I0815 01:27:36.688737 803761 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0815 01:27:36.688750 803761 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0815 01:27:36.688775 803761 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.31.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:no-preload-891255 NodeName:no-preload-891255 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodP
ath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0815 01:27:36.688924 803761 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "no-preload-891255"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0815 01:27:36.689001 803761 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.0
I0815 01:27:36.699237 803761 binaries.go:44] Found k8s binaries, skipping transfer
I0815 01:27:36.699330 803761 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0815 01:27:36.709004 803761 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (321 bytes)
I0815 01:27:36.728658 803761 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0815 01:27:36.747120 803761 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2171 bytes)
I0815 01:27:36.772634 803761 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0815 01:27:36.776453 803761 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0815 01:27:36.792503 803761 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:27:36.887664 803761 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0815 01:27:36.903374 803761 certs.go:68] Setting up /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255 for IP: 192.168.76.2
I0815 01:27:36.903449 803761 certs.go:194] generating shared ca certs ...
I0815 01:27:36.903481 803761 certs.go:226] acquiring lock for ca certs: {Name:mkd44da6bd4b219dfe871c9c58d5756252de3a14 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:27:36.903641 803761 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19443-587265/.minikube/ca.key
I0815 01:27:36.903745 803761 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.key
I0815 01:27:36.903773 803761 certs.go:256] generating profile certs ...
I0815 01:27:36.903936 803761 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/client.key
I0815 01:27:36.904039 803761 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/apiserver.key.69237bdd
I0815 01:27:36.904102 803761 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/proxy-client.key
I0815 01:27:36.904243 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660.pem (1338 bytes)
W0815 01:27:36.904312 803761 certs.go:480] ignoring /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660_empty.pem, impossibly tiny 0 bytes
I0815 01:27:36.904345 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca-key.pem (1675 bytes)
I0815 01:27:36.904396 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/ca.pem (1082 bytes)
I0815 01:27:36.904448 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/cert.pem (1123 bytes)
I0815 01:27:36.904499 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/certs/key.pem (1675 bytes)
I0815 01:27:36.904575 803761 certs.go:484] found cert: /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem (1708 bytes)
I0815 01:27:36.905247 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0815 01:27:36.931179 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0815 01:27:36.962131 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0815 01:27:36.987576 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0815 01:27:37.022483 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1424 bytes)
I0815 01:27:37.060458 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0815 01:27:37.110653 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0815 01:27:37.152558 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/profiles/no-preload-891255/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0815 01:27:37.179801 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/files/etc/ssl/certs/5926602.pem --> /usr/share/ca-certificates/5926602.pem (1708 bytes)
I0815 01:27:37.209981 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0815 01:27:37.239473 803761 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19443-587265/.minikube/certs/592660.pem --> /usr/share/ca-certificates/592660.pem (1338 bytes)
I0815 01:27:37.266845 803761 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0815 01:27:37.287391 803761 ssh_runner.go:195] Run: openssl version
I0815 01:27:37.295180 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/5926602.pem && ln -fs /usr/share/ca-certificates/5926602.pem /etc/ssl/certs/5926602.pem"
I0815 01:27:37.306727 803761 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/5926602.pem
I0815 01:27:37.310424 803761 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 15 00:47 /usr/share/ca-certificates/5926602.pem
I0815 01:27:37.310536 803761 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/5926602.pem
I0815 01:27:37.320089 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/5926602.pem /etc/ssl/certs/3ec20f2e.0"
I0815 01:27:37.333178 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0815 01:27:37.343331 803761 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0815 01:27:37.347389 803761 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 15 00:36 /usr/share/ca-certificates/minikubeCA.pem
I0815 01:27:37.347460 803761 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0815 01:27:37.355090 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0815 01:27:37.365007 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/592660.pem && ln -fs /usr/share/ca-certificates/592660.pem /etc/ssl/certs/592660.pem"
I0815 01:27:37.374829 803761 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/592660.pem
I0815 01:27:37.378492 803761 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 15 00:47 /usr/share/ca-certificates/592660.pem
I0815 01:27:37.378559 803761 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/592660.pem
I0815 01:27:37.386206 803761 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/592660.pem /etc/ssl/certs/51391683.0"
I0815 01:27:37.395770 803761 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0815 01:27:37.399598 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0815 01:27:37.406724 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0815 01:27:37.414299 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0815 01:27:37.421554 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0815 01:27:37.429092 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0815 01:27:37.437083 803761 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0815 01:27:37.444633 803761 kubeadm.go:392] StartCluster: {Name:no-preload-891255 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723650208-19443@sha256:2be48dc5c74cde3c1d15ac913a640f4a2331b48358b81777568fb487d2757002 Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:no-preload-891255 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISer
verNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins
:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0815 01:27:37.444736 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0815 01:27:37.444817 803761 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0815 01:27:37.492095 803761 cri.go:89] found id: "ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1"
I0815 01:27:37.492173 803761 cri.go:89] found id: "5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd"
I0815 01:27:37.492193 803761 cri.go:89] found id: "2d448a62ba9b867bd5bd34037fec9b78055264519f2d2574275038fbe92e1f7e"
I0815 01:27:37.492203 803761 cri.go:89] found id: "f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39"
I0815 01:27:37.492217 803761 cri.go:89] found id: "adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7"
I0815 01:27:37.492224 803761 cri.go:89] found id: "d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678"
I0815 01:27:37.492227 803761 cri.go:89] found id: "2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22"
I0815 01:27:37.492231 803761 cri.go:89] found id: "b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb"
I0815 01:27:37.492239 803761 cri.go:89] found id: ""
I0815 01:27:37.492294 803761 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0815 01:27:37.507684 803761 cri.go:116] JSON = null
W0815 01:27:37.507794 803761 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0815 01:27:37.507934 803761 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0815 01:27:37.521514 803761 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0815 01:27:37.521581 803761 kubeadm.go:593] restartPrimaryControlPlane start ...
I0815 01:27:37.521656 803761 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0815 01:27:37.533796 803761 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0815 01:27:37.534503 803761 kubeconfig.go:47] verify endpoint returned: get endpoint: "no-preload-891255" does not appear in /home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:27:37.534840 803761 kubeconfig.go:62] /home/jenkins/minikube-integration/19443-587265/kubeconfig needs updating (will repair): [kubeconfig missing "no-preload-891255" cluster setting kubeconfig missing "no-preload-891255" context setting]
I0815 01:27:37.535367 803761 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19443-587265/kubeconfig: {Name:mka65351b6674d2edd84b4cf38d527ec03739af8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:27:37.537063 803761 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0815 01:27:37.548715 803761 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.76.2
I0815 01:27:37.548796 803761 kubeadm.go:597] duration metric: took 27.194048ms to restartPrimaryControlPlane
I0815 01:27:37.548820 803761 kubeadm.go:394] duration metric: took 104.197866ms to StartCluster
I0815 01:27:37.548867 803761 settings.go:142] acquiring lock: {Name:mkf353d296e2684cbdd29a016c10a0eb45e9f213 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:27:37.548977 803761 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19443-587265/kubeconfig
I0815 01:27:37.550010 803761 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19443-587265/kubeconfig: {Name:mka65351b6674d2edd84b4cf38d527ec03739af8 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0815 01:27:37.550303 803761 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0815 01:27:37.550814 803761 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0815 01:27:37.550891 803761 addons.go:69] Setting storage-provisioner=true in profile "no-preload-891255"
I0815 01:27:37.550913 803761 addons.go:234] Setting addon storage-provisioner=true in "no-preload-891255"
W0815 01:27:37.550919 803761 addons.go:243] addon storage-provisioner should already be in state true
I0815 01:27:37.550942 803761 host.go:66] Checking if "no-preload-891255" exists ...
I0815 01:27:37.551502 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:37.551798 803761 config.go:182] Loaded profile config "no-preload-891255": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.0
I0815 01:27:37.551955 803761 addons.go:69] Setting metrics-server=true in profile "no-preload-891255"
I0815 01:27:37.552009 803761 addons.go:234] Setting addon metrics-server=true in "no-preload-891255"
W0815 01:27:37.552063 803761 addons.go:243] addon metrics-server should already be in state true
I0815 01:27:37.552105 803761 host.go:66] Checking if "no-preload-891255" exists ...
I0815 01:27:37.552639 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:37.555148 803761 out.go:177] * Verifying Kubernetes components...
I0815 01:27:37.555398 803761 addons.go:69] Setting default-storageclass=true in profile "no-preload-891255"
I0815 01:27:37.555431 803761 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "no-preload-891255"
I0815 01:27:37.556297 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:37.556648 803761 addons.go:69] Setting dashboard=true in profile "no-preload-891255"
I0815 01:27:37.557025 803761 addons.go:234] Setting addon dashboard=true in "no-preload-891255"
W0815 01:27:37.558157 803761 addons.go:243] addon dashboard should already be in state true
I0815 01:27:37.558249 803761 host.go:66] Checking if "no-preload-891255" exists ...
I0815 01:27:37.560283 803761 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0815 01:27:37.561161 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:37.617832 803761 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0815 01:27:37.622897 803761 addons.go:234] Setting addon default-storageclass=true in "no-preload-891255"
W0815 01:27:37.622920 803761 addons.go:243] addon default-storageclass should already be in state true
I0815 01:27:37.622944 803761 host.go:66] Checking if "no-preload-891255" exists ...
I0815 01:27:37.623413 803761 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:27:37.623436 803761 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0815 01:27:37.623509 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:37.623985 803761 cli_runner.go:164] Run: docker container inspect no-preload-891255 --format={{.State.Status}}
I0815 01:27:37.638249 803761 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0815 01:27:37.638364 803761 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0815 01:27:37.640551 803761 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0815 01:27:34.824704 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:37.324127 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:39.325365 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:37.640662 803761 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0815 01:27:37.640676 803761 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0815 01:27:37.640754 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:37.642155 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0815 01:27:37.642183 803761 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0815 01:27:37.642249 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:37.698773 803761 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0815 01:27:37.698796 803761 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0815 01:27:37.698857 803761 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" no-preload-891255
I0815 01:27:37.717506 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:37.723999 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:37.733964 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:37.747263 803761 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:33810 SSHKeyPath:/home/jenkins/minikube-integration/19443-587265/.minikube/machines/no-preload-891255/id_rsa Username:docker}
I0815 01:27:37.799926 803761 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0815 01:27:37.870916 803761 node_ready.go:35] waiting up to 6m0s for node "no-preload-891255" to be "Ready" ...
I0815 01:27:37.915707 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0815 01:27:37.915730 803761 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0815 01:27:37.996627 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0815 01:27:37.996659 803761 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0815 01:27:38.028878 803761 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0815 01:27:38.055598 803761 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0815 01:27:38.055640 803761 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0815 01:27:38.071588 803761 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0815 01:27:38.225178 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0815 01:27:38.225258 803761 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0815 01:27:38.269345 803761 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0815 01:27:38.269376 803761 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0815 01:27:38.296602 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0815 01:27:38.296629 803761 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0815 01:27:38.353701 803761 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:27:38.353738 803761 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0815 01:27:38.382420 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0815 01:27:38.382458 803761 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
I0815 01:27:38.514518 803761 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0815 01:27:38.560916 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0815 01:27:38.560995 803761 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0815 01:27:38.688554 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0815 01:27:38.688628 803761 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0815 01:27:38.734488 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0815 01:27:38.734567 803761 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0815 01:27:38.840227 803761 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:27:38.840302 803761 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0815 01:27:38.896730 803761 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0815 01:27:41.839092 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:44.325718 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:43.155439 803761 node_ready.go:49] node "no-preload-891255" has status "Ready":"True"
I0815 01:27:43.155464 803761 node_ready.go:38] duration metric: took 5.284505523s for node "no-preload-891255" to be "Ready" ...
I0815 01:27:43.155474 803761 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0815 01:27:43.236802 803761 pod_ready.go:78] waiting up to 6m0s for pod "coredns-6f6b679f8f-q45lr" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.297601 803761 pod_ready.go:92] pod "coredns-6f6b679f8f-q45lr" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:43.297671 803761 pod_ready.go:81] duration metric: took 60.793612ms for pod "coredns-6f6b679f8f-q45lr" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.297701 803761 pod_ready.go:78] waiting up to 6m0s for pod "etcd-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.341163 803761 pod_ready.go:92] pod "etcd-no-preload-891255" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:43.341239 803761 pod_ready.go:81] duration metric: took 43.509638ms for pod "etcd-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.341270 803761 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.367071 803761 pod_ready.go:92] pod "kube-apiserver-no-preload-891255" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:43.367144 803761 pod_ready.go:81] duration metric: took 25.852612ms for pod "kube-apiserver-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.367176 803761 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.400093 803761 pod_ready.go:92] pod "kube-controller-manager-no-preload-891255" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:43.400165 803761 pod_ready.go:81] duration metric: took 32.96864ms for pod "kube-controller-manager-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:43.400216 803761 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-7k9tc" in "kube-system" namespace to be "Ready" ...
I0815 01:27:45.406853 803761 pod_ready.go:102] pod "kube-proxy-7k9tc" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:45.906165 803761 pod_ready.go:92] pod "kube-proxy-7k9tc" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:45.906245 803761 pod_ready.go:81] duration metric: took 2.506004597s for pod "kube-proxy-7k9tc" in "kube-system" namespace to be "Ready" ...
I0815 01:27:45.906277 803761 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:46.133972 803761 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.10505343s)
I0815 01:27:46.134095 803761 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: (8.062468966s)
W0815 01:27:46.134352 803761 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0815 01:27:46.134174 803761 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.619577449s)
I0815 01:27:46.134411 803761 addons.go:475] Verifying addon metrics-server=true in "no-preload-891255"
I0815 01:27:46.134272 803761 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (7.237510537s)
I0815 01:27:46.134537 803761 retry.go:31] will retry after 160.382742ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
error: error validating "/etc/kubernetes/addons/storageclass.yaml": error validating data: failed to download openapi: Get "https://localhost:8443/openapi/v2?timeout=32s": dial tcp [::1]:8443: connect: connection refused; if you choose to ignore these errors, turn validation off with --validate=false
I0815 01:27:46.136513 803761 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p no-preload-891255 addons enable metrics-server
I0815 01:27:46.295970 803761 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.31.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0815 01:27:46.489160 803761 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0815 01:27:46.824159 798414 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:48.323803 798414 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:48.323828 798414 pod_ready.go:81] duration metric: took 1m9.006777258s for pod "kube-controller-manager-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.323840 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-hdj25" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.328838 798414 pod_ready.go:92] pod "kube-proxy-hdj25" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:48.328866 798414 pod_ready.go:81] duration metric: took 4.98266ms for pod "kube-proxy-hdj25" in "kube-system" namespace to be "Ready" ...
I0815 01:27:48.328878 798414 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:46.490615 803761 addons.go:510] duration metric: took 8.939796419s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0815 01:27:47.913124 803761 pod_ready.go:102] pod "kube-scheduler-no-preload-891255" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:50.337069 798414 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:52.836311 798414 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:54.334906 798414 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:54.334933 798414 pod_ready.go:81] duration metric: took 6.006014766s for pod "kube-scheduler-old-k8s-version-145466" in "kube-system" namespace to be "Ready" ...
I0815 01:27:54.334945 798414 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace to be "Ready" ...
I0815 01:27:50.413174 803761 pod_ready.go:102] pod "kube-scheduler-no-preload-891255" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:52.413472 803761 pod_ready.go:102] pod "kube-scheduler-no-preload-891255" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:54.914461 803761 pod_ready.go:102] pod "kube-scheduler-no-preload-891255" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:56.341453 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:58.841137 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:27:57.420989 803761 pod_ready.go:92] pod "kube-scheduler-no-preload-891255" in "kube-system" namespace has status "Ready":"True"
I0815 01:27:57.421017 803761 pod_ready.go:81] duration metric: took 11.514711642s for pod "kube-scheduler-no-preload-891255" in "kube-system" namespace to be "Ready" ...
I0815 01:27:57.421029 803761 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace to be "Ready" ...
I0815 01:27:59.426644 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:01.341468 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:03.342277 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:01.428229 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:03.927438 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:05.841177 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:07.841420 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:06.428013 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:08.927156 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:09.841515 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:11.841903 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:14.341251 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:10.927748 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:12.927949 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:16.341817 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:18.905274 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:15.428036 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:17.927452 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:21.340938 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:23.343888 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:20.427658 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:22.926849 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:24.927327 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:25.840886 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:27.841307 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:27.427742 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:29.928619 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:29.842382 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:31.845262 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:34.340679 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:32.426860 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:34.928177 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:36.340926 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:38.348235 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:37.426923 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:39.427676 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:40.841788 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:43.341955 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:41.927220 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:44.427318 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:45.343754 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:47.841262 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:46.427528 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:48.428585 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:50.341583 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:52.342360 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:50.927664 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:52.927962 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:54.928112 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:54.842055 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:57.341767 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:57.426776 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:59.427474 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:28:59.842110 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:02.341631 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:01.428036 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:03.927170 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:04.841905 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:07.341412 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:09.361881 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:05.927728 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:08.427194 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:11.840761 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:13.845821 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:10.926907 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:12.927545 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:14.927893 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:16.341158 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:18.341582 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:17.427649 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:19.428474 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:20.342140 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:22.842042 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:21.927110 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:24.427410 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:24.842165 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:27.341302 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:26.927017 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:28.927263 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:29.844818 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:32.340995 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:34.341661 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:30.927788 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:33.427114 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:36.841766 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:39.342526 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:35.427179 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:37.927113 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:41.841158 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:43.842484 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:40.427490 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:42.927907 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:44.928392 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:46.341306 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:48.840994 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:47.426868 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:49.927329 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:50.842218 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:53.341747 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:52.426796 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:54.426847 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:55.841257 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:58.341465 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:56.927611 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:29:59.427411 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:00.380117 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:02.840765 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:01.427446 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:03.427927 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:05.341086 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:07.341145 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:09.341333 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:05.927282 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:08.427203 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:11.342252 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:13.841939 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:10.427470 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:12.427509 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:14.427889 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:16.345255 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:18.843266 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:16.428131 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:18.927570 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:21.341071 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:23.341252 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:21.426786 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:23.427241 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:25.341574 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:27.842623 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:25.427318 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:27.928194 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:30.341285 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:32.342057 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:30.427166 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:32.926803 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:34.927877 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:34.841107 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:36.841164 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:38.841855 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:37.426777 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:39.427906 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:41.340950 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:43.341511 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:41.927037 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:43.927338 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:45.342593 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:47.840983 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:46.426893 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:48.927895 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:49.842312 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:52.341179 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:51.427096 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:53.427519 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:54.841042 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:56.841647 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:59.340948 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:55.927137 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:30:58.426574 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:01.840886 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:03.841711 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:00.427607 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:02.926729 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:06.341111 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:08.845348 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:05.428060 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:07.927610 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:11.340908 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:13.340956 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:10.428551 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:12.927001 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:14.927235 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:15.341002 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:17.342150 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:16.927528 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:18.927597 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:19.841230 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:21.841934 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:24.341985 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:21.428065 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:23.926563 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:26.841205 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:28.841630 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:25.927125 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:27.927322 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:29.927384 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:31.341383 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:33.841991 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:32.426491 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:34.427005 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:36.341927 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:38.840908 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:36.427436 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:38.926916 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:41.340767 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:43.341898 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:40.927564 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:42.927616 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:44.927766 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:45.342464 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:47.842026 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:47.428985 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:49.926419 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:50.341226 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:52.841594 798414 pod_ready.go:102] pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:54.341770 798414 pod_ready.go:81] duration metric: took 4m0.006810385s for pod "metrics-server-9975d5f86-qvcw4" in "kube-system" namespace to be "Ready" ...
E0815 01:31:54.341793 798414 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0815 01:31:54.341801 798414 pod_ready.go:38] duration metric: took 5m26.123866447s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0815 01:31:54.341815 798414 api_server.go:52] waiting for apiserver process to appear ...
I0815 01:31:54.341844 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:31:54.341906 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:31:54.389104 798414 cri.go:89] found id: "5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:31:54.389129 798414 cri.go:89] found id: "d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:31:54.389135 798414 cri.go:89] found id: ""
I0815 01:31:54.389142 798414 logs.go:276] 2 containers: [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f]
I0815 01:31:54.389202 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.392818 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.396470 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:31:54.396573 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:31:54.440719 798414 cri.go:89] found id: "1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:31:54.440738 798414 cri.go:89] found id: "149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:31:54.440744 798414 cri.go:89] found id: ""
I0815 01:31:54.440751 798414 logs.go:276] 2 containers: [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d]
I0815 01:31:54.440852 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.444415 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.447805 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:31:54.447916 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:31:54.487638 798414 cri.go:89] found id: "08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:31:54.487661 798414 cri.go:89] found id: "56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:31:54.487666 798414 cri.go:89] found id: ""
I0815 01:31:54.487673 798414 logs.go:276] 2 containers: [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4]
I0815 01:31:54.487731 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.491365 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.494774 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:31:54.494846 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:31:54.556835 798414 cri.go:89] found id: "ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:31:54.556860 798414 cri.go:89] found id: "f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:31:54.556865 798414 cri.go:89] found id: ""
I0815 01:31:54.556873 798414 logs.go:276] 2 containers: [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d]
I0815 01:31:54.556935 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.560493 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.563824 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:31:54.563938 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:31:51.927026 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:54.427978 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:54.604088 798414 cri.go:89] found id: "65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:31:54.604161 798414 cri.go:89] found id: "8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:31:54.604173 798414 cri.go:89] found id: ""
I0815 01:31:54.604181 798414 logs.go:276] 2 containers: [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5]
I0815 01:31:54.604282 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.607763 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.611024 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:31:54.611091 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:31:54.649541 798414 cri.go:89] found id: "30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:31:54.649565 798414 cri.go:89] found id: "d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:31:54.649570 798414 cri.go:89] found id: ""
I0815 01:31:54.649577 798414 logs.go:276] 2 containers: [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f]
I0815 01:31:54.649635 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.653540 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.656990 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:31:54.657061 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:31:54.705048 798414 cri.go:89] found id: "6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:31:54.705073 798414 cri.go:89] found id: "8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:31:54.705078 798414 cri.go:89] found id: ""
I0815 01:31:54.705086 798414 logs.go:276] 2 containers: [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1]
I0815 01:31:54.705142 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.708794 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.712050 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:31:54.712119 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:31:54.761867 798414 cri.go:89] found id: "666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:31:54.761892 798414 cri.go:89] found id: ""
I0815 01:31:54.761901 798414 logs.go:276] 1 containers: [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6]
I0815 01:31:54.761956 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.765515 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:31:54.765592 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:31:54.814259 798414 cri.go:89] found id: "fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:31:54.814287 798414 cri.go:89] found id: "6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:31:54.814292 798414 cri.go:89] found id: ""
I0815 01:31:54.814299 798414 logs.go:276] 2 containers: [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414]
I0815 01:31:54.814357 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.818624 798414 ssh_runner.go:195] Run: which crictl
I0815 01:31:54.822284 798414 logs.go:123] Gathering logs for kube-scheduler [f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d] ...
I0815 01:31:54.822315 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:31:54.866675 798414 logs.go:123] Gathering logs for kindnet [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b] ...
I0815 01:31:54.866707 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:31:54.927529 798414 logs.go:123] Gathering logs for etcd [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43] ...
I0815 01:31:54.927578 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:31:54.977111 798414 logs.go:123] Gathering logs for coredns [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9] ...
I0815 01:31:54.977140 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:31:55.029599 798414 logs.go:123] Gathering logs for kube-proxy [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418] ...
I0815 01:31:55.029630 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:31:55.072848 798414 logs.go:123] Gathering logs for kube-controller-manager [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd] ...
I0815 01:31:55.072876 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:31:55.141657 798414 logs.go:123] Gathering logs for kube-controller-manager [d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f] ...
I0815 01:31:55.141692 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:31:55.207613 798414 logs.go:123] Gathering logs for kubelet ...
I0815 01:31:55.207649 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:31:55.272355 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206496 664 reflector.go:138] object-"kube-system"/"coredns-token-hqq2w": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-hqq2w" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.272573 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206838 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.272786 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207185 664 reflector.go:138] object-"kube-system"/"kindnet-token-pflml": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pflml" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273001 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207498 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-rl52n": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-rl52n" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273206 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207678 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273517 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.264938 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k6pcd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k6pcd" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.273725 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.114283 664 reflector.go:138] object-"default"/"default-token-j4wgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-j4wgn" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.276976 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.390721 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.278325 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.415771 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.281690 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:42 old-k8s-version-145466 kubelet[664]: E0815 01:26:42.146871 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.282438 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:49 old-k8s-version-145466 kubelet[664]: E0815 01:26:49.166497 664 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-fksp6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-fksp6" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:31:55.283970 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:57 old-k8s-version-145466 kubelet[664]: E0815 01:26:57.141841 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.284430 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:00 old-k8s-version-145466 kubelet[664]: E0815 01:27:00.751687 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.284885 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:01 old-k8s-version-145466 kubelet[664]: E0815 01:27:01.756479 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.285325 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:02 old-k8s-version-145466 kubelet[664]: E0815 01:27:02.761739 664 pod_workers.go:191] Error syncing pod d7e84c38-c90e-427c-bfc5-45adf788d6fe ("storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"
W0815 01:31:55.285983 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:08 old-k8s-version-145466 kubelet[664]: E0815 01:27:08.528394 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.288438 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:11 old-k8s-version-145466 kubelet[664]: E0815 01:27:11.140512 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.289156 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:20 old-k8s-version-145466 kubelet[664]: E0815 01:27:20.809846 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.289339 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:24 old-k8s-version-145466 kubelet[664]: E0815 01:27:24.133897 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.289665 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:28 old-k8s-version-145466 kubelet[664]: E0815 01:27:28.528406 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.289850 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:38 old-k8s-version-145466 kubelet[664]: E0815 01:27:38.186907 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.290435 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:42 old-k8s-version-145466 kubelet[664]: E0815 01:27:42.877822 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.290766 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:48 old-k8s-version-145466 kubelet[664]: E0815 01:27:48.529042 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.293220 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:53 old-k8s-version-145466 kubelet[664]: E0815 01:27:53.141943 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.293549 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:02 old-k8s-version-145466 kubelet[664]: E0815 01:28:02.132198 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.293733 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:04 old-k8s-version-145466 kubelet[664]: E0815 01:28:04.133346 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.293928 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:16 old-k8s-version-145466 kubelet[664]: E0815 01:28:16.144812 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.294252 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:17 old-k8s-version-145466 kubelet[664]: E0815 01:28:17.132160 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.294839 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:29 old-k8s-version-145466 kubelet[664]: E0815 01:28:29.028736 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.295025 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:31 old-k8s-version-145466 kubelet[664]: E0815 01:28:31.132458 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.295352 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:38 old-k8s-version-145466 kubelet[664]: E0815 01:28:38.528400 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.295538 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:44 old-k8s-version-145466 kubelet[664]: E0815 01:28:44.132479 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.295872 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:51 old-k8s-version-145466 kubelet[664]: E0815 01:28:51.133407 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.296062 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:56 old-k8s-version-145466 kubelet[664]: E0815 01:28:56.135291 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.296387 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:04 old-k8s-version-145466 kubelet[664]: E0815 01:29:04.132344 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.296569 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:10 old-k8s-version-145466 kubelet[664]: E0815 01:29:10.132536 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.296893 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:17 old-k8s-version-145466 kubelet[664]: E0815 01:29:17.132162 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.299321 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:23 old-k8s-version-145466 kubelet[664]: E0815 01:29:23.139752 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:31:55.299673 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:28 old-k8s-version-145466 kubelet[664]: E0815 01:29:28.139064 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.299863 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:34 old-k8s-version-145466 kubelet[664]: E0815 01:29:34.132664 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.300190 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:40 old-k8s-version-145466 kubelet[664]: E0815 01:29:40.133096 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.300379 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:49 old-k8s-version-145466 kubelet[664]: E0815 01:29:49.133068 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.300968 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:51 old-k8s-version-145466 kubelet[664]: E0815 01:29:51.281771 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301293 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:58 old-k8s-version-145466 kubelet[664]: E0815 01:29:58.528848 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301485 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:00 old-k8s-version-145466 kubelet[664]: E0815 01:30:00.160446 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.301814 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:10 old-k8s-version-145466 kubelet[664]: E0815 01:30:10.135730 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.301998 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:11 old-k8s-version-145466 kubelet[664]: E0815 01:30:11.132453 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.302337 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:22 old-k8s-version-145466 kubelet[664]: E0815 01:30:22.132670 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.302530 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:26 old-k8s-version-145466 kubelet[664]: E0815 01:30:26.135386 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.302859 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:33 old-k8s-version-145466 kubelet[664]: E0815 01:30:33.132520 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.303041 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:41 old-k8s-version-145466 kubelet[664]: E0815 01:30:41.132604 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.303369 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: E0815 01:30:45.132399 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.303551 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:54 old-k8s-version-145466 kubelet[664]: E0815 01:30:54.132932 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.303895 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: E0815 01:30:56.132938 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.304080 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:06 old-k8s-version-145466 kubelet[664]: E0815 01:31:06.132954 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.304404 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: E0815 01:31:11.132193 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.304589 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:20 old-k8s-version-145466 kubelet[664]: E0815 01:31:20.132734 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.304913 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: E0815 01:31:23.132188 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.305096 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.305420 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.305603 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:55.305928 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:55.306109 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0815 01:31:55.306119 798414 logs.go:123] Gathering logs for kube-apiserver [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645] ...
I0815 01:31:55.306135 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:31:55.368137 798414 logs.go:123] Gathering logs for kindnet [8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1] ...
I0815 01:31:55.368175 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:31:55.418716 798414 logs.go:123] Gathering logs for kubernetes-dashboard [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6] ...
I0815 01:31:55.418749 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:31:55.463235 798414 logs.go:123] Gathering logs for storage-provisioner [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3] ...
I0815 01:31:55.463266 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:31:55.524586 798414 logs.go:123] Gathering logs for container status ...
I0815 01:31:55.524614 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:31:55.586139 798414 logs.go:123] Gathering logs for etcd [149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d] ...
I0815 01:31:55.586167 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:31:55.628448 798414 logs.go:123] Gathering logs for kube-scheduler [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8] ...
I0815 01:31:55.628476 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:31:55.667804 798414 logs.go:123] Gathering logs for kube-apiserver [d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f] ...
I0815 01:31:55.667828 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:31:55.730471 798414 logs.go:123] Gathering logs for coredns [56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4] ...
I0815 01:31:55.730513 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:31:55.779800 798414 logs.go:123] Gathering logs for kube-proxy [8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5] ...
I0815 01:31:55.779833 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:31:55.823656 798414 logs.go:123] Gathering logs for storage-provisioner [6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414] ...
I0815 01:31:55.823733 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:31:55.861500 798414 logs.go:123] Gathering logs for containerd ...
I0815 01:31:55.861528 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:31:55.921791 798414 logs.go:123] Gathering logs for dmesg ...
I0815 01:31:55.921824 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:31:55.943979 798414 logs.go:123] Gathering logs for describe nodes ...
I0815 01:31:55.944021 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:31:56.104386 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:56.104411 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:31:56.104602 798414 out.go:239] X Problems detected in kubelet:
W0815 01:31:56.104620 798414 out.go:239] Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:56.104640 798414 out.go:239] Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:56.104655 798414 out.go:239] Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:31:56.104675 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:31:56.104690 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
I0815 01:31:56.104697 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:56.104709 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:31:56.927528 803761 pod_ready.go:102] pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace has status "Ready":"False"
I0815 01:31:57.426424 803761 pod_ready.go:81] duration metric: took 4m0.005381694s for pod "metrics-server-6867b74b74-nsksn" in "kube-system" namespace to be "Ready" ...
E0815 01:31:57.426450 803761 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0815 01:31:57.426466 803761 pod_ready.go:38] duration metric: took 4m14.270979766s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0815 01:31:57.426480 803761 api_server.go:52] waiting for apiserver process to appear ...
I0815 01:31:57.426508 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:31:57.426580 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:31:57.462402 803761 cri.go:89] found id: "46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951"
I0815 01:31:57.462429 803761 cri.go:89] found id: "b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb"
I0815 01:31:57.462435 803761 cri.go:89] found id: ""
I0815 01:31:57.462443 803761 logs.go:276] 2 containers: [46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951 b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb]
I0815 01:31:57.462534 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.466167 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.469589 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:31:57.469662 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:31:57.522815 803761 cri.go:89] found id: "af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3"
I0815 01:31:57.522840 803761 cri.go:89] found id: "adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7"
I0815 01:31:57.522845 803761 cri.go:89] found id: ""
I0815 01:31:57.522853 803761 logs.go:276] 2 containers: [af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3 adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7]
I0815 01:31:57.522944 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.526918 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.530681 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:31:57.530752 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:31:57.570743 803761 cri.go:89] found id: "6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af"
I0815 01:31:57.570769 803761 cri.go:89] found id: "ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1"
I0815 01:31:57.570774 803761 cri.go:89] found id: ""
I0815 01:31:57.570782 803761 logs.go:276] 2 containers: [6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1]
I0815 01:31:57.570871 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.574486 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.577683 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:31:57.577780 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:31:57.617135 803761 cri.go:89] found id: "914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54"
I0815 01:31:57.617157 803761 cri.go:89] found id: "d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678"
I0815 01:31:57.617162 803761 cri.go:89] found id: ""
I0815 01:31:57.617169 803761 logs.go:276] 2 containers: [914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54 d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678]
I0815 01:31:57.617253 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.620788 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.624602 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:31:57.624717 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:31:57.665191 803761 cri.go:89] found id: "f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d"
I0815 01:31:57.665260 803761 cri.go:89] found id: "f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39"
I0815 01:31:57.665281 803761 cri.go:89] found id: ""
I0815 01:31:57.665304 803761 logs.go:276] 2 containers: [f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39]
I0815 01:31:57.665374 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.669052 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.672595 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:31:57.672699 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:31:57.711112 803761 cri.go:89] found id: "9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3"
I0815 01:31:57.711149 803761 cri.go:89] found id: "2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22"
I0815 01:31:57.711155 803761 cri.go:89] found id: ""
I0815 01:31:57.711163 803761 logs.go:276] 2 containers: [9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3 2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22]
I0815 01:31:57.711241 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.714858 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.718200 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:31:57.718324 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:31:57.754894 803761 cri.go:89] found id: "01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0"
I0815 01:31:57.754937 803761 cri.go:89] found id: "5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd"
I0815 01:31:57.754943 803761 cri.go:89] found id: ""
I0815 01:31:57.754951 803761 logs.go:276] 2 containers: [01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0 5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd]
I0815 01:31:57.755016 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.759055 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.762518 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:31:57.762604 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:31:57.801833 803761 cri.go:89] found id: "10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84"
I0815 01:31:57.801858 803761 cri.go:89] found id: ""
I0815 01:31:57.801867 803761 logs.go:276] 1 containers: [10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84]
I0815 01:31:57.801922 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.805384 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:31:57.805455 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:31:57.845472 803761 cri.go:89] found id: "a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b"
I0815 01:31:57.845494 803761 cri.go:89] found id: "c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499"
I0815 01:31:57.845500 803761 cri.go:89] found id: ""
I0815 01:31:57.845507 803761 logs.go:276] 2 containers: [a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499]
I0815 01:31:57.845567 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.849242 803761 ssh_runner.go:195] Run: which crictl
I0815 01:31:57.852792 803761 logs.go:123] Gathering logs for kube-controller-manager [2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22] ...
I0815 01:31:57.852818 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22"
I0815 01:31:57.914973 803761 logs.go:123] Gathering logs for kindnet [5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd] ...
I0815 01:31:57.915028 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd"
I0815 01:31:57.983460 803761 logs.go:123] Gathering logs for storage-provisioner [c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499] ...
I0815 01:31:57.983552 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499"
I0815 01:31:58.037566 803761 logs.go:123] Gathering logs for container status ...
I0815 01:31:58.037602 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:31:58.092420 803761 logs.go:123] Gathering logs for kube-apiserver [b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb] ...
I0815 01:31:58.092450 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb"
I0815 01:31:58.158190 803761 logs.go:123] Gathering logs for etcd [af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3] ...
I0815 01:31:58.158226 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3"
I0815 01:31:58.210070 803761 logs.go:123] Gathering logs for kube-scheduler [d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678] ...
I0815 01:31:58.210101 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678"
I0815 01:31:58.252782 803761 logs.go:123] Gathering logs for kube-controller-manager [9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3] ...
I0815 01:31:58.252811 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3"
I0815 01:31:58.329436 803761 logs.go:123] Gathering logs for kindnet [01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0] ...
I0815 01:31:58.329472 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0"
I0815 01:31:58.419135 803761 logs.go:123] Gathering logs for storage-provisioner [a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b] ...
I0815 01:31:58.419170 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b"
I0815 01:31:58.463305 803761 logs.go:123] Gathering logs for kubelet ...
I0815 01:31:58.463348 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:31:58.512928 803761 logs.go:138] Found kubelet problem: Aug 15 01:27:46 no-preload-891255 kubelet[655]: W0815 01:27:46.751836 655 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-891255" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-891255' and this object
W0815 01:31:58.513184 803761 logs.go:138] Found kubelet problem: Aug 15 01:27:46 no-preload-891255 kubelet[655]: E0815 01:27:46.751954 655 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-891255\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-891255' and this object" logger="UnhandledError"
I0815 01:31:58.547581 803761 logs.go:123] Gathering logs for etcd [adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7] ...
I0815 01:31:58.547617 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7"
I0815 01:31:58.597280 803761 logs.go:123] Gathering logs for coredns [6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af] ...
I0815 01:31:58.597310 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af"
I0815 01:31:58.642388 803761 logs.go:123] Gathering logs for kube-scheduler [914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54] ...
I0815 01:31:58.642428 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54"
I0815 01:31:58.682792 803761 logs.go:123] Gathering logs for describe nodes ...
I0815 01:31:58.682817 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:31:58.835519 803761 logs.go:123] Gathering logs for coredns [ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1] ...
I0815 01:31:58.835554 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1"
I0815 01:31:58.874278 803761 logs.go:123] Gathering logs for kubernetes-dashboard [10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84] ...
I0815 01:31:58.874305 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84"
I0815 01:31:58.920418 803761 logs.go:123] Gathering logs for containerd ...
I0815 01:31:58.920447 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:31:58.991319 803761 logs.go:123] Gathering logs for dmesg ...
I0815 01:31:58.991358 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:31:59.008024 803761 logs.go:123] Gathering logs for kube-apiserver [46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951] ...
I0815 01:31:59.008058 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951"
I0815 01:31:59.062243 803761 logs.go:123] Gathering logs for kube-proxy [f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d] ...
I0815 01:31:59.062276 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d"
I0815 01:31:59.105325 803761 logs.go:123] Gathering logs for kube-proxy [f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39] ...
I0815 01:31:59.105352 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39"
I0815 01:31:59.161607 803761 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:59.161633 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:31:59.161680 803761 out.go:239] X Problems detected in kubelet:
W0815 01:31:59.161690 803761 out.go:239] Aug 15 01:27:46 no-preload-891255 kubelet[655]: W0815 01:27:46.751836 655 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-891255" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-891255' and this object
W0815 01:31:59.161700 803761 out.go:239] Aug 15 01:27:46 no-preload-891255 kubelet[655]: E0815 01:27:46.751954 655 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-891255\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-891255' and this object" logger="UnhandledError"
I0815 01:31:59.161708 803761 out.go:304] Setting ErrFile to fd 2...
I0815 01:31:59.161724 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:32:06.105579 798414 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0815 01:32:06.118752 798414 api_server.go:72] duration metric: took 5m57.813041637s to wait for apiserver process to appear ...
I0815 01:32:06.118774 798414 api_server.go:88] waiting for apiserver healthz status ...
I0815 01:32:06.118810 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:32:06.118865 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:32:06.165617 798414 cri.go:89] found id: "5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:32:06.165640 798414 cri.go:89] found id: "d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:32:06.165646 798414 cri.go:89] found id: ""
I0815 01:32:06.165653 798414 logs.go:276] 2 containers: [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f]
I0815 01:32:06.165708 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.169565 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.173032 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:32:06.173102 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:32:06.214710 798414 cri.go:89] found id: "1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:32:06.214731 798414 cri.go:89] found id: "149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:32:06.214735 798414 cri.go:89] found id: ""
I0815 01:32:06.214743 798414 logs.go:276] 2 containers: [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d]
I0815 01:32:06.214801 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.219522 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.223511 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:32:06.223588 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:32:06.275179 798414 cri.go:89] found id: "08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:32:06.275200 798414 cri.go:89] found id: "56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:32:06.275205 798414 cri.go:89] found id: ""
I0815 01:32:06.275212 798414 logs.go:276] 2 containers: [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4]
I0815 01:32:06.275271 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.279316 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.283547 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:32:06.283635 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:32:06.325027 798414 cri.go:89] found id: "ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:32:06.325047 798414 cri.go:89] found id: "f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:32:06.325051 798414 cri.go:89] found id: ""
I0815 01:32:06.325059 798414 logs.go:276] 2 containers: [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d]
I0815 01:32:06.325114 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.328921 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.332566 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:32:06.332658 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:32:06.374132 798414 cri.go:89] found id: "65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:32:06.374165 798414 cri.go:89] found id: "8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:32:06.374173 798414 cri.go:89] found id: ""
I0815 01:32:06.374184 798414 logs.go:276] 2 containers: [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5]
I0815 01:32:06.374246 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.379778 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.383769 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:32:06.383887 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:32:06.424513 798414 cri.go:89] found id: "30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:32:06.424585 798414 cri.go:89] found id: "d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:32:06.424598 798414 cri.go:89] found id: ""
I0815 01:32:06.424607 798414 logs.go:276] 2 containers: [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f]
I0815 01:32:06.424671 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.428875 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.433119 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:32:06.433275 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:32:06.478733 798414 cri.go:89] found id: "6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:32:06.478760 798414 cri.go:89] found id: "8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:32:06.478767 798414 cri.go:89] found id: ""
I0815 01:32:06.478775 798414 logs.go:276] 2 containers: [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1]
I0815 01:32:06.478845 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.482927 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.486698 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:32:06.486788 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:32:06.536723 798414 cri.go:89] found id: "666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:32:06.536748 798414 cri.go:89] found id: ""
I0815 01:32:06.536757 798414 logs.go:276] 1 containers: [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6]
I0815 01:32:06.536832 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.540620 798414 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:32:06.540726 798414 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:32:06.577792 798414 cri.go:89] found id: "fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:32:06.577814 798414 cri.go:89] found id: "6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:32:06.577820 798414 cri.go:89] found id: ""
I0815 01:32:06.577827 798414 logs.go:276] 2 containers: [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414]
I0815 01:32:06.577881 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.581525 798414 ssh_runner.go:195] Run: which crictl
I0815 01:32:06.585154 798414 logs.go:123] Gathering logs for kube-proxy [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418] ...
I0815 01:32:06.585180 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418"
I0815 01:32:06.629579 798414 logs.go:123] Gathering logs for kubernetes-dashboard [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6] ...
I0815 01:32:06.629604 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6"
I0815 01:32:06.671944 798414 logs.go:123] Gathering logs for storage-provisioner [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3] ...
I0815 01:32:06.671982 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3"
I0815 01:32:06.713621 798414 logs.go:123] Gathering logs for container status ...
I0815 01:32:06.713664 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:32:06.771192 798414 logs.go:123] Gathering logs for describe nodes ...
I0815 01:32:06.771223 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:32:06.916807 798414 logs.go:123] Gathering logs for coredns [56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4] ...
I0815 01:32:06.916881 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4"
I0815 01:32:06.967152 798414 logs.go:123] Gathering logs for kube-scheduler [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8] ...
I0815 01:32:06.967233 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8"
I0815 01:32:07.009506 798414 logs.go:123] Gathering logs for kube-proxy [8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5] ...
I0815 01:32:07.009537 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5"
I0815 01:32:07.049321 798414 logs.go:123] Gathering logs for containerd ...
I0815 01:32:07.049360 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:32:07.113454 798414 logs.go:123] Gathering logs for kubelet ...
I0815 01:32:07.113494 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:32:07.177310 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206496 664 reflector.go:138] object-"kube-system"/"coredns-token-hqq2w": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-hqq2w" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177533 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.206838 664 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177748 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207185 664 reflector.go:138] object-"kube-system"/"kindnet-token-pflml": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-pflml" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.177967 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207498 664 reflector.go:138] object-"kube-system"/"kube-proxy-token-rl52n": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-rl52n" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178175 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.207678 664 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178488 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.264938 664 reflector.go:138] object-"kube-system"/"storage-provisioner-token-k6pcd": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-k6pcd" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.178700 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:28 old-k8s-version-145466 kubelet[664]: E0815 01:26:28.114283 664 reflector.go:138] object-"default"/"default-token-j4wgn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-j4wgn" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.181924 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.390721 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.183276 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:29 old-k8s-version-145466 kubelet[664]: E0815 01:26:29.415771 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.186634 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:42 old-k8s-version-145466 kubelet[664]: E0815 01:26:42.146871 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.187378 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:49 old-k8s-version-145466 kubelet[664]: E0815 01:26:49.166497 664 reflector.go:138] object-"kubernetes-dashboard"/"kubernetes-dashboard-token-fksp6": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kubernetes-dashboard-token-fksp6" is forbidden: User "system:node:old-k8s-version-145466" cannot list resource "secrets" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'old-k8s-version-145466' and this object
W0815 01:32:07.188913 798414 logs.go:138] Found kubelet problem: Aug 15 01:26:57 old-k8s-version-145466 kubelet[664]: E0815 01:26:57.141841 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.189372 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:00 old-k8s-version-145466 kubelet[664]: E0815 01:27:00.751687 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.189828 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:01 old-k8s-version-145466 kubelet[664]: E0815 01:27:01.756479 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.190265 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:02 old-k8s-version-145466 kubelet[664]: E0815 01:27:02.761739 664 pod_workers.go:191] Error syncing pod d7e84c38-c90e-427c-bfc5-45adf788d6fe ("storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(d7e84c38-c90e-427c-bfc5-45adf788d6fe)"
W0815 01:32:07.190931 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:08 old-k8s-version-145466 kubelet[664]: E0815 01:27:08.528394 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.193356 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:11 old-k8s-version-145466 kubelet[664]: E0815 01:27:11.140512 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.194070 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:20 old-k8s-version-145466 kubelet[664]: E0815 01:27:20.809846 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.194257 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:24 old-k8s-version-145466 kubelet[664]: E0815 01:27:24.133897 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.194585 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:28 old-k8s-version-145466 kubelet[664]: E0815 01:27:28.528406 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.194768 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:38 old-k8s-version-145466 kubelet[664]: E0815 01:27:38.186907 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.195356 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:42 old-k8s-version-145466 kubelet[664]: E0815 01:27:42.877822 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.195681 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:48 old-k8s-version-145466 kubelet[664]: E0815 01:27:48.529042 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.198128 798414 logs.go:138] Found kubelet problem: Aug 15 01:27:53 old-k8s-version-145466 kubelet[664]: E0815 01:27:53.141943 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.198456 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:02 old-k8s-version-145466 kubelet[664]: E0815 01:28:02.132198 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.198668 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:04 old-k8s-version-145466 kubelet[664]: E0815 01:28:04.133346 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.198852 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:16 old-k8s-version-145466 kubelet[664]: E0815 01:28:16.144812 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.199177 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:17 old-k8s-version-145466 kubelet[664]: E0815 01:28:17.132160 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.199766 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:29 old-k8s-version-145466 kubelet[664]: E0815 01:28:29.028736 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.199975 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:31 old-k8s-version-145466 kubelet[664]: E0815 01:28:31.132458 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.200308 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:38 old-k8s-version-145466 kubelet[664]: E0815 01:28:38.528400 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.200493 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:44 old-k8s-version-145466 kubelet[664]: E0815 01:28:44.132479 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.200817 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:51 old-k8s-version-145466 kubelet[664]: E0815 01:28:51.133407 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.201005 798414 logs.go:138] Found kubelet problem: Aug 15 01:28:56 old-k8s-version-145466 kubelet[664]: E0815 01:28:56.135291 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.201329 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:04 old-k8s-version-145466 kubelet[664]: E0815 01:29:04.132344 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.201512 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:10 old-k8s-version-145466 kubelet[664]: E0815 01:29:10.132536 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.201837 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:17 old-k8s-version-145466 kubelet[664]: E0815 01:29:17.132162 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.204300 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:23 old-k8s-version-145466 kubelet[664]: E0815 01:29:23.139752 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.204628 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:28 old-k8s-version-145466 kubelet[664]: E0815 01:29:28.139064 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.204812 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:34 old-k8s-version-145466 kubelet[664]: E0815 01:29:34.132664 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.205154 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:40 old-k8s-version-145466 kubelet[664]: E0815 01:29:40.133096 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.205337 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:49 old-k8s-version-145466 kubelet[664]: E0815 01:29:49.133068 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.205925 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:51 old-k8s-version-145466 kubelet[664]: E0815 01:29:51.281771 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206255 798414 logs.go:138] Found kubelet problem: Aug 15 01:29:58 old-k8s-version-145466 kubelet[664]: E0815 01:29:58.528848 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206438 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:00 old-k8s-version-145466 kubelet[664]: E0815 01:30:00.160446 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.206767 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:10 old-k8s-version-145466 kubelet[664]: E0815 01:30:10.135730 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.206949 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:11 old-k8s-version-145466 kubelet[664]: E0815 01:30:11.132453 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.207272 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:22 old-k8s-version-145466 kubelet[664]: E0815 01:30:22.132670 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.207454 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:26 old-k8s-version-145466 kubelet[664]: E0815 01:30:26.135386 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.207781 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:33 old-k8s-version-145466 kubelet[664]: E0815 01:30:33.132520 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.207971 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:41 old-k8s-version-145466 kubelet[664]: E0815 01:30:41.132604 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.208297 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: E0815 01:30:45.132399 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.208479 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:54 old-k8s-version-145466 kubelet[664]: E0815 01:30:54.132932 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.208803 798414 logs.go:138] Found kubelet problem: Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: E0815 01:30:56.132938 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.208985 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:06 old-k8s-version-145466 kubelet[664]: E0815 01:31:06.132954 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.209308 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: E0815 01:31:11.132193 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.209490 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:20 old-k8s-version-145466 kubelet[664]: E0815 01:31:20.132734 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.209814 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: E0815 01:31:23.132188 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.209997 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.210321 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.210506 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.210833 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.211019 798414 logs.go:138] Found kubelet problem: Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.213451 798414 logs.go:138] Found kubelet problem: Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.213780 798414 logs.go:138] Found kubelet problem: Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
I0815 01:32:07.213790 798414 logs.go:123] Gathering logs for etcd [149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d] ...
I0815 01:32:07.213804 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d"
I0815 01:32:07.268209 798414 logs.go:123] Gathering logs for coredns [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9] ...
I0815 01:32:07.268240 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9"
I0815 01:32:07.312267 798414 logs.go:123] Gathering logs for kube-scheduler [f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d] ...
I0815 01:32:07.312297 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d"
I0815 01:32:07.353856 798414 logs.go:123] Gathering logs for kube-controller-manager [d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f] ...
I0815 01:32:07.353886 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f"
I0815 01:32:07.412945 798414 logs.go:123] Gathering logs for kindnet [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b] ...
I0815 01:32:07.412980 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b"
I0815 01:32:07.480680 798414 logs.go:123] Gathering logs for etcd [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43] ...
I0815 01:32:07.480758 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43"
I0815 01:32:07.557927 798414 logs.go:123] Gathering logs for kube-controller-manager [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd] ...
I0815 01:32:07.557960 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd"
I0815 01:32:07.614762 798414 logs.go:123] Gathering logs for kindnet [8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1] ...
I0815 01:32:07.614799 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1"
I0815 01:32:07.664776 798414 logs.go:123] Gathering logs for storage-provisioner [6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414] ...
I0815 01:32:07.664822 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414"
I0815 01:32:07.703095 798414 logs.go:123] Gathering logs for dmesg ...
I0815 01:32:07.703130 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:32:07.722145 798414 logs.go:123] Gathering logs for kube-apiserver [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645] ...
I0815 01:32:07.722176 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645"
I0815 01:32:07.778182 798414 logs.go:123] Gathering logs for kube-apiserver [d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f] ...
I0815 01:32:07.778222 798414 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f"
I0815 01:32:07.835453 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:07.835484 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:32:07.835549 798414 out.go:239] X Problems detected in kubelet:
W0815 01:32:07.835563 798414 out.go:239] Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.835572 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
W0815 01:32:07.835580 798414 out.go:239] Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0815 01:32:07.835587 798414 out.go:239] Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0815 01:32:07.835601 798414 out.go:239] Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
I0815 01:32:07.835607 798414 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:07.835621 798414 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:32:09.162660 803761 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0815 01:32:09.176219 803761 api_server.go:72] duration metric: took 4m31.625848926s to wait for apiserver process to appear ...
I0815 01:32:09.176246 803761 api_server.go:88] waiting for apiserver healthz status ...
I0815 01:32:09.176282 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0815 01:32:09.176342 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0815 01:32:09.225626 803761 cri.go:89] found id: "46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951"
I0815 01:32:09.225648 803761 cri.go:89] found id: "b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb"
I0815 01:32:09.225654 803761 cri.go:89] found id: ""
I0815 01:32:09.225661 803761 logs.go:276] 2 containers: [46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951 b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb]
I0815 01:32:09.225718 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.231370 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.234938 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0815 01:32:09.235010 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0815 01:32:09.273734 803761 cri.go:89] found id: "af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3"
I0815 01:32:09.273758 803761 cri.go:89] found id: "adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7"
I0815 01:32:09.273762 803761 cri.go:89] found id: ""
I0815 01:32:09.273770 803761 logs.go:276] 2 containers: [af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3 adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7]
I0815 01:32:09.273826 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.278872 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.284237 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0815 01:32:09.284313 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0815 01:32:09.325430 803761 cri.go:89] found id: "6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af"
I0815 01:32:09.325455 803761 cri.go:89] found id: "ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1"
I0815 01:32:09.325461 803761 cri.go:89] found id: ""
I0815 01:32:09.325468 803761 logs.go:276] 2 containers: [6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1]
I0815 01:32:09.325523 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.329072 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.332735 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0815 01:32:09.332857 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0815 01:32:09.373294 803761 cri.go:89] found id: "914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54"
I0815 01:32:09.373319 803761 cri.go:89] found id: "d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678"
I0815 01:32:09.373325 803761 cri.go:89] found id: ""
I0815 01:32:09.373332 803761 logs.go:276] 2 containers: [914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54 d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678]
I0815 01:32:09.373389 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.377328 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.380946 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0815 01:32:09.381027 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0815 01:32:09.423505 803761 cri.go:89] found id: "f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d"
I0815 01:32:09.423533 803761 cri.go:89] found id: "f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39"
I0815 01:32:09.423538 803761 cri.go:89] found id: ""
I0815 01:32:09.423546 803761 logs.go:276] 2 containers: [f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39]
I0815 01:32:09.423601 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.427228 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.430939 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0815 01:32:09.431047 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0815 01:32:09.469187 803761 cri.go:89] found id: "9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3"
I0815 01:32:09.469217 803761 cri.go:89] found id: "2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22"
I0815 01:32:09.469222 803761 cri.go:89] found id: ""
I0815 01:32:09.469230 803761 logs.go:276] 2 containers: [9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3 2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22]
I0815 01:32:09.469287 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.473210 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.476881 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0815 01:32:09.477023 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0815 01:32:09.526236 803761 cri.go:89] found id: "01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0"
I0815 01:32:09.526303 803761 cri.go:89] found id: "5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd"
I0815 01:32:09.526325 803761 cri.go:89] found id: ""
I0815 01:32:09.526353 803761 logs.go:276] 2 containers: [01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0 5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd]
I0815 01:32:09.526440 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.530392 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.534110 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0815 01:32:09.534181 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0815 01:32:09.586994 803761 cri.go:89] found id: "10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84"
I0815 01:32:09.587067 803761 cri.go:89] found id: ""
I0815 01:32:09.587089 803761 logs.go:276] 1 containers: [10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84]
I0815 01:32:09.587183 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.590783 803761 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0815 01:32:09.590855 803761 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0815 01:32:09.630342 803761 cri.go:89] found id: "a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b"
I0815 01:32:09.630415 803761 cri.go:89] found id: "c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499"
I0815 01:32:09.630427 803761 cri.go:89] found id: ""
I0815 01:32:09.630436 803761 logs.go:276] 2 containers: [a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499]
I0815 01:32:09.630535 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.634191 803761 ssh_runner.go:195] Run: which crictl
I0815 01:32:09.637544 803761 logs.go:123] Gathering logs for kubelet ...
I0815 01:32:09.637616 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0815 01:32:09.685304 803761 logs.go:138] Found kubelet problem: Aug 15 01:27:46 no-preload-891255 kubelet[655]: W0815 01:27:46.751836 655 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-891255" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-891255' and this object
W0815 01:32:09.685590 803761 logs.go:138] Found kubelet problem: Aug 15 01:27:46 no-preload-891255 kubelet[655]: E0815 01:27:46.751954 655 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-891255\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-891255' and this object" logger="UnhandledError"
I0815 01:32:09.720692 803761 logs.go:123] Gathering logs for dmesg ...
I0815 01:32:09.720730 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0815 01:32:09.738598 803761 logs.go:123] Gathering logs for describe nodes ...
I0815 01:32:09.738633 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.31.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0815 01:32:09.874914 803761 logs.go:123] Gathering logs for kube-apiserver [46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951] ...
I0815 01:32:09.874946 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 46c3cc70d2d5a312e15d7057df64d22a49f37589065ccc894bc8031b4a798951"
I0815 01:32:09.936346 803761 logs.go:123] Gathering logs for etcd [af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3] ...
I0815 01:32:09.936385 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 af8ae98559be9b10288ede748b6487069e6ae8e0717a8ea5cae3f97dae2473a3"
I0815 01:32:09.987047 803761 logs.go:123] Gathering logs for coredns [ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1] ...
I0815 01:32:09.987082 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ba8534f9d63589367129cbff9b8d4f2498b7f86093c708dd66e2ccf978b436f1"
I0815 01:32:10.038745 803761 logs.go:123] Gathering logs for kube-controller-manager [9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3] ...
I0815 01:32:10.038779 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 9612b5845ae0c1e1f75da16acadeb15251654526c47e57438d92794112c4cff3"
I0815 01:32:10.122899 803761 logs.go:123] Gathering logs for kubernetes-dashboard [10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84] ...
I0815 01:32:10.122937 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 10c17d67b1c99bc8c8761398268c9dce700200fe24f2d6a147a048129527ab84"
I0815 01:32:10.174661 803761 logs.go:123] Gathering logs for storage-provisioner [c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499] ...
I0815 01:32:10.174698 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c88884152535dacfc5c62be43dc4ce7ff717b81e2d387dc5c6383a1ba266b499"
I0815 01:32:10.222050 803761 logs.go:123] Gathering logs for kube-proxy [f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39] ...
I0815 01:32:10.222092 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f3022d73668343c2effb73b4ac408bd5f9d53392de4778783676da2189835e39"
I0815 01:32:10.267800 803761 logs.go:123] Gathering logs for kube-controller-manager [2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22] ...
I0815 01:32:10.267912 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b053d2cfe8aa99ea78837e3b3ce749a8c262ba3833f5bc139cbaa2c7657cb22"
I0815 01:32:10.326327 803761 logs.go:123] Gathering logs for kube-apiserver [b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb] ...
I0815 01:32:10.326360 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 b690b96148f2277712ec84a51bec7a82401a426f009a95180cc964bb26bea9fb"
I0815 01:32:10.395215 803761 logs.go:123] Gathering logs for etcd [adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7] ...
I0815 01:32:10.395254 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 adc2315cf56607b40babf5046072a405155130dd7cd2b6c3e46f88da9eaa92f7"
I0815 01:32:10.441696 803761 logs.go:123] Gathering logs for kube-scheduler [914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54] ...
I0815 01:32:10.441729 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 914fef8b03c5cf0aba5b06b0736a9992a3324577daf003ec30aa0830552ddd54"
I0815 01:32:10.482218 803761 logs.go:123] Gathering logs for kube-scheduler [d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678] ...
I0815 01:32:10.482246 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 d9f6521d16507f27bd6386ff17f8b963c257f216da7f467c9210fb9178524678"
I0815 01:32:10.535521 803761 logs.go:123] Gathering logs for kube-proxy [f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d] ...
I0815 01:32:10.535598 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f5b1c582f2613d5f66426be71311ebe74f79284ef08901e9fc908368b140053d"
I0815 01:32:10.577446 803761 logs.go:123] Gathering logs for containerd ...
I0815 01:32:10.577474 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0815 01:32:10.638110 803761 logs.go:123] Gathering logs for coredns [6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af] ...
I0815 01:32:10.638146 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 6c08c4834233480bedf28f39119c36ba7f9f59e52e6cd70fd88af53fe3b374af"
I0815 01:32:10.680386 803761 logs.go:123] Gathering logs for kindnet [01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0] ...
I0815 01:32:10.680415 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 01f1d14032652bdf25ee75b1524737966f86b4c47f7b6e32348b4b50e444e7e0"
I0815 01:32:10.740104 803761 logs.go:123] Gathering logs for kindnet [5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd] ...
I0815 01:32:10.740143 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 5a854e83e9a5556e1a2a9298080f95e0ec2337600228c065637e73d90913c7cd"
I0815 01:32:10.785124 803761 logs.go:123] Gathering logs for storage-provisioner [a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b] ...
I0815 01:32:10.785153 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a413edeaf5acb43b535f0f12013ec424c1692ef01fdf855f50845dd0e017228b"
I0815 01:32:10.828865 803761 logs.go:123] Gathering logs for container status ...
I0815 01:32:10.828897 803761 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0815 01:32:10.876248 803761 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:10.876277 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0815 01:32:10.876372 803761 out.go:239] X Problems detected in kubelet:
W0815 01:32:10.876387 803761 out.go:239] Aug 15 01:27:46 no-preload-891255 kubelet[655]: W0815 01:27:46.751836 655 reflector.go:561] object-"kubernetes-dashboard"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:no-preload-891255" cannot list resource "configmaps" in API group "" in the namespace "kubernetes-dashboard": no relationship found between node 'no-preload-891255' and this object
W0815 01:32:10.876423 803761 out.go:239] Aug 15 01:27:46 no-preload-891255 kubelet[655]: E0815 01:27:46.751954 655 reflector.go:158] "Unhandled Error" err="object-\"kubernetes-dashboard\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:no-preload-891255\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kubernetes-dashboard\": no relationship found between node 'no-preload-891255' and this object" logger="UnhandledError"
I0815 01:32:10.876443 803761 out.go:304] Setting ErrFile to fd 2...
I0815 01:32:10.876450 803761 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0815 01:32:17.836740 798414 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0815 01:32:17.849082 798414 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0815 01:32:17.851627 798414 out.go:177]
W0815 01:32:17.854581 798414 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0815 01:32:17.854634 798414 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0815 01:32:17.854660 798414 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0815 01:32:17.854666 798414 out.go:239] *
W0815 01:32:17.856250 798414 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0815 01:32:17.863608 798414 out.go:177]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
ceecc0a151db1 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 256358d254fa2 dashboard-metrics-scraper-8d5bb5db8-g4fn2
fe2ee809aa8dd ba04bb24b9575 5 minutes ago Running storage-provisioner 2 f61c6a02b1206 storage-provisioner
666623d148109 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 d1a327b708080 kubernetes-dashboard-cd95d586-9rfcm
6d60100a03bf0 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 f61c6a02b1206 storage-provisioner
05adb0e367237 1611cd07b61d5 5 minutes ago Running busybox 1 3ce93744f731a busybox
65046ef81eb0e 25a5233254979 5 minutes ago Running kube-proxy 1 a8eb6473a66d9 kube-proxy-hdj25
08e5b39842980 db91994f4ee8f 5 minutes ago Running coredns 1 5b60a432208b4 coredns-74ff55c5b-sc7dc
6741807183f4b 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 d358b0afd2e3b kindnet-tjp7r
ff181601fe7c6 e7605f88f17d6 6 minutes ago Running kube-scheduler 1 afcfd312c5f71 kube-scheduler-old-k8s-version-145466
5733bb4495fb8 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 960d382659d6f kube-apiserver-old-k8s-version-145466
30a64d5ca6b72 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 d82bcfd9d2fb2 kube-controller-manager-old-k8s-version-145466
1b6894d83d05f 05b738aa1bc63 6 minutes ago Running etcd 1 6ec154575c292 etcd-old-k8s-version-145466
a16f69b01656d 1611cd07b61d5 6 minutes ago Exited busybox 0 c5c25271141c3 busybox
56c5221502ba9 db91994f4ee8f 8 minutes ago Exited coredns 0 8a4283617db0b coredns-74ff55c5b-sc7dc
8217cbe4e252e 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 6c20dde772567 kindnet-tjp7r
8f785a4ea8b90 25a5233254979 8 minutes ago Exited kube-proxy 0 2d21edfe13d78 kube-proxy-hdj25
d14132b610c98 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 6b9c935ace106 kube-apiserver-old-k8s-version-145466
f5298f1e3fd8d e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 6950503ba4c3c kube-scheduler-old-k8s-version-145466
149a435a4cfc7 05b738aa1bc63 8 minutes ago Exited etcd 0 cc28364af1257 etcd-old-k8s-version-145466
d52d38339364a 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 7dffe829ec7ec kube-controller-manager-old-k8s-version-145466
==> containerd <==
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.158270676Z" level=info msg="CreateContainer within sandbox \"256358d254fa2d5a65f15e6d5458cbedb81cc3040ccfaf81be360cb5755c6353\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898\""
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.159493352Z" level=info msg="StartContainer for \"69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898\""
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.245170590Z" level=info msg="StartContainer for \"69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898\" returns successfully"
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.276801621Z" level=info msg="shim disconnected" id=69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898 namespace=k8s.io
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.276862363Z" level=warning msg="cleaning up after shim disconnected" id=69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898 namespace=k8s.io
Aug 15 01:28:28 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:28.276872636Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Aug 15 01:28:29 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:29.030245271Z" level=info msg="RemoveContainer for \"6249a06c2078fc3dad88b4b501240da141283597db8273aa323f219f3d2a0f6c\""
Aug 15 01:28:29 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:28:29.037200996Z" level=info msg="RemoveContainer for \"6249a06c2078fc3dad88b4b501240da141283597db8273aa323f219f3d2a0f6c\" returns successfully"
Aug 15 01:29:23 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:23.133462632Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:29:23 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:23.138092356Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Aug 15 01:29:23 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:23.139284624Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 15 01:29:23 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:23.139394704Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.134179879Z" level=info msg="CreateContainer within sandbox \"256358d254fa2d5a65f15e6d5458cbedb81cc3040ccfaf81be360cb5755c6353\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.149054836Z" level=info msg="CreateContainer within sandbox \"256358d254fa2d5a65f15e6d5458cbedb81cc3040ccfaf81be360cb5755c6353\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87\""
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.149719031Z" level=info msg="StartContainer for \"ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87\""
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.212546873Z" level=info msg="StartContainer for \"ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87\" returns successfully"
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.238538797Z" level=info msg="shim disconnected" id=ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87 namespace=k8s.io
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.238721139Z" level=warning msg="cleaning up after shim disconnected" id=ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87 namespace=k8s.io
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.238746075Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.283263925Z" level=info msg="RemoveContainer for \"69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898\""
Aug 15 01:29:51 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:29:51.289561763Z" level=info msg="RemoveContainer for \"69aa712e81322b2e2af7b2cfe18455615127bb9d18cb3e11ed7bbbb5bb146898\" returns successfully"
Aug 15 01:32:05 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:32:05.133004407Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:32:05 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:32:05.161963823Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Aug 15 01:32:05 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:32:05.163791174Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 15 01:32:05 old-k8s-version-145466 containerd[571]: time="2024-08-15T01:32:05.163899391Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [08e5b398429806625e2d4b21b843272b9863738aaf76eccca2fdcd7f138d10d9] <==
[INFO] plugin/ready: Still waiting on: "kubernetes"
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:37783 - 29628 "HINFO IN 578344926777194319.6772010912184897932. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.019690577s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0815 01:27:02.393855 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-15 01:26:32.392745126 +0000 UTC m=+0.048514107) (total time: 30.00099711s):
Trace[2019727887]: [30.00099711s] [30.00099711s] END
E0815 01:27:02.393884 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0815 01:27:02.395346 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-15 01:26:32.394938618 +0000 UTC m=+0.050707599) (total time: 30.000386274s):
Trace[939984059]: [30.000386274s] [30.000386274s] END
E0815 01:27:02.395360 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0815 01:27:02.395768 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-15 01:26:32.395283199 +0000 UTC m=+0.051052180) (total time: 30.00046958s):
Trace[1474941318]: [30.00046958s] [30.00046958s] END
E0815 01:27:02.395791 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [56c5221502ba92aed6a2e6121e471828798a80e497e1f649f9110eb04afaffe4] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:43232 - 7411 "HINFO IN 3400014083538877995.5734621554272673838. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.011882298s
==> describe nodes <==
Name: old-k8s-version-145466
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-145466
kubernetes.io/os=linux
minikube.k8s.io/commit=a560a51f794134545edbbeb49e1ab4a0b1355168
minikube.k8s.io/name=old-k8s-version-145466
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_08_15T01_23_45_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 15 Aug 2024 01:23:42 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-145466
AcquireTime: <unset>
RenewTime: Thu, 15 Aug 2024 01:32:10 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Thu, 15 Aug 2024 01:27:19 +0000 Thu, 15 Aug 2024 01:23:36 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 15 Aug 2024 01:27:19 +0000 Thu, 15 Aug 2024 01:23:36 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 15 Aug 2024 01:27:19 +0000 Thu, 15 Aug 2024 01:23:36 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 15 Aug 2024 01:27:19 +0000 Thu, 15 Aug 2024 01:24:00 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-145466
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022360Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022360Ki
pods: 110
System Info:
Machine ID: ebd81c19db5d488baf92b14afd5371dd
System UUID: fb0de378-2e0e-4b19-ac82-306e2a38f206
Boot ID: ea2065b4-362f-4442-9b74-bf31c8d731d6
Kernel Version: 5.15.0-1067-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.20
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m44s
kube-system coredns-74ff55c5b-sc7dc 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m19s
kube-system etcd-old-k8s-version-145466 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m25s
kube-system kindnet-tjp7r 100m (5%!)(MISSING) 100m (5%!)(MISSING) 50Mi (0%!)(MISSING) 50Mi (0%!)(MISSING) 8m19s
kube-system kube-apiserver-old-k8s-version-145466 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m25s
kube-system kube-controller-manager-old-k8s-version-145466 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m25s
kube-system kube-proxy-hdj25 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m19s
kube-system kube-scheduler-old-k8s-version-145466 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m25s
kube-system metrics-server-9975d5f86-qvcw4 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m33s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m17s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-g4fn2 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m30s
kubernetes-dashboard kubernetes-dashboard-cd95d586-9rfcm 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m30s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%!)(MISSING) 100m (5%!)(MISSING)
memory 420Mi (5%!)(MISSING) 220Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m45s (x5 over 8m45s) kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m45s (x4 over 8m45s) kubelet Node old-k8s-version-145466 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m45s (x4 over 8m45s) kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientPID
Normal Starting 8m26s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m26s kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m26s kubelet Node old-k8s-version-145466 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m26s kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m26s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m19s kubelet Node old-k8s-version-145466 status is now: NodeReady
Normal Starting 8m18s kube-proxy Starting kube-proxy.
Normal Starting 6m4s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m3s (x8 over 6m3s) kubelet Node old-k8s-version-145466 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m3s (x7 over 6m3s) kubelet Node old-k8s-version-145466 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m3s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m47s kube-proxy Starting kube-proxy.
==> dmesg <==
==> etcd [149a435a4cfc7f9897ab3f6a5b272f0e4fc49d2f9292ac33c52ef454a4a9446d] <==
2024-08-15 01:23:35.718141 I | etcdserver/membership: added member 9f0758e1c58a86ed [https://192.168.85.2:2380] to cluster 68eaea490fab4e05
raft2024/08/15 01:23:36 INFO: 9f0758e1c58a86ed is starting a new election at term 1
raft2024/08/15 01:23:36 INFO: 9f0758e1c58a86ed became candidate at term 2
raft2024/08/15 01:23:36 INFO: 9f0758e1c58a86ed received MsgVoteResp from 9f0758e1c58a86ed at term 2
raft2024/08/15 01:23:36 INFO: 9f0758e1c58a86ed became leader at term 2
raft2024/08/15 01:23:36 INFO: raft.node: 9f0758e1c58a86ed elected leader 9f0758e1c58a86ed at term 2
2024-08-15 01:23:36.670519 I | etcdserver: setting up the initial cluster version to 3.4
2024-08-15 01:23:36.675225 N | etcdserver/membership: set the initial cluster version to 3.4
2024-08-15 01:23:36.675290 I | etcdserver/api: enabled capabilities for version 3.4
2024-08-15 01:23:36.675316 I | etcdserver: published {Name:old-k8s-version-145466 ClientURLs:[https://192.168.85.2:2379]} to cluster 68eaea490fab4e05
2024-08-15 01:23:36.675493 I | embed: ready to serve client requests
2024-08-15 01:23:36.677185 I | embed: serving client requests on 192.168.85.2:2379
2024-08-15 01:23:36.677392 I | embed: ready to serve client requests
2024-08-15 01:23:36.684115 I | embed: serving client requests on 127.0.0.1:2379
2024-08-15 01:24:03.047221 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:24:10.582093 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:24:20.582006 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:24:30.582062 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:24:40.582030 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:24:50.582166 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:25:00.582125 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:25:10.582141 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:25:20.582171 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:25:30.582225 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:25:40.582270 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [1b6894d83d05f469576beb2ef6554bcad7b2283941fd3fa12f5c9ce2de192c43] <==
2024-08-15 01:28:10.428912 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:28:20.428777 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:28:30.429040 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:28:40.428847 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:28:50.428916 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:00.428908 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:10.428827 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:20.428754 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:30.428814 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:40.428771 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:29:50.428849 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:00.453259 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:10.428969 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:20.428994 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:30.428833 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:40.428851 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:30:50.428762 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:00.428823 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:10.429495 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:20.429019 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:30.431826 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:40.428804 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:31:50.428688 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:32:00.431564 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-15 01:32:10.429032 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
01:32:19 up 5:14, 0 users, load average: 0.79, 1.88, 2.57
Linux old-k8s-version-145466 5.15.0-1067-aws #73~20.04.1-Ubuntu SMP Wed Jul 24 17:31:05 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [6741807183f4bf2b67c5393c3b58db57e71d92cf2e28212ea1f5069e6f3d1d7b] <==
I0815 01:31:02.761887 1 main.go:299] handling current node
I0815 01:31:12.761794 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:31:12.761829 1 main.go:299] handling current node
I0815 01:31:22.761787 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:31:22.761831 1 main.go:299] handling current node
W0815 01:31:23.317325 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0815 01:31:23.317365 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
W0815 01:31:29.849178 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0815 01:31:29.851933 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0815 01:31:32.761853 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:31:32.761891 1 main.go:299] handling current node
W0815 01:31:34.923755 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0815 01:31:34.923790 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0815 01:31:42.762201 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:31:42.762240 1 main.go:299] handling current node
I0815 01:31:52.761389 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:31:52.761423 1 main.go:299] handling current node
W0815 01:31:59.974678 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0815 01:31:59.974816 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
I0815 01:32:02.761136 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:32:02.761173 1 main.go:299] handling current node
W0815 01:32:09.702652 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0815 01:32:09.702764 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0815 01:32:12.761363 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:32:12.761569 1 main.go:299] handling current node
==> kindnet [8217cbe4e252e51d6fedbec42329d5818041e8f1e19f3b35dd319e85b22c6bb1] <==
E0815 01:24:37.624513 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
W0815 01:24:37.932821 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0815 01:24:37.932866 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0815 01:24:44.540653 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:24:44.540693 1 main.go:299] handling current node
W0815 01:24:47.723621 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0815 01:24:47.723667 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0815 01:24:54.541258 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:24:54.541290 1 main.go:299] handling current node
I0815 01:25:04.540309 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:25:04.540346 1 main.go:299] handling current node
W0815 01:25:14.069085 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0815 01:25:14.069127 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0815 01:25:14.541096 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:25:14.541133 1 main.go:299] handling current node
W0815 01:25:18.807342 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0815 01:25:18.807379 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0815 01:25:24.541171 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:25:24.541200 1 main.go:299] handling current node
W0815 01:25:24.824544 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0815 01:25:24.824579 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
I0815 01:25:34.540965 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:25:34.541009 1 main.go:299] handling current node
I0815 01:25:44.543584 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0815 01:25:44.543618 1 main.go:299] handling current node
==> kube-apiserver [5733bb4495fb8d0f6478c63776251feb2c864959867e0608d1ba170e61e5d645] <==
I0815 01:28:43.416445 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:28:43.416460 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0815 01:29:18.028608 1 client.go:360] parsed scheme: "passthrough"
I0815 01:29:18.028658 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:29:18.028669 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0815 01:29:32.475293 1 handler_proxy.go:102] no RequestInfo found in the context
E0815 01:29:32.475374 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0815 01:29:32.475383 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0815 01:29:56.178111 1 client.go:360] parsed scheme: "passthrough"
I0815 01:29:56.178354 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:29:56.178370 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0815 01:30:27.651831 1 client.go:360] parsed scheme: "passthrough"
I0815 01:30:27.651892 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:30:27.651901 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0815 01:31:07.122375 1 client.go:360] parsed scheme: "passthrough"
I0815 01:31:07.122422 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:31:07.122431 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0815 01:31:29.034847 1 handler_proxy.go:102] no RequestInfo found in the context
E0815 01:31:29.034922 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0815 01:31:29.034931 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0815 01:31:43.145654 1 client.go:360] parsed scheme: "passthrough"
I0815 01:31:43.145702 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:31:43.145711 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [d14132b610c987368bddf5640aaa164f967d7378bf448e729248592d6c4ff00f] <==
I0815 01:23:43.013548 1 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
I0815 01:23:43.013580 1 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
I0815 01:23:43.023227 1 storage_scheduling.go:132] created PriorityClass system-node-critical with value 2000001000
I0815 01:23:43.027689 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0815 01:23:43.027717 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0815 01:23:43.493924 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0815 01:23:43.526449 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0815 01:23:43.595552 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.85.2]
I0815 01:23:43.596653 1 controller.go:606] quota admission added evaluator for: endpoints
I0815 01:23:43.608076 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0815 01:23:44.073054 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0815 01:23:44.676775 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0815 01:23:45.178961 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0815 01:23:45.283638 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0815 01:24:00.767564 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0815 01:24:00.820635 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0815 01:24:18.025629 1 client.go:360] parsed scheme: "passthrough"
I0815 01:24:18.025885 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:24:18.025905 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0815 01:24:55.232287 1 client.go:360] parsed scheme: "passthrough"
I0815 01:24:55.232334 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:24:55.232343 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0815 01:25:29.987615 1 client.go:360] parsed scheme: "passthrough"
I0815 01:25:29.987660 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0815 01:25:29.987668 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [30a64d5ca6b72871b84648d9590c4402c003b7b395e81783d62d49251a4ef4fd] <==
W0815 01:27:54.752844 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:28:20.830994 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:28:26.403422 1 request.go:655] Throttling request took 1.048478243s, request: GET:https://192.168.85.2:8443/apis/apiextensions.k8s.io/v1?timeout=32s
W0815 01:28:27.255027 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:28:51.332851 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:28:58.905487 1 request.go:655] Throttling request took 1.048251571s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0815 01:28:59.756892 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:29:21.834566 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:29:31.407357 1 request.go:655] Throttling request took 1.048374805s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1beta1?timeout=32s
W0815 01:29:32.258864 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:29:52.335912 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:30:03.909335 1 request.go:655] Throttling request took 1.048331651s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0815 01:30:04.760755 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:30:22.837793 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:30:36.411301 1 request.go:655] Throttling request took 1.048442144s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0815 01:30:37.262732 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:30:53.339483 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:31:08.913104 1 request.go:655] Throttling request took 1.048313768s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0815 01:31:09.764564 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:31:23.842455 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:31:41.415164 1 request.go:655] Throttling request took 1.048517094s, request: GET:https://192.168.85.2:8443/apis/autoscaling/v1?timeout=32s
W0815 01:31:42.266735 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0815 01:31:54.345487 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0815 01:32:13.917233 1 request.go:655] Throttling request took 1.048224787s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0815 01:32:14.768664 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [d52d38339364a7a925d98b1f1d7ec2475b6fe8f625acabfb0531c1c8de3f379f] <==
I0815 01:24:00.782490 1 shared_informer.go:247] Caches are synced for TTL
I0815 01:24:00.788564 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0815 01:24:00.798091 1 shared_informer.go:247] Caches are synced for persistent volume
I0815 01:24:00.798336 1 shared_informer.go:247] Caches are synced for daemon sets
I0815 01:24:00.801807 1 shared_informer.go:247] Caches are synced for resource quota
I0815 01:24:00.803506 1 shared_informer.go:247] Caches are synced for endpoint_slice
I0815 01:24:00.854588 1 shared_informer.go:247] Caches are synced for expand
I0815 01:24:00.854941 1 shared_informer.go:247] Caches are synced for attach detach
I0815 01:24:00.912252 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-bstqt"
I0815 01:24:00.961708 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-sc7dc"
I0815 01:24:00.965299 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-hdj25"
I0815 01:24:01.000533 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-tjp7r"
I0815 01:24:01.040177 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
E0815 01:24:01.172115 1 daemon_controller.go:320] kube-system/kindnet failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kindnet", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"c2189e2e-177f-4b5c-a7b0-a13bcb776305", ResourceVersion:"282", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63859281825, loc:(*time.Location)(0x632eb80)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"},\"name\":\"kindnet\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"k
indnet\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"kindnet\",\"k8s-app\":\"kindnet\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"HOST_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.hostIP\"}}},{\"name\":\"POD_IP\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"status.podIP\"}}},{\"name\":\"POD_SUBNET\",\"value\":\"10.244.0.0/16\"}],\"image\":\"docker.io/kindest/kindnetd:v20240813-c6f155d6\",\"name\":\"kindnet-cni\",\"resources\":{\"limits\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"50Mi\"}},\"securityContext\":{\"capabilities\":{\"add\":[\"NET_RAW\",\"NET_ADMIN\"]},\"privileged\":false},\"volumeMounts\":[{\"mountPath\":\"/etc/cni/net.d\",\"name\":\"cni-cfg\"},{\"mountPath\":\"/run/xtables.lock\",\"name\":\"xtables-lock\",\"readOnly\":false},{\"mountPath\":\"/lib/modules\",\"name\":\"lib-modules\",\"readOnly\":true}]}],\"hostNetwork\":true,\"serviceAccountName\":\"kindnet\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"operator\
":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/etc/cni/net.d\",\"type\":\"DirectoryOrCreate\"},\"name\":\"cni-cfg\"},{\"hostPath\":{\"path\":\"/run/xtables.lock\",\"type\":\"FileOrCreate\"},\"name\":\"xtables-lock\"},{\"hostPath\":{\"path\":\"/lib/modules\"},\"name\":\"lib-modules\"}]}}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0x400165de20), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0x400165de40)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0x400165de60), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string
{"app":"kindnet", "k8s-app":"kindnet", "tier":"node"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"cni-cfg", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x400165de80), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil),
FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"xtables-lock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x400165dea0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.Glust
erfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"lib-modules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0x400165dec0), EmptyDi
r:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil),
PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kindnet-cni", Image:"docker.io/kindest/kindnetd:v20240813-c6f155d6", Command:[]string(nil), Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"HOST_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x400165dee0)}, v1.EnvVar{Name:"POD_IP", Value:"", ValueFrom:(*v1.EnvVarSource)(0x400165df20)}, v1.EnvVar{Name:"POD_SUBNET", Value:"10.244.0.0/16", ValueFrom:(*v1.EnvVarSource)(nil)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:
0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"cni-cfg", ReadOnly:false, MountPath:"/etc/cni/net.d", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"xtables-lock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"lib-modules", ReadOnly:true, MountPath:"/lib/modules", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:
(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0x40016630e0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0x40005a7c88), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string(nil), ServiceAccountName:"kindnet", DeprecatedServiceAccount:"kindnet", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0x4000ab7b90), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}},
HostAliases:[]v1.HostAlias(nil), PriorityClassName:"", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0x4000280540)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0x40005a7cd0)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kindnet": the object has been modified; please apply your changes to the latest version and try again
I0815 01:24:01.340859 1 shared_informer.go:247] Caches are synced for garbage collector
I0815 01:24:01.348339 1 shared_informer.go:247] Caches are synced for garbage collector
I0815 01:24:01.348358 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0815 01:24:01.549570 1 shared_informer.go:240] Waiting for caches to sync for resource quota
I0815 01:24:01.549612 1 shared_informer.go:247] Caches are synced for resource quota
I0815 01:24:02.241177 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0815 01:24:02.264040 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-bstqt"
I0815 01:24:05.750207 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0815 01:25:45.816398 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
E0815 01:25:45.979465 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
E0815 01:25:45.980526 1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
==> kube-proxy [65046ef81eb0e1914eb58367c5b7d798a0c27c3e9f4bc8bb8fbab2ee0b4a1418] <==
I0815 01:26:32.758265 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0815 01:26:32.758335 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0815 01:26:32.783588 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0815 01:26:32.783673 1 server_others.go:185] Using iptables Proxier.
I0815 01:26:32.784428 1 server.go:650] Version: v1.20.0
I0815 01:26:32.785156 1 config.go:315] Starting service config controller
I0815 01:26:32.785170 1 shared_informer.go:240] Waiting for caches to sync for service config
I0815 01:26:32.785188 1 config.go:224] Starting endpoint slice config controller
I0815 01:26:32.785191 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0815 01:26:32.885294 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0815 01:26:32.885382 1 shared_informer.go:247] Caches are synced for service config
==> kube-proxy [8f785a4ea8b90af1cd5fd5d68fa1033fe4557f9cbcc027b15570ec5b806eaea5] <==
I0815 01:24:01.960168 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0815 01:24:01.960285 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0815 01:24:01.983998 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0815 01:24:01.984093 1 server_others.go:185] Using iptables Proxier.
I0815 01:24:01.984296 1 server.go:650] Version: v1.20.0
I0815 01:24:01.984790 1 config.go:315] Starting service config controller
I0815 01:24:01.984799 1 shared_informer.go:240] Waiting for caches to sync for service config
I0815 01:24:01.986638 1 config.go:224] Starting endpoint slice config controller
I0815 01:24:01.986650 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0815 01:24:02.087061 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0815 01:24:02.087139 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [f5298f1e3fd8d068a40daf1067d57275eb2c8c6a1f8685c470fe2886ad1c900d] <==
I0815 01:23:37.937960 1 serving.go:331] Generated self-signed cert in-memory
W0815 01:23:42.208258 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0815 01:23:42.208312 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0815 01:23:42.208322 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0815 01:23:42.208328 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0815 01:23:42.381409 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0815 01:23:42.384110 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0815 01:23:42.384149 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0815 01:23:42.386949 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0815 01:23:42.420343 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0815 01:23:42.420749 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0815 01:23:42.420945 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0815 01:23:42.422845 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0815 01:23:42.429146 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0815 01:23:42.429489 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0815 01:23:42.429691 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0815 01:23:42.429883 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0815 01:23:42.430074 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0815 01:23:42.430255 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0815 01:23:42.430445 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0815 01:23:42.430628 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
I0815 01:23:44.086724 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [ff181601fe7c6d0976ee6276372afd0764d4728118e73d5eddde8cfc09c201c8] <==
I0815 01:26:20.918231 1 serving.go:331] Generated self-signed cert in-memory
I0815 01:26:29.042376 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0815 01:26:29.042470 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0815 01:26:29.042477 1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
I0815 01:26:29.042490 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0815 01:26:29.137189 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0815 01:26:29.137212 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0815 01:26:29.137237 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0815 01:26:29.137242 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0815 01:26:29.238073 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0815 01:26:29.238216 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0815 01:26:29.250610 1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController
==> kubelet <==
Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: I0815 01:30:45.131965 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:30:45 old-k8s-version-145466 kubelet[664]: E0815 01:30:45.132399 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:30:54 old-k8s-version-145466 kubelet[664]: E0815 01:30:54.132932 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: I0815 01:30:56.132194 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:30:56 old-k8s-version-145466 kubelet[664]: E0815 01:30:56.132938 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:06 old-k8s-version-145466 kubelet[664]: E0815 01:31:06.132954 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: I0815 01:31:11.131796 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:31:11 old-k8s-version-145466 kubelet[664]: E0815 01:31:11.132193 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:20 old-k8s-version-145466 kubelet[664]: E0815 01:31:20.132734 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: I0815 01:31:23.131725 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:31:23 old-k8s-version-145466 kubelet[664]: E0815 01:31:23.132188 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:31 old-k8s-version-145466 kubelet[664]: E0815 01:31:31.132519 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: I0815 01:31:38.133151 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:31:38 old-k8s-version-145466 kubelet[664]: E0815 01:31:38.133982 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:42 old-k8s-version-145466 kubelet[664]: E0815 01:31:42.132790 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: I0815 01:31:53.131928 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.132720 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:31:53 old-k8s-version-145466 kubelet[664]: E0815 01:31:53.133955 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164072 664 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164122 664 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164644 664 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-vxmd2,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-qvcw4_kube-system(0f08217
8-be0b-47dc-9d79-6089e5d972a6): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 15 01:32:05 old-k8s-version-145466 kubelet[664]: E0815 01:32:05.164689 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: I0815 01:32:06.132249 664 scope.go:95] [topologymanager] RemoveContainer - Container ID: ceecc0a151db1f23c823edd31cebc430d00d44479d0fcacd59b0a5acc4ba9f87
Aug 15 01:32:06 old-k8s-version-145466 kubelet[664]: E0815 01:32:06.132574 664 pod_workers.go:191] Error syncing pod 0ae7bccd-aa53-4eba-9226-957e7ab0147b ("dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-g4fn2_kubernetes-dashboard(0ae7bccd-aa53-4eba-9226-957e7ab0147b)"
Aug 15 01:32:16 old-k8s-version-145466 kubelet[664]: E0815 01:32:16.155763 664 pod_workers.go:191] Error syncing pod 0f082178-be0b-47dc-9d79-6089e5d972a6 ("metrics-server-9975d5f86-qvcw4_kube-system(0f082178-be0b-47dc-9d79-6089e5d972a6)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
==> kubernetes-dashboard [666623d148109918104f481f973b56d48ff8178973721f5b99dc53ea7fb06de6] <==
2024/08/15 01:26:54 Using namespace: kubernetes-dashboard
2024/08/15 01:26:54 Using in-cluster config to connect to apiserver
2024/08/15 01:26:54 Using secret token for csrf signing
2024/08/15 01:26:54 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/08/15 01:26:54 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/08/15 01:26:54 Successful initial request to the apiserver, version: v1.20.0
2024/08/15 01:26:54 Generating JWE encryption key
2024/08/15 01:26:54 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/08/15 01:26:54 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/08/15 01:26:55 Initializing JWE encryption key from synchronized object
2024/08/15 01:26:55 Creating in-cluster Sidecar client
2024/08/15 01:26:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:26:55 Serving insecurely on HTTP port: 9090
2024/08/15 01:27:25 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:27:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:28:25 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:28:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:29:25 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:29:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:30:25 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:30:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:31:25 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:31:55 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/15 01:26:54 Starting overwatch
==> storage-provisioner [6d60100a03bf073eb7d9cd0206c2b7f7ccc1226eac801a645d2f49361b8d4414] <==
I0815 01:26:32.645466 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0815 01:27:02.651915 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [fe2ee809aa8dd95a25065b4f7bcfc4da6c0dd8d21bb0d35584900de4bbf8e1e3] <==
I0815 01:27:15.246335 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0815 01:27:15.267369 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0815 01:27:15.267562 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0815 01:27:32.722309 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0815 01:27:32.722636 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-145466_6e433635-ed67-4409-b228-c693b9385afe!
I0815 01:27:32.724277 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"69e73094-9862-4159-a698-8b361262d33c", APIVersion:"v1", ResourceVersion:"847", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-145466_6e433635-ed67-4409-b228-c693b9385afe became leader
I0815 01:27:32.823919 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-145466_6e433635-ed67-4409-b228-c693b9385afe!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-145466 -n old-k8s-version-145466
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-145466 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-qvcw4
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-145466 describe pod metrics-server-9975d5f86-qvcw4
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-145466 describe pod metrics-server-9975d5f86-qvcw4: exit status 1 (202.519986ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-qvcw4" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-145466 describe pod metrics-server-9975d5f86-qvcw4: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (381.71s)