=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-127648 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0
E0819 21:11:09.997034 1145018 cert_rotation.go:171] "Unhandled Error" err="key failed with : open /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/functional-219483/client.crt: no such file or directory" logger="UnhandledError"
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-127648 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0: exit status 102 (6m20.209385911s)
-- stdout --
* [old-k8s-version-127648] minikube v1.33.1 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=19423
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/19423-1139612/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/19423-1139612/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
* Using the docker driver based on existing profile
* Starting "old-k8s-version-127648" primary control-plane node in "old-k8s-version-127648" cluster
* Pulling base image v0.0.44-1723740748-19452 ...
* Restarting existing docker container for "old-k8s-version-127648" ...
* Preparing Kubernetes v1.20.0 on containerd 1.7.20 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-127648 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
-- /stdout --
** stderr **
I0819 21:10:31.807433 1351451 out.go:345] Setting OutFile to fd 1 ...
I0819 21:10:31.807677 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:10:31.807706 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:10:31.807727 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:10:31.808011 1351451 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19423-1139612/.minikube/bin
I0819 21:10:31.808472 1351451 out.go:352] Setting JSON to false
I0819 21:10:31.812943 1351451 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":17579,"bootTime":1724084253,"procs":232,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1067-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0819 21:10:31.813059 1351451 start.go:139] virtualization:
I0819 21:10:31.816415 1351451 out.go:177] * [old-k8s-version-127648] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0819 21:10:31.820324 1351451 out.go:177] - MINIKUBE_LOCATION=19423
I0819 21:10:31.820429 1351451 notify.go:220] Checking for updates...
I0819 21:10:31.825901 1351451 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0819 21:10:31.829739 1351451 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19423-1139612/kubeconfig
I0819 21:10:31.832641 1351451 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19423-1139612/.minikube
I0819 21:10:31.835290 1351451 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0819 21:10:31.838537 1351451 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0819 21:10:31.842036 1351451 config.go:182] Loaded profile config "old-k8s-version-127648": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0819 21:10:31.845959 1351451 out.go:177] * Kubernetes 1.31.0 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.31.0
I0819 21:10:31.848844 1351451 driver.go:394] Setting default libvirt URI to qemu:///system
I0819 21:10:31.898759 1351451 docker.go:123] docker version: linux-27.1.2:Docker Engine - Community
I0819 21:10:31.898951 1351451 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0819 21:10:32.012154 1351451 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:49 OomKillDisable:true NGoroutines:68 SystemTime:2024-08-19 21:10:31.994921597 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0819 21:10:32.012318 1351451 docker.go:307] overlay module found
I0819 21:10:32.016091 1351451 out.go:177] * Using the docker driver based on existing profile
I0819 21:10:32.019003 1351451 start.go:297] selected driver: docker
I0819 21:10:32.019029 1351451 start.go:901] validating driver "docker" against &{Name:old-k8s-version-127648 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-127648 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:fa
lse MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0819 21:10:32.019138 1351451 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0819 21:10:32.019798 1351451 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0819 21:10:32.125508 1351451 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:49 OomKillDisable:true NGoroutines:68 SystemTime:2024-08-19 21:10:32.115993209 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0819 21:10:32.125863 1351451 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0819 21:10:32.125892 1351451 cni.go:84] Creating CNI manager for ""
I0819 21:10:32.125901 1351451 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0819 21:10:32.125942 1351451 start.go:340] cluster config:
{Name:old-k8s-version-127648 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-127648 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:do
cker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0819 21:10:32.128947 1351451 out.go:177] * Starting "old-k8s-version-127648" primary control-plane node in "old-k8s-version-127648" cluster
I0819 21:10:32.131505 1351451 cache.go:121] Beginning downloading kic base image for docker with containerd
I0819 21:10:32.134321 1351451 out.go:177] * Pulling base image v0.0.44-1723740748-19452 ...
I0819 21:10:32.137090 1351451 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0819 21:10:32.137163 1351451 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4
I0819 21:10:32.137179 1351451 cache.go:56] Caching tarball of preloaded images
I0819 21:10:32.137275 1351451 preload.go:172] Found /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0819 21:10:32.137291 1351451 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on containerd
I0819 21:10:32.137413 1351451 profile.go:143] Saving config to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/config.json ...
I0819 21:10:32.137643 1351451 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local docker daemon
W0819 21:10:32.177066 1351451 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d is of wrong architecture
I0819 21:10:32.177085 1351451 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d to local cache
I0819 21:10:32.177159 1351451 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local cache directory
I0819 21:10:32.177175 1351451 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local cache directory, skipping pull
I0819 21:10:32.177180 1351451 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d exists in cache, skipping pull
I0819 21:10:32.177188 1351451 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d as a tarball
I0819 21:10:32.177194 1351451 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d from local cache
I0819 21:10:32.321959 1351451 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d from cached tarball
I0819 21:10:32.321991 1351451 cache.go:194] Successfully downloaded all kic artifacts
I0819 21:10:32.322030 1351451 start.go:360] acquireMachinesLock for old-k8s-version-127648: {Name:mkb8c05c7d87494b229aca13a160b294c80d34e6 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0819 21:10:32.322098 1351451 start.go:364] duration metric: took 41.928µs to acquireMachinesLock for "old-k8s-version-127648"
I0819 21:10:32.322119 1351451 start.go:96] Skipping create...Using existing machine configuration
I0819 21:10:32.322125 1351451 fix.go:54] fixHost starting:
I0819 21:10:32.322411 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:32.353925 1351451 fix.go:112] recreateIfNeeded on old-k8s-version-127648: state=Stopped err=<nil>
W0819 21:10:32.353954 1351451 fix.go:138] unexpected machine state, will restart: <nil>
I0819 21:10:32.357075 1351451 out.go:177] * Restarting existing docker container for "old-k8s-version-127648" ...
I0819 21:10:32.360143 1351451 cli_runner.go:164] Run: docker start old-k8s-version-127648
I0819 21:10:32.818714 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:32.853351 1351451 kic.go:430] container "old-k8s-version-127648" state is running.
I0819 21:10:32.853751 1351451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-127648
I0819 21:10:32.894375 1351451 profile.go:143] Saving config to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/config.json ...
I0819 21:10:32.894617 1351451 machine.go:93] provisionDockerMachine start ...
I0819 21:10:32.894690 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:32.930659 1351451 main.go:141] libmachine: Using SSH client type: native
I0819 21:10:32.930921 1351451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34223 <nil> <nil>}
I0819 21:10:32.930951 1351451 main.go:141] libmachine: About to run SSH command:
hostname
I0819 21:10:32.931722 1351451 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:58280->127.0.0.1:34223: read: connection reset by peer
I0819 21:10:36.112275 1351451 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-127648
I0819 21:10:36.112304 1351451 ubuntu.go:169] provisioning hostname "old-k8s-version-127648"
I0819 21:10:36.112378 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:36.149973 1351451 main.go:141] libmachine: Using SSH client type: native
I0819 21:10:36.150287 1351451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34223 <nil> <nil>}
I0819 21:10:36.150299 1351451 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-127648 && echo "old-k8s-version-127648" | sudo tee /etc/hostname
I0819 21:10:36.341592 1351451 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-127648
I0819 21:10:36.341695 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:36.399653 1351451 main.go:141] libmachine: Using SSH client type: native
I0819 21:10:36.399919 1351451 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34223 <nil> <nil>}
I0819 21:10:36.399940 1351451 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-127648' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-127648/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-127648' | sudo tee -a /etc/hosts;
fi
fi
I0819 21:10:36.553669 1351451 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0819 21:10:36.553699 1351451 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19423-1139612/.minikube CaCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19423-1139612/.minikube}
I0819 21:10:36.553743 1351451 ubuntu.go:177] setting up certificates
I0819 21:10:36.553753 1351451 provision.go:84] configureAuth start
I0819 21:10:36.553825 1351451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-127648
I0819 21:10:36.580339 1351451 provision.go:143] copyHostCerts
I0819 21:10:36.580419 1351451 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem, removing ...
I0819 21:10:36.580434 1351451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem
I0819 21:10:36.580510 1351451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem (1675 bytes)
I0819 21:10:36.580609 1351451 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem, removing ...
I0819 21:10:36.580619 1351451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem
I0819 21:10:36.580651 1351451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem (1078 bytes)
I0819 21:10:36.580720 1351451 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem, removing ...
I0819 21:10:36.580731 1351451 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem
I0819 21:10:36.580756 1351451 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem (1123 bytes)
I0819 21:10:36.580810 1351451 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-127648 san=[127.0.0.1 192.168.85.2 localhost minikube old-k8s-version-127648]
I0819 21:10:37.201475 1351451 provision.go:177] copyRemoteCerts
I0819 21:10:37.201558 1351451 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0819 21:10:37.201602 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:37.239994 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:37.345867 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0819 21:10:37.394048 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0819 21:10:37.436502 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0819 21:10:37.474592 1351451 provision.go:87] duration metric: took 920.814332ms to configureAuth
I0819 21:10:37.474699 1351451 ubuntu.go:193] setting minikube options for container-runtime
I0819 21:10:37.475040 1351451 config.go:182] Loaded profile config "old-k8s-version-127648": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0819 21:10:37.475105 1351451 machine.go:96] duration metric: took 4.580473033s to provisionDockerMachine
I0819 21:10:37.475140 1351451 start.go:293] postStartSetup for "old-k8s-version-127648" (driver="docker")
I0819 21:10:37.475187 1351451 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0819 21:10:37.475326 1351451 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0819 21:10:37.475424 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:37.506393 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:37.610991 1351451 ssh_runner.go:195] Run: cat /etc/os-release
I0819 21:10:37.616730 1351451 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0819 21:10:37.616764 1351451 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0819 21:10:37.616774 1351451 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0819 21:10:37.616781 1351451 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0819 21:10:37.616791 1351451 filesync.go:126] Scanning /home/jenkins/minikube-integration/19423-1139612/.minikube/addons for local assets ...
I0819 21:10:37.616866 1351451 filesync.go:126] Scanning /home/jenkins/minikube-integration/19423-1139612/.minikube/files for local assets ...
I0819 21:10:37.616958 1351451 filesync.go:149] local asset: /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem -> 11450182.pem in /etc/ssl/certs
I0819 21:10:37.617071 1351451 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0819 21:10:37.627267 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem --> /etc/ssl/certs/11450182.pem (1708 bytes)
I0819 21:10:37.659451 1351451 start.go:296] duration metric: took 184.235206ms for postStartSetup
I0819 21:10:37.659616 1351451 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0819 21:10:37.659704 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:37.693216 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:37.790740 1351451 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0819 21:10:37.799325 1351451 fix.go:56] duration metric: took 5.477191693s for fixHost
I0819 21:10:37.799349 1351451 start.go:83] releasing machines lock for "old-k8s-version-127648", held for 5.477242326s
I0819 21:10:37.799442 1351451 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-127648
I0819 21:10:37.826333 1351451 ssh_runner.go:195] Run: cat /version.json
I0819 21:10:37.826367 1351451 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0819 21:10:37.826407 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:37.826441 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:37.858615 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:37.870112 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:38.110943 1351451 ssh_runner.go:195] Run: systemctl --version
I0819 21:10:38.116635 1351451 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0819 21:10:38.122872 1351451 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0819 21:10:38.154989 1351451 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0819 21:10:38.155141 1351451 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0819 21:10:38.166967 1351451 cni.go:259] no active bridge cni configs found in "/etc/cni/net.d" - nothing to disable
I0819 21:10:38.167039 1351451 start.go:495] detecting cgroup driver to use...
I0819 21:10:38.167089 1351451 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0819 21:10:38.167164 1351451 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0819 21:10:38.189488 1351451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0819 21:10:38.206623 1351451 docker.go:217] disabling cri-docker service (if available) ...
I0819 21:10:38.206720 1351451 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0819 21:10:38.223154 1351451 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0819 21:10:38.243426 1351451 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0819 21:10:38.387206 1351451 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0819 21:10:38.494621 1351451 docker.go:233] disabling docker service ...
I0819 21:10:38.494743 1351451 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0819 21:10:38.509929 1351451 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0819 21:10:38.522540 1351451 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0819 21:10:38.634381 1351451 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0819 21:10:38.745232 1351451 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0819 21:10:38.758961 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0819 21:10:38.793969 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0819 21:10:38.807848 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0819 21:10:38.821714 1351451 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0819 21:10:38.821830 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0819 21:10:38.833678 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0819 21:10:38.852884 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0819 21:10:38.866232 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0819 21:10:38.879603 1351451 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0819 21:10:38.891592 1351451 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0819 21:10:38.903062 1351451 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0819 21:10:38.916968 1351451 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0819 21:10:38.932376 1351451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0819 21:10:39.033828 1351451 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0819 21:10:39.238716 1351451 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0819 21:10:39.238819 1351451 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0819 21:10:39.246054 1351451 start.go:563] Will wait 60s for crictl version
I0819 21:10:39.246179 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:10:39.253073 1351451 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0819 21:10:39.314092 1351451 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.20
RuntimeApiVersion: v1
I0819 21:10:39.314219 1351451 ssh_runner.go:195] Run: containerd --version
I0819 21:10:39.343113 1351451 ssh_runner.go:195] Run: containerd --version
I0819 21:10:39.370994 1351451 out.go:177] * Preparing Kubernetes v1.20.0 on containerd 1.7.20 ...
I0819 21:10:39.373700 1351451 cli_runner.go:164] Run: docker network inspect old-k8s-version-127648 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0819 21:10:39.390185 1351451 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0819 21:10:39.393898 1351451 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0819 21:10:39.408364 1351451 kubeadm.go:883] updating cluster {Name:old-k8s-version-127648 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-127648 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/
home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0819 21:10:39.408486 1351451 preload.go:131] Checking if preload exists for k8s version v1.20.0 and runtime containerd
I0819 21:10:39.408545 1351451 ssh_runner.go:195] Run: sudo crictl images --output json
I0819 21:10:39.463488 1351451 containerd.go:627] all images are preloaded for containerd runtime.
I0819 21:10:39.463513 1351451 containerd.go:534] Images already preloaded, skipping extraction
I0819 21:10:39.463574 1351451 ssh_runner.go:195] Run: sudo crictl images --output json
I0819 21:10:39.520443 1351451 containerd.go:627] all images are preloaded for containerd runtime.
I0819 21:10:39.520505 1351451 cache_images.go:84] Images are preloaded, skipping loading
I0819 21:10:39.520534 1351451 kubeadm.go:934] updating node { 192.168.85.2 8443 v1.20.0 containerd true true} ...
I0819 21:10:39.520695 1351451 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --hostname-override=old-k8s-version-127648 --kubeconfig=/etc/kubernetes/kubelet.conf --network-plugin=cni --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-127648 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0819 21:10:39.520811 1351451 ssh_runner.go:195] Run: sudo crictl info
I0819 21:10:39.574368 1351451 cni.go:84] Creating CNI manager for ""
I0819 21:10:39.574388 1351451 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0819 21:10:39.574397 1351451 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0819 21:10:39.574419 1351451 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-127648 NodeName:old-k8s-version-127648 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt
StaticPodPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0819 21:10:39.574551 1351451 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: "old-k8s-version-127648"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0819 21:10:39.574614 1351451 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0819 21:10:39.584608 1351451 binaries.go:44] Found k8s binaries, skipping transfer
I0819 21:10:39.584726 1351451 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0819 21:10:39.593988 1351451 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (442 bytes)
I0819 21:10:39.613323 1351451 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0819 21:10:39.632628 1351451 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2125 bytes)
I0819 21:10:39.653356 1351451 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0819 21:10:39.657355 1351451 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0819 21:10:39.671698 1351451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0819 21:10:39.815919 1351451 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0819 21:10:39.835036 1351451 certs.go:68] Setting up /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648 for IP: 192.168.85.2
I0819 21:10:39.835114 1351451 certs.go:194] generating shared ca certs ...
I0819 21:10:39.835147 1351451 certs.go:226] acquiring lock for ca certs: {Name:mk862c79d80b8fe3a5df83b1592928b3403a862f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:10:39.835341 1351451 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.key
I0819 21:10:39.835426 1351451 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.key
I0819 21:10:39.835454 1351451 certs.go:256] generating profile certs ...
I0819 21:10:39.835585 1351451 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/client.key
I0819 21:10:39.835708 1351451 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/apiserver.key.a0042f6e
I0819 21:10:39.835786 1351451 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/proxy-client.key
I0819 21:10:39.835946 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018.pem (1338 bytes)
W0819 21:10:39.836018 1351451 certs.go:480] ignoring /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018_empty.pem, impossibly tiny 0 bytes
I0819 21:10:39.836062 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem (1679 bytes)
I0819 21:10:39.836121 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem (1078 bytes)
I0819 21:10:39.836176 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem (1123 bytes)
I0819 21:10:39.836247 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem (1675 bytes)
I0819 21:10:39.836345 1351451 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem (1708 bytes)
I0819 21:10:39.837405 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0819 21:10:39.892294 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0819 21:10:39.953768 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0819 21:10:40.040289 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0819 21:10:40.090732 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0819 21:10:40.135928 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0819 21:10:40.179957 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0819 21:10:40.211488 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/old-k8s-version-127648/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1679 bytes)
I0819 21:10:40.244062 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0819 21:10:40.273823 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018.pem --> /usr/share/ca-certificates/1145018.pem (1338 bytes)
I0819 21:10:40.315960 1351451 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem --> /usr/share/ca-certificates/11450182.pem (1708 bytes)
I0819 21:10:40.355511 1351451 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0819 21:10:40.383715 1351451 ssh_runner.go:195] Run: openssl version
I0819 21:10:40.391434 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0819 21:10:40.406871 1351451 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0819 21:10:40.411089 1351451 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 19 20:21 /usr/share/ca-certificates/minikubeCA.pem
I0819 21:10:40.411161 1351451 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0819 21:10:40.425164 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0819 21:10:40.443259 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1145018.pem && ln -fs /usr/share/ca-certificates/1145018.pem /etc/ssl/certs/1145018.pem"
I0819 21:10:40.458205 1351451 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1145018.pem
I0819 21:10:40.462438 1351451 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 19 20:31 /usr/share/ca-certificates/1145018.pem
I0819 21:10:40.462509 1351451 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1145018.pem
I0819 21:10:40.475897 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1145018.pem /etc/ssl/certs/51391683.0"
I0819 21:10:40.489413 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/11450182.pem && ln -fs /usr/share/ca-certificates/11450182.pem /etc/ssl/certs/11450182.pem"
I0819 21:10:40.500287 1351451 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/11450182.pem
I0819 21:10:40.504787 1351451 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 19 20:31 /usr/share/ca-certificates/11450182.pem
I0819 21:10:40.504921 1351451 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/11450182.pem
I0819 21:10:40.515288 1351451 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/11450182.pem /etc/ssl/certs/3ec20f2e.0"
I0819 21:10:40.528049 1351451 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0819 21:10:40.533915 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0819 21:10:40.543963 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0819 21:10:40.555447 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0819 21:10:40.567696 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0819 21:10:40.576305 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0819 21:10:40.583874 1351451 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0819 21:10:40.591444 1351451 kubeadm.go:392] StartCluster: {Name:old-k8s-version-127648 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-127648 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/hom
e/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0819 21:10:40.591630 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0819 21:10:40.591723 1351451 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0819 21:10:40.651198 1351451 cri.go:89] found id: "f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:10:40.651278 1351451 cri.go:89] found id: "aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:10:40.651298 1351451 cri.go:89] found id: "f2320644894e9ea7972856fdbc0a5bc4af919db56252536190f012afe85043a2"
I0819 21:10:40.651319 1351451 cri.go:89] found id: "63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:10:40.651368 1351451 cri.go:89] found id: "de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:10:40.651394 1351451 cri.go:89] found id: "ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:10:40.651414 1351451 cri.go:89] found id: "62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:10:40.651448 1351451 cri.go:89] found id: "7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:10:40.651474 1351451 cri.go:89] found id: ""
I0819 21:10:40.651560 1351451 ssh_runner.go:195] Run: sudo runc --root /run/containerd/runc/k8s.io list -f json
I0819 21:10:40.670058 1351451 cri.go:116] JSON = null
W0819 21:10:40.670159 1351451 kubeadm.go:399] unpause failed: list paused: list returned 0 containers, but ps returned 8
I0819 21:10:40.670269 1351451 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0819 21:10:40.682793 1351451 kubeadm.go:408] found existing configuration files, will attempt cluster restart
I0819 21:10:40.682864 1351451 kubeadm.go:593] restartPrimaryControlPlane start ...
I0819 21:10:40.682946 1351451 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0819 21:10:40.696090 1351451 kubeadm.go:130] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0819 21:10:40.696661 1351451 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-127648" does not appear in /home/jenkins/minikube-integration/19423-1139612/kubeconfig
I0819 21:10:40.696844 1351451 kubeconfig.go:62] /home/jenkins/minikube-integration/19423-1139612/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-127648" cluster setting kubeconfig missing "old-k8s-version-127648" context setting]
I0819 21:10:40.697198 1351451 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/kubeconfig: {Name:mk04c9370af3a3baaacd607c194f214d66561798 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:10:40.701501 1351451 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0819 21:10:40.711804 1351451 kubeadm.go:630] The running cluster does not require reconfiguration: 192.168.85.2
I0819 21:10:40.711887 1351451 kubeadm.go:597] duration metric: took 29.003271ms to restartPrimaryControlPlane
I0819 21:10:40.711913 1351451 kubeadm.go:394] duration metric: took 120.477493ms to StartCluster
I0819 21:10:40.711959 1351451 settings.go:142] acquiring lock: {Name:mk42a43a496b3883d027e9bc4cab1df0994edc4f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:10:40.712058 1351451 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/19423-1139612/kubeconfig
I0819 21:10:40.712788 1351451 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/kubeconfig: {Name:mk04c9370af3a3baaacd607c194f214d66561798 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:10:40.713056 1351451 start.go:235] Will wait 6m0s for node &{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0819 21:10:40.713472 1351451 addons.go:507] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volcano:false volumesnapshots:false yakd:false]
I0819 21:10:40.713553 1351451 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-127648"
I0819 21:10:40.713575 1351451 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-127648"
W0819 21:10:40.713581 1351451 addons.go:243] addon storage-provisioner should already be in state true
I0819 21:10:40.713606 1351451 host.go:66] Checking if "old-k8s-version-127648" exists ...
I0819 21:10:40.714068 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:40.714462 1351451 config.go:182] Loaded profile config "old-k8s-version-127648": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0819 21:10:40.714555 1351451 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-127648"
I0819 21:10:40.714609 1351451 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-127648"
I0819 21:10:40.714936 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:40.717610 1351451 addons.go:69] Setting dashboard=true in profile "old-k8s-version-127648"
I0819 21:10:40.717931 1351451 addons.go:234] Setting addon dashboard=true in "old-k8s-version-127648"
W0819 21:10:40.717962 1351451 addons.go:243] addon dashboard should already be in state true
I0819 21:10:40.718131 1351451 host.go:66] Checking if "old-k8s-version-127648" exists ...
I0819 21:10:40.717812 1351451 out.go:177] * Verifying Kubernetes components...
I0819 21:10:40.717863 1351451 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-127648"
I0819 21:10:40.719317 1351451 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-127648"
W0819 21:10:40.719624 1351451 addons.go:243] addon metrics-server should already be in state true
I0819 21:10:40.721122 1351451 host.go:66] Checking if "old-k8s-version-127648" exists ...
I0819 21:10:40.721739 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:40.722461 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:40.723031 1351451 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0819 21:10:40.785281 1351451 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-127648"
W0819 21:10:40.785304 1351451 addons.go:243] addon default-storageclass should already be in state true
I0819 21:10:40.785329 1351451 host.go:66] Checking if "old-k8s-version-127648" exists ...
I0819 21:10:40.785758 1351451 cli_runner.go:164] Run: docker container inspect old-k8s-version-127648 --format={{.State.Status}}
I0819 21:10:40.798057 1351451 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0819 21:10:40.800865 1351451 addons.go:431] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0819 21:10:40.800889 1351451 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0819 21:10:40.800952 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:40.808055 1351451 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0819 21:10:40.810777 1351451 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0819 21:10:40.817960 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0819 21:10:40.817987 1351451 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0819 21:10:40.818055 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:40.837240 1351451 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0819 21:10:40.840151 1351451 addons.go:431] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0819 21:10:40.840177 1351451 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0819 21:10:40.840359 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:40.846849 1351451 addons.go:431] installing /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:40.846880 1351451 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0819 21:10:40.846940 1351451 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-127648
I0819 21:10:40.883627 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:40.885214 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:40.909690 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:40.917931 1351451 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34223 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/old-k8s-version-127648/id_rsa Username:docker}
I0819 21:10:40.970596 1351451 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0819 21:10:41.006366 1351451 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-127648" to be "Ready" ...
I0819 21:10:41.084525 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0819 21:10:41.141217 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0819 21:10:41.141291 1351451 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0819 21:10:41.147672 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:41.166212 1351451 addons.go:431] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0819 21:10:41.166237 1351451 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0819 21:10:41.206731 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0819 21:10:41.206799 1351451 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0819 21:10:41.249074 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0819 21:10:41.249144 1351451 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
I0819 21:10:41.324056 1351451 addons.go:431] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0819 21:10:41.324131 1351451 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0819 21:10:41.360967 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0819 21:10:41.361039 1351451 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
I0819 21:10:41.436084 1351451 addons.go:431] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0819 21:10:41.436163 1351451 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0819 21:10:41.442248 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-role.yaml
I0819 21:10:41.442325 1351451 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0819 21:10:41.485555 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.485654 1351451 retry.go:31] will retry after 196.377202ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:41.492338 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.492423 1351451 retry.go:31] will retry after 140.06513ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.510747 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0819 21:10:41.512080 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0819 21:10:41.512105 1351451 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0819 21:10:41.553674 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0819 21:10:41.553708 1351451 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0819 21:10:41.577545 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0819 21:10:41.577570 1351451 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0819 21:10:41.632700 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:41.646113 1351451 addons.go:431] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0819 21:10:41.646140 1351451 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0819 21:10:41.677708 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0819 21:10:41.682979 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0819 21:10:41.690846 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.690883 1351451 retry.go:31] will retry after 179.399959ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:41.807994 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.808061 1351451 retry.go:31] will retry after 419.862184ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:41.852551 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:41.852581 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.852607 1351451 retry.go:31] will retry after 540.297667ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.852583 1351451 retry.go:31] will retry after 236.25235ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.870834 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:41.949175 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:41.949210 1351451 retry.go:31] will retry after 330.427572ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.089070 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:42.177802 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.177848 1351451 retry.go:31] will retry after 349.10568ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.228137 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:42.280598 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:42.321170 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.321255 1351451 retry.go:31] will retry after 488.893693ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:42.363796 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.363828 1351451 retry.go:31] will retry after 718.005645ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.394019 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0819 21:10:42.469216 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.469250 1351451 retry.go:31] will retry after 376.503836ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.528105 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:42.636715 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.636745 1351451 retry.go:31] will retry after 437.536352ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.811118 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:42.846575 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0819 21:10:42.906725 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.906759 1351451 retry.go:31] will retry after 563.675609ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:42.937866 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:42.937902 1351451 retry.go:31] will retry after 998.439955ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.008176 1351451 node_ready.go:53] error getting node "old-k8s-version-127648": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-127648": dial tcp 192.168.85.2:8443: connect: connection refused
I0819 21:10:43.075421 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0819 21:10:43.082971 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:43.197804 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.197851 1351451 retry.go:31] will retry after 793.067347ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:43.223830 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.223867 1351451 retry.go:31] will retry after 612.714323ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.471368 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0819 21:10:43.557249 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.557334 1351451 retry.go:31] will retry after 942.13527ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.837685 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:43.923461 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.923494 1351451 retry.go:31] will retry after 1.65024491s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:43.936741 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0819 21:10:43.992135 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:44.086590 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:44.086629 1351451 retry.go:31] will retry after 1.332632345s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:44.138164 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:44.138201 1351451 retry.go:31] will retry after 962.769833ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:44.499780 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0819 21:10:44.623057 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:44.623136 1351451 retry.go:31] will retry after 2.287201949s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:45.101319 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:45.284996 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:45.285036 1351451 retry.go:31] will retry after 1.084324381s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:45.420408 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0819 21:10:45.506901 1351451 node_ready.go:53] error getting node "old-k8s-version-127648": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-127648": dial tcp 192.168.85.2:8443: connect: connection refused
I0819 21:10:45.574229 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:45.588900 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:45.588935 1351451 retry.go:31] will retry after 1.629647865s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0819 21:10:45.694482 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:45.694524 1351451 retry.go:31] will retry after 1.690652631s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:46.369521 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:46.442499 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:46.442542 1351451 retry.go:31] will retry after 2.244735002s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:46.910581 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0819 21:10:46.991848 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:46.991885 1351451 retry.go:31] will retry after 2.069015847s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:47.218839 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0819 21:10:47.291199 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:47.291242 1351451 retry.go:31] will retry after 3.172213501s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:47.385363 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0819 21:10:47.497528 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:47.497561 1351451 retry.go:31] will retry after 3.672476856s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:47.507017 1351451 node_ready.go:53] error getting node "old-k8s-version-127648": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-127648": dial tcp 192.168.85.2:8443: connect: connection refused
I0819 21:10:48.688197 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0819 21:10:48.807898 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:48.807938 1351451 retry.go:31] will retry after 3.643169621s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:49.062114 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0819 21:10:49.260888 1351451 addons.go:457] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:49.260917 1351451 retry.go:31] will retry after 5.236532051s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0819 21:10:49.507425 1351451 node_ready.go:53] error getting node "old-k8s-version-127648": Get "https://192.168.85.2:8443/api/v1/nodes/old-k8s-version-127648": dial tcp 192.168.85.2:8443: connect: connection refused
I0819 21:10:50.463920 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0819 21:10:51.170741 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0819 21:10:52.452269 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0819 21:10:54.497786 1351451 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0819 21:10:57.747314 1351451 node_ready.go:49] node "old-k8s-version-127648" has status "Ready":"True"
I0819 21:10:57.747340 1351451 node_ready.go:38] duration metric: took 16.740876149s for node "old-k8s-version-127648" to be "Ready" ...
I0819 21:10:57.747350 1351451 pod_ready.go:36] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0819 21:10:58.022764 1351451 pod_ready.go:79] waiting up to 6m0s for pod "coredns-74ff55c5b-fj4wf" in "kube-system" namespace to be "Ready" ...
I0819 21:10:58.083204 1351451 pod_ready.go:93] pod "coredns-74ff55c5b-fj4wf" in "kube-system" namespace has status "Ready":"True"
I0819 21:10:58.083228 1351451 pod_ready.go:82] duration metric: took 60.342814ms for pod "coredns-74ff55c5b-fj4wf" in "kube-system" namespace to be "Ready" ...
I0819 21:10:58.083241 1351451 pod_ready.go:79] waiting up to 6m0s for pod "etcd-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:10:58.139204 1351451 pod_ready.go:93] pod "etcd-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"True"
I0819 21:10:58.139284 1351451 pod_ready.go:82] duration metric: took 56.034265ms for pod "etcd-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:10:58.139315 1351451 pod_ready.go:79] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:10:59.128838 1351451 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (8.664870881s)
I0819 21:10:59.128999 1351451 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (7.958209998s)
I0819 21:10:59.129032 1351451 addons.go:475] Verifying addon metrics-server=true in "old-k8s-version-127648"
I0819 21:10:59.256561 1351451 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (6.804241421s)
I0819 21:10:59.256847 1351451 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (4.759030983s)
I0819 21:10:59.260790 1351451 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-127648 addons enable metrics-server
I0819 21:10:59.265635 1351451 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0819 21:10:59.268363 1351451 addons.go:510] duration metric: took 18.554884277s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0819 21:11:00.155163 1351451 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:02.645316 1351451 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:04.645953 1351451 pod_ready.go:103] pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:05.646762 1351451 pod_ready.go:93] pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"True"
I0819 21:11:05.646789 1351451 pod_ready.go:82] duration metric: took 7.507452867s for pod "kube-apiserver-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:11:05.646803 1351451 pod_ready.go:79] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:11:07.653842 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:10.154524 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:12.653666 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:15.171378 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:17.654744 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:19.674952 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:22.154249 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:24.172829 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:26.653689 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:29.153348 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:31.154640 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:33.154697 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:35.161624 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:37.652533 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:39.653480 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:41.653570 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:43.655326 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:45.655565 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:48.155245 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:50.654345 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:52.655015 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:55.153998 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:57.183466 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:11:59.653846 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:02.156580 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:04.653671 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:07.160193 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:09.653525 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:11.655775 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:14.153884 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:16.653892 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:18.657952 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:21.154059 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:23.653805 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:25.654002 1351451 pod_ready.go:103] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:26.653004 1351451 pod_ready.go:93] pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"True"
I0819 21:12:26.653029 1351451 pod_ready.go:82] duration metric: took 1m21.006217827s for pod "kube-controller-manager-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:12:26.653042 1351451 pod_ready.go:79] waiting up to 6m0s for pod "kube-proxy-l9jdt" in "kube-system" namespace to be "Ready" ...
I0819 21:12:26.658652 1351451 pod_ready.go:93] pod "kube-proxy-l9jdt" in "kube-system" namespace has status "Ready":"True"
I0819 21:12:26.658677 1351451 pod_ready.go:82] duration metric: took 5.603221ms for pod "kube-proxy-l9jdt" in "kube-system" namespace to be "Ready" ...
I0819 21:12:26.658688 1351451 pod_ready.go:79] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:12:26.668147 1351451 pod_ready.go:93] pod "kube-scheduler-old-k8s-version-127648" in "kube-system" namespace has status "Ready":"True"
I0819 21:12:26.668172 1351451 pod_ready.go:82] duration metric: took 9.475906ms for pod "kube-scheduler-old-k8s-version-127648" in "kube-system" namespace to be "Ready" ...
I0819 21:12:26.668185 1351451 pod_ready.go:79] waiting up to 6m0s for pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace to be "Ready" ...
I0819 21:12:28.673562 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:31.174497 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:33.174684 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:35.675171 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:38.191116 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:40.674520 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:42.675355 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:45.175341 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:47.674116 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:49.675079 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:52.174906 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:54.177250 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:56.177355 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:12:58.676661 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:01.175391 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:03.674960 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:06.175294 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:08.675339 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:11.174392 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:13.174622 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:15.675666 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:18.174919 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:20.175271 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:22.674917 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:25.173768 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:27.673440 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:29.674244 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:31.674376 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:33.674890 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:36.174897 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:38.175117 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:40.175705 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:42.181717 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:44.674515 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:46.675863 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:49.174704 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:51.175110 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:53.674489 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:55.674991 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:13:57.675430 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:00.231655 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:02.674756 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:05.175239 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:07.676805 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:10.174401 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:12.175301 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:14.674778 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:17.176545 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:19.674152 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:22.175115 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:24.176959 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:26.673821 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:28.678129 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:31.174283 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:33.174491 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:35.175725 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:37.674268 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:40.174741 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:42.175401 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:44.176832 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:46.177396 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:48.682776 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:51.175706 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:53.175937 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:55.673783 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:14:57.674811 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:00.196785 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:02.674349 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:05.173705 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:07.175338 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:09.176722 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:11.674821 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:14.175635 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:16.674816 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:18.675309 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:21.175280 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:23.676677 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:26.178446 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:28.675193 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:31.175846 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:33.678605 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:36.173782 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:38.174728 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:40.675597 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:43.175235 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:45.175886 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:47.674048 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:49.674457 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:51.675075 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:53.676251 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:56.175158 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:15:58.176637 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:00.199660 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:02.685086 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:05.175119 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:07.674770 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:10.175067 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:12.674503 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:15.176484 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:17.674418 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:19.683024 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:22.175433 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:24.181379 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:26.675944 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:26.675980 1351451 pod_ready.go:82] duration metric: took 4m0.007787366s for pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace to be "Ready" ...
E0819 21:16:26.675991 1351451 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0819 21:16:26.676030 1351451 pod_ready.go:39] duration metric: took 5m28.928635211s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0819 21:16:26.676055 1351451 api_server.go:52] waiting for apiserver process to appear ...
I0819 21:16:26.676101 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0819 21:16:26.676201 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0819 21:16:26.722017 1351451 cri.go:89] found id: "863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:26.722043 1351451 cri.go:89] found id: "ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:26.722049 1351451 cri.go:89] found id: ""
I0819 21:16:26.722056 1351451 logs.go:276] 2 containers: [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce]
I0819 21:16:26.722117 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.725648 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.729023 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0819 21:16:26.729144 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0819 21:16:26.770425 1351451 cri.go:89] found id: "fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:26.770446 1351451 cri.go:89] found id: "de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:26.770451 1351451 cri.go:89] found id: ""
I0819 21:16:26.770458 1351451 logs.go:276] 2 containers: [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5]
I0819 21:16:26.770516 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.774330 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.777988 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0819 21:16:26.778056 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0819 21:16:26.831135 1351451 cri.go:89] found id: "1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:26.831205 1351451 cri.go:89] found id: "f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:26.831224 1351451 cri.go:89] found id: ""
I0819 21:16:26.831251 1351451 logs.go:276] 2 containers: [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657]
I0819 21:16:26.831331 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.835121 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.838706 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0819 21:16:26.838823 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0819 21:16:26.885971 1351451 cri.go:89] found id: "c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:26.886020 1351451 cri.go:89] found id: "62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:26.886043 1351451 cri.go:89] found id: ""
I0819 21:16:26.886073 1351451 logs.go:276] 2 containers: [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a]
I0819 21:16:26.886149 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.890432 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.894051 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0819 21:16:26.894167 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0819 21:16:26.940512 1351451 cri.go:89] found id: "3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:26.940585 1351451 cri.go:89] found id: "63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:26.940605 1351451 cri.go:89] found id: ""
I0819 21:16:26.940633 1351451 logs.go:276] 2 containers: [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e]
I0819 21:16:26.940708 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.944532 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.947982 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0819 21:16:26.948103 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0819 21:16:27.064038 1351451 cri.go:89] found id: "a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:27.064065 1351451 cri.go:89] found id: "7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:27.064071 1351451 cri.go:89] found id: ""
I0819 21:16:27.064079 1351451 logs.go:276] 2 containers: [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525]
I0819 21:16:27.064160 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.075192 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.084336 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0819 21:16:27.084446 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0819 21:16:27.209506 1351451 cri.go:89] found id: "2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:27.209543 1351451 cri.go:89] found id: "aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:27.209548 1351451 cri.go:89] found id: ""
I0819 21:16:27.209556 1351451 logs.go:276] 2 containers: [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e]
I0819 21:16:27.209653 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.214678 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.218252 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0819 21:16:27.218352 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0819 21:16:27.270015 1351451 cri.go:89] found id: "18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:27.270059 1351451 cri.go:89] found id: ""
I0819 21:16:27.270067 1351451 logs.go:276] 1 containers: [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1]
I0819 21:16:27.270165 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.274425 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0819 21:16:27.274551 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0819 21:16:27.324064 1351451 cri.go:89] found id: "3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:27.324100 1351451 cri.go:89] found id: "556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:27.324105 1351451 cri.go:89] found id: ""
I0819 21:16:27.324112 1351451 logs.go:276] 2 containers: [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd]
I0819 21:16:27.324194 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.351328 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.355700 1351451 logs.go:123] Gathering logs for coredns [f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657] ...
I0819 21:16:27.355736 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:27.406666 1351451 logs.go:123] Gathering logs for kube-scheduler [62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a] ...
I0819 21:16:27.406696 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:27.461106 1351451 logs.go:123] Gathering logs for kube-proxy [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a] ...
I0819 21:16:27.461139 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:27.513081 1351451 logs.go:123] Gathering logs for storage-provisioner [556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd] ...
I0819 21:16:27.513107 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:27.570112 1351451 logs.go:123] Gathering logs for kubelet ...
I0819 21:16:27.570155 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0819 21:16:27.636753 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.621751 665 reflector.go:138] object-"kube-system"/"kindnet-token-xptkw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xptkw" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637011 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622026 665 reflector.go:138] object-"kube-system"/"coredns-token-jj86v": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-jj86v" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637232 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622171 665 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637539 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622316 665 reflector.go:138] object-"kube-system"/"storage-provisioner-token-ssctg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-ssctg" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637752 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622453 665 reflector.go:138] object-"default"/"default-token-vbtl7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-vbtl7" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637956 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671400 665 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.638181 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671660 665 reflector.go:138] object-"kube-system"/"metrics-server-token-x764s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-x764s" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.646452 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.574230 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.646679 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.789676 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.650451 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:15 old-k8s-version-127648 kubelet[665]: E0819 21:11:15.561064 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.652683 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:22 old-k8s-version-127648 kubelet[665]: E0819 21:11:22.882357 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.653041 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:23 old-k8s-version-127648 kubelet[665]: E0819 21:11:23.880032 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.653248 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:27 old-k8s-version-127648 kubelet[665]: E0819 21:11:27.549335 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.654048 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:30 old-k8s-version-127648 kubelet[665]: E0819 21:11:30.903604 665 pod_workers.go:191] Error syncing pod 74e4d116-4e4e-4dc5-af07-3013282e840a ("storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"
W0819 21:16:27.654403 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:31 old-k8s-version-127648 kubelet[665]: E0819 21:11:31.736669 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.657331 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:41 old-k8s-version-127648 kubelet[665]: E0819 21:11:41.559000 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.658003 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:45 old-k8s-version-127648 kubelet[665]: E0819 21:11:45.947294 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.658476 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:51 old-k8s-version-127648 kubelet[665]: E0819 21:11:51.747256 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.658664 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:54 old-k8s-version-127648 kubelet[665]: E0819 21:11:54.549415 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.658852 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:06 old-k8s-version-127648 kubelet[665]: E0819 21:12:06.549632 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.659552 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:08 old-k8s-version-127648 kubelet[665]: E0819 21:12:08.101007 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.659888 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:11 old-k8s-version-127648 kubelet[665]: E0819 21:12:11.736495 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.660109 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:18 old-k8s-version-127648 kubelet[665]: E0819 21:12:18.550570 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.660468 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:22 old-k8s-version-127648 kubelet[665]: E0819 21:12:22.548984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.663052 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:29 old-k8s-version-127648 kubelet[665]: E0819 21:12:29.557879 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.663411 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:35 old-k8s-version-127648 kubelet[665]: E0819 21:12:35.549004 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.663625 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:41 old-k8s-version-127648 kubelet[665]: E0819 21:12:41.549476 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.663977 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:47 old-k8s-version-127648 kubelet[665]: E0819 21:12:47.548954 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.664192 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:56 old-k8s-version-127648 kubelet[665]: E0819 21:12:56.550012 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.664817 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:59 old-k8s-version-127648 kubelet[665]: E0819 21:12:59.243984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.665194 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:01 old-k8s-version-127648 kubelet[665]: E0819 21:13:01.736702 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.665402 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:08 old-k8s-version-127648 kubelet[665]: E0819 21:13:08.550071 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.665753 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:15 old-k8s-version-127648 kubelet[665]: E0819 21:13:15.549079 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.666078 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:21 old-k8s-version-127648 kubelet[665]: E0819 21:13:21.549597 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.666481 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:27 old-k8s-version-127648 kubelet[665]: E0819 21:13:27.549537 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.666691 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:36 old-k8s-version-127648 kubelet[665]: E0819 21:13:36.549526 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.667050 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:39 old-k8s-version-127648 kubelet[665]: E0819 21:13:39.549198 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.669612 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:50 old-k8s-version-127648 kubelet[665]: E0819 21:13:50.557660 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.670005 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:52 old-k8s-version-127648 kubelet[665]: E0819 21:13:52.549116 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.670216 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:01 old-k8s-version-127648 kubelet[665]: E0819 21:14:01.549487 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.670581 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:06 old-k8s-version-127648 kubelet[665]: E0819 21:14:06.548964 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.670789 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:16 old-k8s-version-127648 kubelet[665]: E0819 21:14:16.553951 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.671457 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:20 old-k8s-version-127648 kubelet[665]: E0819 21:14:20.469865 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.671816 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:21 old-k8s-version-127648 kubelet[665]: E0819 21:14:21.736453 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.672060 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:31 old-k8s-version-127648 kubelet[665]: E0819 21:14:31.549363 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.672464 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:36 old-k8s-version-127648 kubelet[665]: E0819 21:14:36.549968 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.672679 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:46 old-k8s-version-127648 kubelet[665]: E0819 21:14:46.549614 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.673041 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:48 old-k8s-version-127648 kubelet[665]: E0819 21:14:48.551815 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.673268 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:57 old-k8s-version-127648 kubelet[665]: E0819 21:14:57.549314 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.673723 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:03 old-k8s-version-127648 kubelet[665]: E0819 21:15:03.548948 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.673940 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:08 old-k8s-version-127648 kubelet[665]: E0819 21:15:08.550121 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.674342 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: E0819 21:15:18.549558 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.674597 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:19 old-k8s-version-127648 kubelet[665]: E0819 21:15:19.549582 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675013 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: E0819 21:15:31.549167 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.675241 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:34 old-k8s-version-127648 kubelet[665]: E0819 21:15:34.549642 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675558 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.549479 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675963 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.676343 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.676553 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.676780 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.677133 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
I0819 21:16:27.677148 1351451 logs.go:123] Gathering logs for kube-apiserver [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e] ...
I0819 21:16:27.677163 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:27.783187 1351451 logs.go:123] Gathering logs for kube-apiserver [ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce] ...
I0819 21:16:27.783234 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:27.878025 1351451 logs.go:123] Gathering logs for etcd [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267] ...
I0819 21:16:27.878059 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:27.971482 1351451 logs.go:123] Gathering logs for kubernetes-dashboard [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1] ...
I0819 21:16:27.971509 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:28.026406 1351451 logs.go:123] Gathering logs for container status ...
I0819 21:16:28.026441 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0819 21:16:28.077604 1351451 logs.go:123] Gathering logs for dmesg ...
I0819 21:16:28.077635 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0819 21:16:28.106351 1351451 logs.go:123] Gathering logs for describe nodes ...
I0819 21:16:28.106383 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0819 21:16:28.314793 1351451 logs.go:123] Gathering logs for etcd [de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5] ...
I0819 21:16:28.314818 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:28.388732 1351451 logs.go:123] Gathering logs for kindnet [aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e] ...
I0819 21:16:28.388809 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:28.461767 1351451 logs.go:123] Gathering logs for kube-controller-manager [7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525] ...
I0819 21:16:28.461855 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:28.551587 1351451 logs.go:123] Gathering logs for kindnet [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20] ...
I0819 21:16:28.551688 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:28.653556 1351451 logs.go:123] Gathering logs for storage-provisioner [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e] ...
I0819 21:16:28.653593 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:28.695383 1351451 logs.go:123] Gathering logs for containerd ...
I0819 21:16:28.695411 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0819 21:16:28.762252 1351451 logs.go:123] Gathering logs for coredns [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e] ...
I0819 21:16:28.762289 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:28.820470 1351451 logs.go:123] Gathering logs for kube-scheduler [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4] ...
I0819 21:16:28.820516 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:28.873663 1351451 logs.go:123] Gathering logs for kube-proxy [63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e] ...
I0819 21:16:28.873701 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:28.917142 1351451 logs.go:123] Gathering logs for kube-controller-manager [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363] ...
I0819 21:16:28.917183 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:28.979164 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:28.979196 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0819 21:16:28.979271 1351451 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0819 21:16:28.979288 1351451 out.go:270] Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:28.979296 1351451 out.go:270] Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:28.979318 1351451 out.go:270] Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:28.979328 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:28.979333 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
I0819 21:16:28.979346 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:28.979354 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:38.980698 1351451 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0819 21:16:39.000948 1351451 api_server.go:72] duration metric: took 5m58.287822906s to wait for apiserver process to appear ...
I0819 21:16:39.000980 1351451 api_server.go:88] waiting for apiserver healthz status ...
I0819 21:16:39.001038 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0819 21:16:39.001135 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0819 21:16:39.096959 1351451 cri.go:89] found id: "863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:39.096986 1351451 cri.go:89] found id: "ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:39.096998 1351451 cri.go:89] found id: ""
I0819 21:16:39.097007 1351451 logs.go:276] 2 containers: [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce]
I0819 21:16:39.097082 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.103146 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.108342 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0819 21:16:39.108415 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0819 21:16:39.181901 1351451 cri.go:89] found id: "fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:39.181922 1351451 cri.go:89] found id: "de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:39.181931 1351451 cri.go:89] found id: ""
I0819 21:16:39.181942 1351451 logs.go:276] 2 containers: [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5]
I0819 21:16:39.182019 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.187650 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.195541 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0819 21:16:39.195721 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0819 21:16:39.279675 1351451 cri.go:89] found id: "1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:39.279764 1351451 cri.go:89] found id: "f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:39.279788 1351451 cri.go:89] found id: ""
I0819 21:16:39.279817 1351451 logs.go:276] 2 containers: [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657]
I0819 21:16:39.280034 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.285431 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.290629 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0819 21:16:39.290859 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0819 21:16:39.371561 1351451 cri.go:89] found id: "c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:39.371650 1351451 cri.go:89] found id: "62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:39.371676 1351451 cri.go:89] found id: ""
I0819 21:16:39.371728 1351451 logs.go:276] 2 containers: [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a]
I0819 21:16:39.371851 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.376912 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.383459 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0819 21:16:39.383627 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0819 21:16:39.448903 1351451 cri.go:89] found id: "3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:39.448982 1351451 cri.go:89] found id: "63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:39.449012 1351451 cri.go:89] found id: ""
I0819 21:16:39.449034 1351451 logs.go:276] 2 containers: [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e]
I0819 21:16:39.449149 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.454509 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.459726 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0819 21:16:39.459909 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0819 21:16:39.545859 1351451 cri.go:89] found id: "a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:39.545936 1351451 cri.go:89] found id: "7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:39.545961 1351451 cri.go:89] found id: ""
I0819 21:16:39.545981 1351451 logs.go:276] 2 containers: [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525]
I0819 21:16:39.546083 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.561290 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.565853 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0819 21:16:39.566023 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0819 21:16:39.630565 1351451 cri.go:89] found id: "2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:39.630648 1351451 cri.go:89] found id: "aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:39.630667 1351451 cri.go:89] found id: ""
I0819 21:16:39.630691 1351451 logs.go:276] 2 containers: [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e]
I0819 21:16:39.630799 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.636088 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.640406 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0819 21:16:39.640539 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0819 21:16:39.705002 1351451 cri.go:89] found id: "18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:39.705082 1351451 cri.go:89] found id: ""
I0819 21:16:39.705109 1351451 logs.go:276] 1 containers: [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1]
I0819 21:16:39.705209 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.713612 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0819 21:16:39.713752 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0819 21:16:39.771883 1351451 cri.go:89] found id: "3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:39.771963 1351451 cri.go:89] found id: "556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:39.771982 1351451 cri.go:89] found id: ""
I0819 21:16:39.772007 1351451 logs.go:276] 2 containers: [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd]
I0819 21:16:39.772133 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.777471 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.782219 1351451 logs.go:123] Gathering logs for dmesg ...
I0819 21:16:39.782306 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0819 21:16:39.803506 1351451 logs.go:123] Gathering logs for kube-apiserver [ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce] ...
I0819 21:16:39.803589 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:39.902538 1351451 logs.go:123] Gathering logs for kube-controller-manager [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363] ...
I0819 21:16:39.902629 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:39.988902 1351451 logs.go:123] Gathering logs for storage-provisioner [556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd] ...
I0819 21:16:39.988936 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:40.051419 1351451 logs.go:123] Gathering logs for container status ...
I0819 21:16:40.051502 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0819 21:16:40.130468 1351451 logs.go:123] Gathering logs for describe nodes ...
I0819 21:16:40.130553 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0819 21:16:40.440578 1351451 logs.go:123] Gathering logs for kube-apiserver [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e] ...
I0819 21:16:40.440661 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:40.552968 1351451 logs.go:123] Gathering logs for etcd [de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5] ...
I0819 21:16:40.553065 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:40.613652 1351451 logs.go:123] Gathering logs for coredns [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e] ...
I0819 21:16:40.613737 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:40.704428 1351451 logs.go:123] Gathering logs for coredns [f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657] ...
I0819 21:16:40.704502 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:40.762919 1351451 logs.go:123] Gathering logs for storage-provisioner [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e] ...
I0819 21:16:40.762995 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:40.818047 1351451 logs.go:123] Gathering logs for kubernetes-dashboard [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1] ...
I0819 21:16:40.818117 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:40.872667 1351451 logs.go:123] Gathering logs for kube-scheduler [62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a] ...
I0819 21:16:40.872739 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:40.943174 1351451 logs.go:123] Gathering logs for kube-proxy [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a] ...
I0819 21:16:40.943243 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:41.028487 1351451 logs.go:123] Gathering logs for kube-proxy [63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e] ...
I0819 21:16:41.028563 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:41.099036 1351451 logs.go:123] Gathering logs for kube-controller-manager [7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525] ...
I0819 21:16:41.099118 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:41.217328 1351451 logs.go:123] Gathering logs for kindnet [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20] ...
I0819 21:16:41.217417 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:41.421061 1351451 logs.go:123] Gathering logs for kindnet [aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e] ...
I0819 21:16:41.421169 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:41.500448 1351451 logs.go:123] Gathering logs for kubelet ...
I0819 21:16:41.500533 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0819 21:16:41.586682 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.621751 665 reflector.go:138] object-"kube-system"/"kindnet-token-xptkw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xptkw" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.586985 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622026 665 reflector.go:138] object-"kube-system"/"coredns-token-jj86v": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-jj86v" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587223 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622171 665 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587487 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622316 665 reflector.go:138] object-"kube-system"/"storage-provisioner-token-ssctg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-ssctg" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587731 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622453 665 reflector.go:138] object-"default"/"default-token-vbtl7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-vbtl7" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587965 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671400 665 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.588235 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671660 665 reflector.go:138] object-"kube-system"/"metrics-server-token-x764s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-x764s" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.597320 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.574230 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.597559 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.789676 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.600560 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:15 old-k8s-version-127648 kubelet[665]: E0819 21:11:15.561064 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.602849 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:22 old-k8s-version-127648 kubelet[665]: E0819 21:11:22.882357 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.603229 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:23 old-k8s-version-127648 kubelet[665]: E0819 21:11:23.880032 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.603468 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:27 old-k8s-version-127648 kubelet[665]: E0819 21:11:27.549335 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.604309 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:30 old-k8s-version-127648 kubelet[665]: E0819 21:11:30.903604 665 pod_workers.go:191] Error syncing pod 74e4d116-4e4e-4dc5-af07-3013282e840a ("storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"
W0819 21:16:41.604680 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:31 old-k8s-version-127648 kubelet[665]: E0819 21:11:31.736669 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.607638 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:41 old-k8s-version-127648 kubelet[665]: E0819 21:11:41.559000 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.608385 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:45 old-k8s-version-127648 kubelet[665]: E0819 21:11:45.947294 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.608989 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:51 old-k8s-version-127648 kubelet[665]: E0819 21:11:51.747256 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.609218 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:54 old-k8s-version-127648 kubelet[665]: E0819 21:11:54.549415 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.609955 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:06 old-k8s-version-127648 kubelet[665]: E0819 21:12:06.549632 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.610632 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:08 old-k8s-version-127648 kubelet[665]: E0819 21:12:08.101007 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.611019 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:11 old-k8s-version-127648 kubelet[665]: E0819 21:12:11.736495 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.611240 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:18 old-k8s-version-127648 kubelet[665]: E0819 21:12:18.550570 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.611614 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:22 old-k8s-version-127648 kubelet[665]: E0819 21:12:22.548984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.614201 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:29 old-k8s-version-127648 kubelet[665]: E0819 21:12:29.557879 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.614589 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:35 old-k8s-version-127648 kubelet[665]: E0819 21:12:35.549004 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.614826 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:41 old-k8s-version-127648 kubelet[665]: E0819 21:12:41.549476 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.615210 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:47 old-k8s-version-127648 kubelet[665]: E0819 21:12:47.548954 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.615441 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:56 old-k8s-version-127648 kubelet[665]: E0819 21:12:56.550012 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.616091 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:59 old-k8s-version-127648 kubelet[665]: E0819 21:12:59.243984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.616472 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:01 old-k8s-version-127648 kubelet[665]: E0819 21:13:01.736702 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.616702 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:08 old-k8s-version-127648 kubelet[665]: E0819 21:13:08.550071 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.618906 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:15 old-k8s-version-127648 kubelet[665]: E0819 21:13:15.549079 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.619148 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:21 old-k8s-version-127648 kubelet[665]: E0819 21:13:21.549597 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.619523 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:27 old-k8s-version-127648 kubelet[665]: E0819 21:13:27.549537 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.619742 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:36 old-k8s-version-127648 kubelet[665]: E0819 21:13:36.549526 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.620148 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:39 old-k8s-version-127648 kubelet[665]: E0819 21:13:39.549198 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.622878 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:50 old-k8s-version-127648 kubelet[665]: E0819 21:13:50.557660 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.623251 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:52 old-k8s-version-127648 kubelet[665]: E0819 21:13:52.549116 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.623473 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:01 old-k8s-version-127648 kubelet[665]: E0819 21:14:01.549487 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.623831 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:06 old-k8s-version-127648 kubelet[665]: E0819 21:14:06.548964 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.624043 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:16 old-k8s-version-127648 kubelet[665]: E0819 21:14:16.553951 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.624685 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:20 old-k8s-version-127648 kubelet[665]: E0819 21:14:20.469865 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625055 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:21 old-k8s-version-127648 kubelet[665]: E0819 21:14:21.736453 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625267 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:31 old-k8s-version-127648 kubelet[665]: E0819 21:14:31.549363 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.625624 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:36 old-k8s-version-127648 kubelet[665]: E0819 21:14:36.549968 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625840 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:46 old-k8s-version-127648 kubelet[665]: E0819 21:14:46.549614 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.626210 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:48 old-k8s-version-127648 kubelet[665]: E0819 21:14:48.551815 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.626434 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:57 old-k8s-version-127648 kubelet[665]: E0819 21:14:57.549314 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.626812 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:03 old-k8s-version-127648 kubelet[665]: E0819 21:15:03.548948 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.627047 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:08 old-k8s-version-127648 kubelet[665]: E0819 21:15:08.550121 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.627407 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: E0819 21:15:18.549558 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.627629 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:19 old-k8s-version-127648 kubelet[665]: E0819 21:15:19.549582 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.627992 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: E0819 21:15:31.549167 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.628208 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:34 old-k8s-version-127648 kubelet[665]: E0819 21:15:34.549642 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.628441 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.549479 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.628833 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.632353 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.632616 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.632858 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.633216 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.633561 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.633784 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.636275 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
I0819 21:16:41.636305 1351451 logs.go:123] Gathering logs for etcd [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267] ...
I0819 21:16:41.636337 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:41.736897 1351451 logs.go:123] Gathering logs for kube-scheduler [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4] ...
I0819 21:16:41.736971 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:41.824699 1351451 logs.go:123] Gathering logs for containerd ...
I0819 21:16:41.824726 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0819 21:16:41.905377 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:41.905452 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0819 21:16:41.905554 1351451 out.go:270] X Problems detected in kubelet:
X Problems detected in kubelet:
W0819 21:16:41.905600 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.905771 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.905806 1351451 out.go:270] Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.905859 1351451 out.go:270] Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.905906 1351451 out.go:270] Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
I0819 21:16:41.905968 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:41.905993 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:51.906826 1351451 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0819 21:16:51.922052 1351451 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0819 21:16:51.923612 1351451 out.go:201]
W0819 21:16:51.924862 1351451 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0819 21:16:51.925047 1351451 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0819 21:16:51.925129 1351451 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0819 21:16:51.925214 1351451 out.go:270] *
*
W0819 21:16:51.926428 1351451 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0819 21:16:51.927283 1351451 out.go:201]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-127648 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=containerd --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-127648
helpers_test.go:235: (dbg) docker inspect old-k8s-version-127648:
-- stdout --
[
{
"Id": "25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f",
"Created": "2024-08-19T21:07:18.307660453Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 1351802,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-08-19T21:10:32.551900663Z",
"FinishedAt": "2024-08-19T21:10:31.126169811Z"
},
"Image": "sha256:decdd59746a9dba10062a73f6cd4b910c7b4e60613660b1022f8357747681c4d",
"ResolvConfPath": "/var/lib/docker/containers/25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f/hostname",
"HostsPath": "/var/lib/docker/containers/25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f/hosts",
"LogPath": "/var/lib/docker/containers/25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f/25bf339acbc438fb617fe77b6aae580c5c20e7366b20f55663fad012efdbe96f-json.log",
"Name": "/old-k8s-version-127648",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-127648:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-127648",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/7663f8afeabfc8de23c79288e06286482060e83ccc32d878bbe716570d01503f-init/diff:/var/lib/docker/overlay2/56755d81a5447e9a4d21cbfbceb5eeee713182a8ca21fd0322f2eb2e99f83e1f/diff",
"MergedDir": "/var/lib/docker/overlay2/7663f8afeabfc8de23c79288e06286482060e83ccc32d878bbe716570d01503f/merged",
"UpperDir": "/var/lib/docker/overlay2/7663f8afeabfc8de23c79288e06286482060e83ccc32d878bbe716570d01503f/diff",
"WorkDir": "/var/lib/docker/overlay2/7663f8afeabfc8de23c79288e06286482060e83ccc32d878bbe716570d01503f/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Type": "volume",
"Name": "old-k8s-version-127648",
"Source": "/var/lib/docker/volumes/old-k8s-version-127648/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "old-k8s-version-127648",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-127648",
"name.minikube.sigs.k8s.io": "old-k8s-version-127648",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "7e22ca9502a6e7c0ab1675d177df9f74021350a53b5d78a37287d9fc788bc5a0",
"SandboxKey": "/var/run/docker/netns/7e22ca9502a6",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34223"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34224"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34227"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34225"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "34226"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-127648": {
"IPAMConfig": {
"IPv4Address": "192.168.85.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:55:02",
"DriverOpts": null,
"NetworkID": "17c0eb13a6bc73a0a11142e085a84cb4b821e3ea4f2f0e538e5a55422621cfee",
"EndpointID": "9e55644721d53fd545c666d6a59436c4b0061a7507f2a6416829aaed89dc0dda",
"Gateway": "192.168.85.1",
"IPAddress": "192.168.85.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DNSNames": [
"old-k8s-version-127648",
"25bf339acbc4"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-127648 -n old-k8s-version-127648
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-127648 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-127648 logs -n 25: (2.826109866s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
| start | -p cert-expiration-557259 | cert-expiration-557259 | jenkins | v1.33.1 | 19 Aug 24 21:06 UTC | 19 Aug 24 21:06 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=3m | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | force-systemd-env-705767 | force-systemd-env-705767 | jenkins | v1.33.1 | 19 Aug 24 21:06 UTC | 19 Aug 24 21:06 UTC |
| | ssh cat | | | | | |
| | /etc/containerd/config.toml | | | | | |
| delete | -p force-systemd-env-705767 | force-systemd-env-705767 | jenkins | v1.33.1 | 19 Aug 24 21:06 UTC | 19 Aug 24 21:06 UTC |
| start | -p cert-options-913353 | cert-options-913353 | jenkins | v1.33.1 | 19 Aug 24 21:06 UTC | 19 Aug 24 21:07 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| ssh | cert-options-913353 ssh | cert-options-913353 | jenkins | v1.33.1 | 19 Aug 24 21:07 UTC | 19 Aug 24 21:07 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-913353 -- sudo | cert-options-913353 | jenkins | v1.33.1 | 19 Aug 24 21:07 UTC | 19 Aug 24 21:07 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-913353 | cert-options-913353 | jenkins | v1.33.1 | 19 Aug 24 21:07 UTC | 19 Aug 24 21:07 UTC |
| start | -p old-k8s-version-127648 | old-k8s-version-127648 | jenkins | v1.33.1 | 19 Aug 24 21:07 UTC | 19 Aug 24 21:10 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-557259 | cert-expiration-557259 | jenkins | v1.33.1 | 19 Aug 24 21:09 UTC | 19 Aug 24 21:09 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| delete | -p cert-expiration-557259 | cert-expiration-557259 | jenkins | v1.33.1 | 19 Aug 24 21:09 UTC | 19 Aug 24 21:10 UTC |
| start | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:10 UTC | 19 Aug 24 21:11 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
| addons | enable metrics-server -p old-k8s-version-127648 | old-k8s-version-127648 | jenkins | v1.33.1 | 19 Aug 24 21:10 UTC | 19 Aug 24 21:10 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-127648 | old-k8s-version-127648 | jenkins | v1.33.1 | 19 Aug 24 21:10 UTC | 19 Aug 24 21:10 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-127648 | old-k8s-version-127648 | jenkins | v1.33.1 | 19 Aug 24 21:10 UTC | 19 Aug 24 21:10 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-127648 | old-k8s-version-127648 | jenkins | v1.33.1 | 19 Aug 24 21:10 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:11 UTC | 19 Aug 24 21:11 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:11 UTC | 19 Aug 24 21:11 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:11 UTC | 19 Aug 24 21:11 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:11 UTC | 19 Aug 24 21:16 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
| image | no-preload-785099 image list | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | 19 Aug 24 21:16 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | 19 Aug 24 21:16 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | 19 Aug 24 21:16 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | 19 Aug 24 21:16 UTC |
| delete | -p no-preload-785099 | no-preload-785099 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | 19 Aug 24 21:16 UTC |
| start | -p embed-certs-249735 | embed-certs-249735 | jenkins | v1.33.1 | 19 Aug 24 21:16 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=containerd | | | | | |
| | --kubernetes-version=v1.31.0 | | | | | |
|---------|--------------------------------------------------------|--------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/08/19 21:16:23
Running on machine: ip-172-31-29-130
Binary: Built with gc go1.22.5 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0819 21:16:23.688954 1361984 out.go:345] Setting OutFile to fd 1 ...
I0819 21:16:23.689094 1361984 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:23.689105 1361984 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:23.689110 1361984 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:23.689358 1361984 root.go:338] Updating PATH: /home/jenkins/minikube-integration/19423-1139612/.minikube/bin
I0819 21:16:23.689773 1361984 out.go:352] Setting JSON to false
I0819 21:16:23.690813 1361984 start.go:129] hostinfo: {"hostname":"ip-172-31-29-130","uptime":17930,"bootTime":1724084253,"procs":233,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1067-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"36adf542-ef4f-4e2d-a0c8-6868d1383ff9"}
I0819 21:16:23.690894 1361984 start.go:139] virtualization:
I0819 21:16:23.694187 1361984 out.go:177] * [embed-certs-249735] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0819 21:16:23.697575 1361984 out.go:177] - MINIKUBE_LOCATION=19423
I0819 21:16:23.697649 1361984 notify.go:220] Checking for updates...
I0819 21:16:23.702814 1361984 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0819 21:16:23.705441 1361984 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/19423-1139612/kubeconfig
I0819 21:16:23.708105 1361984 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/19423-1139612/.minikube
I0819 21:16:23.710734 1361984 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0819 21:16:23.713293 1361984 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0819 21:16:23.716445 1361984 config.go:182] Loaded profile config "old-k8s-version-127648": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.20.0
I0819 21:16:23.716547 1361984 driver.go:394] Setting default libvirt URI to qemu:///system
I0819 21:16:23.741068 1361984 docker.go:123] docker version: linux-27.1.2:Docker Engine - Community
I0819 21:16:23.741182 1361984 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0819 21:16:23.818298 1361984 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-19 21:16:23.807700033 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0819 21:16:23.818424 1361984 docker.go:307] overlay module found
I0819 21:16:23.821419 1361984 out.go:177] * Using the docker driver based on user configuration
I0819 21:16:23.823970 1361984 start.go:297] selected driver: docker
I0819 21:16:23.823986 1361984 start.go:901] validating driver "docker" against <nil>
I0819 21:16:23.824000 1361984 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0819 21:16:23.824733 1361984 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0819 21:16:23.871438 1361984 info.go:266] docker info: {ID:U5VK:ZNT5:35M3:FHLW:Q7TL:ELFX:BNAG:AV4T:UD2H:SK5L:SEJV:SJJL Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:35 OomKillDisable:true NGoroutines:53 SystemTime:2024-08-19 21:16:23.862037674 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1067-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8214896640 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-29-130 Labels:[] ExperimentalBuild:false ServerVersion:27.1.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:8fc6bcff51318944179630522a095cc9dbf9f353 Expected:8fc6bcff51318944179630522a095cc9dbf9f353} RuncCommit:{ID:v1.1.13-0-g58aa920 Expected:v1.1.13-0-g58aa920} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.16.2] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.29.1]] Warnings:<nil>}}
I0819 21:16:23.871605 1361984 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0819 21:16:23.871843 1361984 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0819 21:16:23.874471 1361984 out.go:177] * Using Docker driver with root privileges
I0819 21:16:23.877158 1361984 cni.go:84] Creating CNI manager for ""
I0819 21:16:23.877192 1361984 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0819 21:16:23.877203 1361984 start_flags.go:319] Found "CNI" CNI - setting NetworkPlugin=cni
I0819 21:16:23.877289 1361984 start.go:340] cluster config:
{Name:embed-certs-249735 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:embed-certs-249735 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: Stati
cIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0819 21:16:23.880143 1361984 out.go:177] * Starting "embed-certs-249735" primary control-plane node in "embed-certs-249735" cluster
I0819 21:16:23.882702 1361984 cache.go:121] Beginning downloading kic base image for docker with containerd
I0819 21:16:23.885412 1361984 out.go:177] * Pulling base image v0.0.44-1723740748-19452 ...
I0819 21:16:23.888184 1361984 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime containerd
I0819 21:16:23.888278 1361984 preload.go:146] Found local preload: /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.0-containerd-overlay2-arm64.tar.lz4
I0819 21:16:23.888276 1361984 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local docker daemon
I0819 21:16:23.888307 1361984 cache.go:56] Caching tarball of preloaded images
I0819 21:16:23.888398 1361984 preload.go:172] Found /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.0-containerd-overlay2-arm64.tar.lz4 in cache, skipping download
I0819 21:16:23.888409 1361984 cache.go:59] Finished verifying existence of preloaded tar for v1.31.0 on containerd
I0819 21:16:23.888514 1361984 profile.go:143] Saving config to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/config.json ...
I0819 21:16:23.888539 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/config.json: {Name:mk091da1bb8ac7d79dd35dbce043f4eda208eed5 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
W0819 21:16:23.915523 1361984 image.go:95] image gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d is of wrong architecture
I0819 21:16:23.915546 1361984 cache.go:149] Downloading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d to local cache
I0819 21:16:23.915620 1361984 image.go:63] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local cache directory
I0819 21:16:23.915643 1361984 image.go:66] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d in local cache directory, skipping pull
I0819 21:16:23.915652 1361984 image.go:135] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d exists in cache, skipping pull
I0819 21:16:23.915660 1361984 cache.go:152] successfully saved gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d as a tarball
I0819 21:16:23.915669 1361984 cache.go:162] Loading gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d from local cache
I0819 21:16:24.045426 1361984 cache.go:164] successfully loaded and using gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d from cached tarball
I0819 21:16:24.045473 1361984 cache.go:194] Successfully downloaded all kic artifacts
I0819 21:16:24.045519 1361984 start.go:360] acquireMachinesLock for embed-certs-249735: {Name:mk380c76ca7ca6e03f15ca8a66f61e37f7b1c31d Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0819 21:16:24.046075 1361984 start.go:364] duration metric: took 529.916µs to acquireMachinesLock for "embed-certs-249735"
I0819 21:16:24.046117 1361984 start.go:93] Provisioning new machine with config: &{Name:embed-certs-249735 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:embed-certs-249735 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableM
etrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}
I0819 21:16:24.046217 1361984 start.go:125] createHost starting for "" (driver="docker")
I0819 21:16:22.175433 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:24.181379 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:26.675944 1351451 pod_ready.go:103] pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace has status "Ready":"False"
I0819 21:16:26.675980 1351451 pod_ready.go:82] duration metric: took 4m0.007787366s for pod "metrics-server-9975d5f86-4glzw" in "kube-system" namespace to be "Ready" ...
E0819 21:16:26.675991 1351451 pod_ready.go:67] WaitExtra: waitPodCondition: context deadline exceeded
I0819 21:16:26.676030 1351451 pod_ready.go:39] duration metric: took 5m28.928635211s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0819 21:16:26.676055 1351451 api_server.go:52] waiting for apiserver process to appear ...
I0819 21:16:26.676101 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0819 21:16:26.676201 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0819 21:16:26.722017 1351451 cri.go:89] found id: "863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:26.722043 1351451 cri.go:89] found id: "ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:26.722049 1351451 cri.go:89] found id: ""
I0819 21:16:26.722056 1351451 logs.go:276] 2 containers: [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce]
I0819 21:16:26.722117 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.725648 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.729023 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0819 21:16:26.729144 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0819 21:16:26.770425 1351451 cri.go:89] found id: "fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:26.770446 1351451 cri.go:89] found id: "de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:26.770451 1351451 cri.go:89] found id: ""
I0819 21:16:26.770458 1351451 logs.go:276] 2 containers: [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5]
I0819 21:16:26.770516 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.774330 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.777988 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0819 21:16:26.778056 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0819 21:16:24.049412 1361984 out.go:235] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0819 21:16:24.049684 1361984 start.go:159] libmachine.API.Create for "embed-certs-249735" (driver="docker")
I0819 21:16:24.049723 1361984 client.go:168] LocalClient.Create starting
I0819 21:16:24.049797 1361984 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem
I0819 21:16:24.049841 1361984 main.go:141] libmachine: Decoding PEM data...
I0819 21:16:24.049862 1361984 main.go:141] libmachine: Parsing certificate...
I0819 21:16:24.049929 1361984 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem
I0819 21:16:24.049952 1361984 main.go:141] libmachine: Decoding PEM data...
I0819 21:16:24.049962 1361984 main.go:141] libmachine: Parsing certificate...
I0819 21:16:24.050369 1361984 cli_runner.go:164] Run: docker network inspect embed-certs-249735 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0819 21:16:24.067497 1361984 cli_runner.go:211] docker network inspect embed-certs-249735 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0819 21:16:24.067610 1361984 network_create.go:284] running [docker network inspect embed-certs-249735] to gather additional debugging logs...
I0819 21:16:24.067632 1361984 cli_runner.go:164] Run: docker network inspect embed-certs-249735
W0819 21:16:24.085046 1361984 cli_runner.go:211] docker network inspect embed-certs-249735 returned with exit code 1
I0819 21:16:24.085083 1361984 network_create.go:287] error running [docker network inspect embed-certs-249735]: docker network inspect embed-certs-249735: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-249735 not found
I0819 21:16:24.085099 1361984 network_create.go:289] output of [docker network inspect embed-certs-249735]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-249735 not found
** /stderr **
I0819 21:16:24.085224 1361984 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0819 21:16:24.102075 1361984 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-459ee00191af IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:04:b1:ec:76} reservation:<nil>}
I0819 21:16:24.102464 1361984 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-c806d94ebfcc IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:63:2d:99:b8} reservation:<nil>}
I0819 21:16:24.102799 1361984 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-510cc45fc900 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:7d:71:58:69} reservation:<nil>}
I0819 21:16:24.103287 1361984 network.go:206] using free private subnet 192.168.76.0/24: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x4001876470}
I0819 21:16:24.103314 1361984 network_create.go:124] attempt to create docker network embed-certs-249735 192.168.76.0/24 with gateway 192.168.76.1 and MTU of 1500 ...
I0819 21:16:24.103379 1361984 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.76.0/24 --gateway=192.168.76.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-249735 embed-certs-249735
I0819 21:16:24.178012 1361984 network_create.go:108] docker network embed-certs-249735 192.168.76.0/24 created
I0819 21:16:24.178046 1361984 kic.go:121] calculated static IP "192.168.76.2" for the "embed-certs-249735" container
I0819 21:16:24.178174 1361984 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0819 21:16:24.206593 1361984 cli_runner.go:164] Run: docker volume create embed-certs-249735 --label name.minikube.sigs.k8s.io=embed-certs-249735 --label created_by.minikube.sigs.k8s.io=true
I0819 21:16:24.230425 1361984 oci.go:103] Successfully created a docker volume embed-certs-249735
I0819 21:16:24.230506 1361984 cli_runner.go:164] Run: docker run --rm --name embed-certs-249735-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-249735 --entrypoint /usr/bin/test -v embed-certs-249735:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d -d /var/lib
I0819 21:16:24.927017 1361984 oci.go:107] Successfully prepared a docker volume embed-certs-249735
I0819 21:16:24.927060 1361984 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime containerd
I0819 21:16:24.927081 1361984 kic.go:194] Starting extracting preloaded images to volume ...
I0819 21:16:24.927174 1361984 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.0-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-249735:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d -I lz4 -xf /preloaded.tar -C /extractDir
I0819 21:16:26.831135 1351451 cri.go:89] found id: "1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:26.831205 1351451 cri.go:89] found id: "f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:26.831224 1351451 cri.go:89] found id: ""
I0819 21:16:26.831251 1351451 logs.go:276] 2 containers: [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657]
I0819 21:16:26.831331 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.835121 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.838706 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0819 21:16:26.838823 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0819 21:16:26.885971 1351451 cri.go:89] found id: "c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:26.886020 1351451 cri.go:89] found id: "62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:26.886043 1351451 cri.go:89] found id: ""
I0819 21:16:26.886073 1351451 logs.go:276] 2 containers: [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a]
I0819 21:16:26.886149 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.890432 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.894051 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0819 21:16:26.894167 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0819 21:16:26.940512 1351451 cri.go:89] found id: "3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:26.940585 1351451 cri.go:89] found id: "63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:26.940605 1351451 cri.go:89] found id: ""
I0819 21:16:26.940633 1351451 logs.go:276] 2 containers: [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e]
I0819 21:16:26.940708 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.944532 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:26.947982 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0819 21:16:26.948103 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0819 21:16:27.064038 1351451 cri.go:89] found id: "a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:27.064065 1351451 cri.go:89] found id: "7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:27.064071 1351451 cri.go:89] found id: ""
I0819 21:16:27.064079 1351451 logs.go:276] 2 containers: [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525]
I0819 21:16:27.064160 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.075192 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.084336 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0819 21:16:27.084446 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0819 21:16:27.209506 1351451 cri.go:89] found id: "2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:27.209543 1351451 cri.go:89] found id: "aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:27.209548 1351451 cri.go:89] found id: ""
I0819 21:16:27.209556 1351451 logs.go:276] 2 containers: [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e]
I0819 21:16:27.209653 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.214678 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.218252 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0819 21:16:27.218352 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0819 21:16:27.270015 1351451 cri.go:89] found id: "18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:27.270059 1351451 cri.go:89] found id: ""
I0819 21:16:27.270067 1351451 logs.go:276] 1 containers: [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1]
I0819 21:16:27.270165 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.274425 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0819 21:16:27.274551 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0819 21:16:27.324064 1351451 cri.go:89] found id: "3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:27.324100 1351451 cri.go:89] found id: "556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:27.324105 1351451 cri.go:89] found id: ""
I0819 21:16:27.324112 1351451 logs.go:276] 2 containers: [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd]
I0819 21:16:27.324194 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.351328 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:27.355700 1351451 logs.go:123] Gathering logs for coredns [f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657] ...
I0819 21:16:27.355736 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:27.406666 1351451 logs.go:123] Gathering logs for kube-scheduler [62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a] ...
I0819 21:16:27.406696 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:27.461106 1351451 logs.go:123] Gathering logs for kube-proxy [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a] ...
I0819 21:16:27.461139 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:27.513081 1351451 logs.go:123] Gathering logs for storage-provisioner [556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd] ...
I0819 21:16:27.513107 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:27.570112 1351451 logs.go:123] Gathering logs for kubelet ...
I0819 21:16:27.570155 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0819 21:16:27.636753 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.621751 665 reflector.go:138] object-"kube-system"/"kindnet-token-xptkw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xptkw" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637011 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622026 665 reflector.go:138] object-"kube-system"/"coredns-token-jj86v": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-jj86v" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637232 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622171 665 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637539 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622316 665 reflector.go:138] object-"kube-system"/"storage-provisioner-token-ssctg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-ssctg" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637752 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622453 665 reflector.go:138] object-"default"/"default-token-vbtl7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-vbtl7" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.637956 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671400 665 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.638181 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671660 665 reflector.go:138] object-"kube-system"/"metrics-server-token-x764s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-x764s" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:27.646452 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.574230 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.646679 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.789676 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.650451 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:15 old-k8s-version-127648 kubelet[665]: E0819 21:11:15.561064 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.652683 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:22 old-k8s-version-127648 kubelet[665]: E0819 21:11:22.882357 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.653041 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:23 old-k8s-version-127648 kubelet[665]: E0819 21:11:23.880032 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.653248 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:27 old-k8s-version-127648 kubelet[665]: E0819 21:11:27.549335 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.654048 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:30 old-k8s-version-127648 kubelet[665]: E0819 21:11:30.903604 665 pod_workers.go:191] Error syncing pod 74e4d116-4e4e-4dc5-af07-3013282e840a ("storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"
W0819 21:16:27.654403 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:31 old-k8s-version-127648 kubelet[665]: E0819 21:11:31.736669 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.657331 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:41 old-k8s-version-127648 kubelet[665]: E0819 21:11:41.559000 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.658003 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:45 old-k8s-version-127648 kubelet[665]: E0819 21:11:45.947294 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.658476 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:51 old-k8s-version-127648 kubelet[665]: E0819 21:11:51.747256 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.658664 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:54 old-k8s-version-127648 kubelet[665]: E0819 21:11:54.549415 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.658852 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:06 old-k8s-version-127648 kubelet[665]: E0819 21:12:06.549632 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.659552 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:08 old-k8s-version-127648 kubelet[665]: E0819 21:12:08.101007 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.659888 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:11 old-k8s-version-127648 kubelet[665]: E0819 21:12:11.736495 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.660109 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:18 old-k8s-version-127648 kubelet[665]: E0819 21:12:18.550570 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.660468 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:22 old-k8s-version-127648 kubelet[665]: E0819 21:12:22.548984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.663052 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:29 old-k8s-version-127648 kubelet[665]: E0819 21:12:29.557879 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.663411 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:35 old-k8s-version-127648 kubelet[665]: E0819 21:12:35.549004 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.663625 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:41 old-k8s-version-127648 kubelet[665]: E0819 21:12:41.549476 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.663977 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:47 old-k8s-version-127648 kubelet[665]: E0819 21:12:47.548954 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.664192 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:56 old-k8s-version-127648 kubelet[665]: E0819 21:12:56.550012 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.664817 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:59 old-k8s-version-127648 kubelet[665]: E0819 21:12:59.243984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.665194 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:01 old-k8s-version-127648 kubelet[665]: E0819 21:13:01.736702 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.665402 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:08 old-k8s-version-127648 kubelet[665]: E0819 21:13:08.550071 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.665753 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:15 old-k8s-version-127648 kubelet[665]: E0819 21:13:15.549079 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.666078 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:21 old-k8s-version-127648 kubelet[665]: E0819 21:13:21.549597 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.666481 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:27 old-k8s-version-127648 kubelet[665]: E0819 21:13:27.549537 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.666691 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:36 old-k8s-version-127648 kubelet[665]: E0819 21:13:36.549526 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.667050 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:39 old-k8s-version-127648 kubelet[665]: E0819 21:13:39.549198 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.669612 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:50 old-k8s-version-127648 kubelet[665]: E0819 21:13:50.557660 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:27.670005 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:52 old-k8s-version-127648 kubelet[665]: E0819 21:13:52.549116 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.670216 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:01 old-k8s-version-127648 kubelet[665]: E0819 21:14:01.549487 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.670581 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:06 old-k8s-version-127648 kubelet[665]: E0819 21:14:06.548964 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.670789 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:16 old-k8s-version-127648 kubelet[665]: E0819 21:14:16.553951 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.671457 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:20 old-k8s-version-127648 kubelet[665]: E0819 21:14:20.469865 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.671816 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:21 old-k8s-version-127648 kubelet[665]: E0819 21:14:21.736453 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.672060 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:31 old-k8s-version-127648 kubelet[665]: E0819 21:14:31.549363 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.672464 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:36 old-k8s-version-127648 kubelet[665]: E0819 21:14:36.549968 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.672679 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:46 old-k8s-version-127648 kubelet[665]: E0819 21:14:46.549614 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.673041 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:48 old-k8s-version-127648 kubelet[665]: E0819 21:14:48.551815 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.673268 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:57 old-k8s-version-127648 kubelet[665]: E0819 21:14:57.549314 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.673723 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:03 old-k8s-version-127648 kubelet[665]: E0819 21:15:03.548948 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.673940 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:08 old-k8s-version-127648 kubelet[665]: E0819 21:15:08.550121 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.674342 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: E0819 21:15:18.549558 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.674597 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:19 old-k8s-version-127648 kubelet[665]: E0819 21:15:19.549582 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675013 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: E0819 21:15:31.549167 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.675241 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:34 old-k8s-version-127648 kubelet[665]: E0819 21:15:34.549642 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675558 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.549479 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.675963 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.676343 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:27.676553 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.676780 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:27.677133 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
I0819 21:16:27.677148 1351451 logs.go:123] Gathering logs for kube-apiserver [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e] ...
I0819 21:16:27.677163 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:27.783187 1351451 logs.go:123] Gathering logs for kube-apiserver [ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce] ...
I0819 21:16:27.783234 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:27.878025 1351451 logs.go:123] Gathering logs for etcd [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267] ...
I0819 21:16:27.878059 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:27.971482 1351451 logs.go:123] Gathering logs for kubernetes-dashboard [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1] ...
I0819 21:16:27.971509 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:28.026406 1351451 logs.go:123] Gathering logs for container status ...
I0819 21:16:28.026441 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0819 21:16:28.077604 1351451 logs.go:123] Gathering logs for dmesg ...
I0819 21:16:28.077635 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0819 21:16:28.106351 1351451 logs.go:123] Gathering logs for describe nodes ...
I0819 21:16:28.106383 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0819 21:16:28.314793 1351451 logs.go:123] Gathering logs for etcd [de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5] ...
I0819 21:16:28.314818 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:28.388732 1351451 logs.go:123] Gathering logs for kindnet [aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e] ...
I0819 21:16:28.388809 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:28.461767 1351451 logs.go:123] Gathering logs for kube-controller-manager [7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525] ...
I0819 21:16:28.461855 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:28.551587 1351451 logs.go:123] Gathering logs for kindnet [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20] ...
I0819 21:16:28.551688 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:28.653556 1351451 logs.go:123] Gathering logs for storage-provisioner [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e] ...
I0819 21:16:28.653593 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:28.695383 1351451 logs.go:123] Gathering logs for containerd ...
I0819 21:16:28.695411 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0819 21:16:28.762252 1351451 logs.go:123] Gathering logs for coredns [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e] ...
I0819 21:16:28.762289 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:28.820470 1351451 logs.go:123] Gathering logs for kube-scheduler [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4] ...
I0819 21:16:28.820516 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:28.873663 1351451 logs.go:123] Gathering logs for kube-proxy [63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e] ...
I0819 21:16:28.873701 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:28.917142 1351451 logs.go:123] Gathering logs for kube-controller-manager [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363] ...
I0819 21:16:28.917183 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:28.979164 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:28.979196 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0819 21:16:28.979271 1351451 out.go:270] X Problems detected in kubelet:
W0819 21:16:28.979288 1351451 out.go:270] Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:28.979296 1351451 out.go:270] Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:28.979318 1351451 out.go:270] Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:28.979328 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:28.979333 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
I0819 21:16:28.979346 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:28.979354 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:29.925889 1361984 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/19423-1139612/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.31.0-containerd-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-249735:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d -I lz4 -xf /preloaded.tar -C /extractDir: (4.998676077s)
I0819 21:16:29.925922 1361984 kic.go:203] duration metric: took 4.998834842s to extract preloaded images to volume ...
W0819 21:16:29.926081 1361984 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0819 21:16:29.926238 1361984 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0819 21:16:29.978802 1361984 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-249735 --name embed-certs-249735 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-249735 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-249735 --network embed-certs-249735 --ip 192.168.76.2 --volume embed-certs-249735:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d
I0819 21:16:30.412373 1361984 cli_runner.go:164] Run: docker container inspect embed-certs-249735 --format={{.State.Running}}
I0819 21:16:30.434243 1361984 cli_runner.go:164] Run: docker container inspect embed-certs-249735 --format={{.State.Status}}
I0819 21:16:30.454077 1361984 cli_runner.go:164] Run: docker exec embed-certs-249735 stat /var/lib/dpkg/alternatives/iptables
I0819 21:16:30.537668 1361984 oci.go:144] the created container "embed-certs-249735" has a running status.
I0819 21:16:30.537700 1361984 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa...
I0819 21:16:31.393136 1361984 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0819 21:16:31.416814 1361984 cli_runner.go:164] Run: docker container inspect embed-certs-249735 --format={{.State.Status}}
I0819 21:16:31.442058 1361984 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0819 21:16:31.442077 1361984 kic_runner.go:114] Args: [docker exec --privileged embed-certs-249735 chown docker:docker /home/docker/.ssh/authorized_keys]
I0819 21:16:31.510492 1361984 cli_runner.go:164] Run: docker container inspect embed-certs-249735 --format={{.State.Status}}
I0819 21:16:31.530279 1361984 machine.go:93] provisionDockerMachine start ...
I0819 21:16:31.530366 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:31.551910 1361984 main.go:141] libmachine: Using SSH client type: native
I0819 21:16:31.552189 1361984 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34233 <nil> <nil>}
I0819 21:16:31.552198 1361984 main.go:141] libmachine: About to run SSH command:
hostname
I0819 21:16:31.692545 1361984 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-249735
I0819 21:16:31.692632 1361984 ubuntu.go:169] provisioning hostname "embed-certs-249735"
I0819 21:16:31.692730 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:31.714398 1361984 main.go:141] libmachine: Using SSH client type: native
I0819 21:16:31.714645 1361984 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34233 <nil> <nil>}
I0819 21:16:31.714656 1361984 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-249735 && echo "embed-certs-249735" | sudo tee /etc/hostname
I0819 21:16:31.864916 1361984 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-249735
I0819 21:16:31.865067 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:31.888316 1361984 main.go:141] libmachine: Using SSH client type: native
I0819 21:16:31.888576 1361984 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e49d0] 0x3e7230 <nil> [] 0s} 127.0.0.1 34233 <nil> <nil>}
I0819 21:16:31.888599 1361984 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-249735' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-249735/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-249735' | sudo tee -a /etc/hosts;
fi
fi
I0819 21:16:32.031542 1361984 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0819 21:16:32.031603 1361984 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/19423-1139612/.minikube CaCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/19423-1139612/.minikube}
I0819 21:16:32.031642 1361984 ubuntu.go:177] setting up certificates
I0819 21:16:32.031655 1361984 provision.go:84] configureAuth start
I0819 21:16:32.031736 1361984 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-249735
I0819 21:16:32.051745 1361984 provision.go:143] copyHostCerts
I0819 21:16:32.051818 1361984 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem, removing ...
I0819 21:16:32.051835 1361984 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem
I0819 21:16:32.051915 1361984 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.pem (1078 bytes)
I0819 21:16:32.052031 1361984 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem, removing ...
I0819 21:16:32.052042 1361984 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem
I0819 21:16:32.052075 1361984 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/cert.pem (1123 bytes)
I0819 21:16:32.052173 1361984 exec_runner.go:144] found /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem, removing ...
I0819 21:16:32.052183 1361984 exec_runner.go:203] rm: /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem
I0819 21:16:32.052210 1361984 exec_runner.go:151] cp: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/19423-1139612/.minikube/key.pem (1675 bytes)
I0819 21:16:32.052323 1361984 provision.go:117] generating server cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem org=jenkins.embed-certs-249735 san=[127.0.0.1 192.168.76.2 embed-certs-249735 localhost minikube]
I0819 21:16:32.312491 1361984 provision.go:177] copyRemoteCerts
I0819 21:16:32.312566 1361984 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0819 21:16:32.312616 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:32.339108 1361984 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34233 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa Username:docker}
I0819 21:16:32.437580 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1078 bytes)
I0819 21:16:32.463439 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0819 21:16:32.489345 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0819 21:16:32.516829 1361984 provision.go:87] duration metric: took 485.153146ms to configureAuth
I0819 21:16:32.516862 1361984 ubuntu.go:193] setting minikube options for container-runtime
I0819 21:16:32.517053 1361984 config.go:182] Loaded profile config "embed-certs-249735": Driver=docker, ContainerRuntime=containerd, KubernetesVersion=v1.31.0
I0819 21:16:32.517069 1361984 machine.go:96] duration metric: took 986.773095ms to provisionDockerMachine
I0819 21:16:32.517076 1361984 client.go:171] duration metric: took 8.467343131s to LocalClient.Create
I0819 21:16:32.517096 1361984 start.go:167] duration metric: took 8.467414702s to libmachine.API.Create "embed-certs-249735"
I0819 21:16:32.517126 1361984 start.go:293] postStartSetup for "embed-certs-249735" (driver="docker")
I0819 21:16:32.517141 1361984 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0819 21:16:32.517200 1361984 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0819 21:16:32.517250 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:32.535123 1361984 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34233 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa Username:docker}
I0819 21:16:32.633953 1361984 ssh_runner.go:195] Run: cat /etc/os-release
I0819 21:16:32.637222 1361984 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0819 21:16:32.637259 1361984 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0819 21:16:32.637273 1361984 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0819 21:16:32.637279 1361984 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0819 21:16:32.637294 1361984 filesync.go:126] Scanning /home/jenkins/minikube-integration/19423-1139612/.minikube/addons for local assets ...
I0819 21:16:32.637353 1361984 filesync.go:126] Scanning /home/jenkins/minikube-integration/19423-1139612/.minikube/files for local assets ...
I0819 21:16:32.637434 1361984 filesync.go:149] local asset: /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem -> 11450182.pem in /etc/ssl/certs
I0819 21:16:32.637540 1361984 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0819 21:16:32.646017 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem --> /etc/ssl/certs/11450182.pem (1708 bytes)
I0819 21:16:32.670914 1361984 start.go:296] duration metric: took 153.768569ms for postStartSetup
I0819 21:16:32.671346 1361984 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-249735
I0819 21:16:32.688807 1361984 profile.go:143] Saving config to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/config.json ...
I0819 21:16:32.689103 1361984 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0819 21:16:32.689158 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:32.706258 1361984 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34233 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa Username:docker}
I0819 21:16:32.797747 1361984 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0819 21:16:32.802724 1361984 start.go:128] duration metric: took 8.756490797s to createHost
I0819 21:16:32.802748 1361984 start.go:83] releasing machines lock for "embed-certs-249735", held for 8.756654157s
I0819 21:16:32.802837 1361984 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-249735
I0819 21:16:32.828533 1361984 ssh_runner.go:195] Run: cat /version.json
I0819 21:16:32.828671 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:32.829048 1361984 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0819 21:16:32.829198 1361984 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-249735
I0819 21:16:32.860188 1361984 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34233 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa Username:docker}
I0819 21:16:32.873332 1361984 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:34233 SSHKeyPath:/home/jenkins/minikube-integration/19423-1139612/.minikube/machines/embed-certs-249735/id_rsa Username:docker}
I0819 21:16:33.086123 1361984 ssh_runner.go:195] Run: systemctl --version
I0819 21:16:33.091002 1361984 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0819 21:16:33.095690 1361984 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0819 21:16:33.121618 1361984 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0819 21:16:33.121741 1361984 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%p, " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0819 21:16:33.151086 1361984 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0819 21:16:33.151110 1361984 start.go:495] detecting cgroup driver to use...
I0819 21:16:33.151164 1361984 detect.go:187] detected "cgroupfs" cgroup driver on host os
I0819 21:16:33.151240 1361984 ssh_runner.go:195] Run: sudo systemctl stop -f crio
I0819 21:16:33.164664 1361984 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0819 21:16:33.176309 1361984 docker.go:217] disabling cri-docker service (if available) ...
I0819 21:16:33.176418 1361984 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.socket
I0819 21:16:33.191433 1361984 ssh_runner.go:195] Run: sudo systemctl stop -f cri-docker.service
I0819 21:16:33.205817 1361984 ssh_runner.go:195] Run: sudo systemctl disable cri-docker.socket
I0819 21:16:33.321871 1361984 ssh_runner.go:195] Run: sudo systemctl mask cri-docker.service
I0819 21:16:33.427827 1361984 docker.go:233] disabling docker service ...
I0819 21:16:33.427951 1361984 ssh_runner.go:195] Run: sudo systemctl stop -f docker.socket
I0819 21:16:33.450591 1361984 ssh_runner.go:195] Run: sudo systemctl stop -f docker.service
I0819 21:16:33.464361 1361984 ssh_runner.go:195] Run: sudo systemctl disable docker.socket
I0819 21:16:33.560103 1361984 ssh_runner.go:195] Run: sudo systemctl mask docker.service
I0819 21:16:33.661097 1361984 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service docker
I0819 21:16:33.673253 1361984 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0819 21:16:33.691821 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.10"|' /etc/containerd/config.toml"
I0819 21:16:33.703033 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0819 21:16:33.714582 1361984 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0819 21:16:33.714707 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0819 21:16:33.731723 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0819 21:16:33.743041 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0819 21:16:33.753762 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0819 21:16:33.765315 1361984 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0819 21:16:33.775088 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0819 21:16:33.786582 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0819 21:16:33.797912 1361984 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0819 21:16:33.808931 1361984 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0819 21:16:33.817923 1361984 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0819 21:16:33.827796 1361984 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0819 21:16:33.921651 1361984 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0819 21:16:34.082490 1361984 start.go:542] Will wait 60s for socket path /run/containerd/containerd.sock
I0819 21:16:34.082594 1361984 ssh_runner.go:195] Run: stat /run/containerd/containerd.sock
I0819 21:16:34.086454 1361984 start.go:563] Will wait 60s for crictl version
I0819 21:16:34.086540 1361984 ssh_runner.go:195] Run: which crictl
I0819 21:16:34.089949 1361984 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0819 21:16:34.128968 1361984 start.go:579] Version: 0.1.0
RuntimeName: containerd
RuntimeVersion: 1.7.20
RuntimeApiVersion: v1
I0819 21:16:34.129067 1361984 ssh_runner.go:195] Run: containerd --version
I0819 21:16:34.151719 1361984 ssh_runner.go:195] Run: containerd --version
I0819 21:16:34.184490 1361984 out.go:177] * Preparing Kubernetes v1.31.0 on containerd 1.7.20 ...
I0819 21:16:34.185560 1361984 cli_runner.go:164] Run: docker network inspect embed-certs-249735 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0819 21:16:34.203095 1361984 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0819 21:16:34.206776 1361984 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0819 21:16:34.217890 1361984 kubeadm.go:883] updating cluster {Name:embed-certs-249735 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:embed-certs-249735 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:
false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0819 21:16:34.218026 1361984 preload.go:131] Checking if preload exists for k8s version v1.31.0 and runtime containerd
I0819 21:16:34.218094 1361984 ssh_runner.go:195] Run: sudo crictl images --output json
I0819 21:16:34.256017 1361984 containerd.go:627] all images are preloaded for containerd runtime.
I0819 21:16:34.256043 1361984 containerd.go:534] Images already preloaded, skipping extraction
I0819 21:16:34.256104 1361984 ssh_runner.go:195] Run: sudo crictl images --output json
I0819 21:16:34.293259 1361984 containerd.go:627] all images are preloaded for containerd runtime.
I0819 21:16:34.293283 1361984 cache_images.go:84] Images are preloaded, skipping loading
I0819 21:16:34.293291 1361984 kubeadm.go:934] updating node { 192.168.76.2 8443 v1.31.0 containerd true true} ...
I0819 21:16:34.293398 1361984 kubeadm.go:946] kubelet [Unit]
Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.31.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-249735 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.31.0 ClusterName:embed-certs-249735 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0819 21:16:34.293471 1361984 ssh_runner.go:195] Run: sudo crictl info
I0819 21:16:34.342565 1361984 cni.go:84] Creating CNI manager for ""
I0819 21:16:34.342590 1361984 cni.go:143] "docker" driver + "containerd" runtime found, recommending kindnet
I0819 21:16:34.342602 1361984 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0819 21:16:34.342623 1361984 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.31.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-249735 NodeName:embed-certs-249735 DNSDomain:cluster.local CRISocket:/run/containerd/containerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPo
dPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///run/containerd/containerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0819 21:16:34.342762 1361984 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: "embed-certs-249735"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.31.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0819 21:16:34.342833 1361984 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.31.0
I0819 21:16:34.352028 1361984 binaries.go:44] Found k8s binaries, skipping transfer
I0819 21:16:34.352101 1361984 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0819 21:16:34.361143 1361984 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (322 bytes)
I0819 21:16:34.379510 1361984 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0819 21:16:34.398600 1361984 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2172 bytes)
I0819 21:16:34.417492 1361984 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0819 21:16:34.421028 1361984 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0819 21:16:34.431797 1361984 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0819 21:16:34.527480 1361984 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0819 21:16:34.543800 1361984 certs.go:68] Setting up /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735 for IP: 192.168.76.2
I0819 21:16:34.543824 1361984 certs.go:194] generating shared ca certs ...
I0819 21:16:34.543840 1361984 certs.go:226] acquiring lock for ca certs: {Name:mk862c79d80b8fe3a5df83b1592928b3403a862f Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:34.543974 1361984 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.key
I0819 21:16:34.544023 1361984 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.key
I0819 21:16:34.544035 1361984 certs.go:256] generating profile certs ...
I0819 21:16:34.544089 1361984 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.key
I0819 21:16:34.544118 1361984 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.crt with IP's: []
I0819 21:16:35.228563 1361984 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.crt ...
I0819 21:16:35.228598 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.crt: {Name:mk3039998ddd64670b8d8c36ece867d672c92078 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.229385 1361984 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.key ...
I0819 21:16:35.229449 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/client.key: {Name:mk7d6a4f9e86da33a92a3b7b846354f6e85f664d Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.229588 1361984 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key.70e1ab11
I0819 21:16:35.229609 1361984 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt.70e1ab11 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.76.2]
I0819 21:16:35.569294 1361984 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt.70e1ab11 ...
I0819 21:16:35.569330 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt.70e1ab11: {Name:mkf3dfd1b66057abbdcb1ef642727c27c969823c Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.569957 1361984 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key.70e1ab11 ...
I0819 21:16:35.569979 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key.70e1ab11: {Name:mk6d37692220f36176e8d97f324c59aa261cb941 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.570112 1361984 certs.go:381] copying /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt.70e1ab11 -> /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt
I0819 21:16:35.570206 1361984 certs.go:385] copying /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key.70e1ab11 -> /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key
I0819 21:16:35.570269 1361984 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.key
I0819 21:16:35.570288 1361984 crypto.go:68] Generating cert /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.crt with IP's: []
I0819 21:16:35.743315 1361984 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.crt ...
I0819 21:16:35.743349 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.crt: {Name:mka563ffb64459326accfd49ffbebb47afa23d20 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.743999 1361984 crypto.go:164] Writing key to /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.key ...
I0819 21:16:35.744022 1361984 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.key: {Name:mk21bfe409f966bfc694d12b27b2463fb04e8b29 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0819 21:16:35.744818 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018.pem (1338 bytes)
W0819 21:16:35.744898 1361984 certs.go:480] ignoring /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018_empty.pem, impossibly tiny 0 bytes
I0819 21:16:35.744927 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca-key.pem (1679 bytes)
I0819 21:16:35.744991 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/ca.pem (1078 bytes)
I0819 21:16:35.745042 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/cert.pem (1123 bytes)
I0819 21:16:35.745096 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/key.pem (1675 bytes)
I0819 21:16:35.745205 1361984 certs.go:484] found cert: /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem (1708 bytes)
I0819 21:16:35.745968 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0819 21:16:35.774708 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0819 21:16:35.802896 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0819 21:16:35.830615 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1679 bytes)
I0819 21:16:35.859453 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0819 21:16:35.885068 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0819 21:16:35.910238 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0819 21:16:35.935909 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/profiles/embed-certs-249735/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0819 21:16:35.960468 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/files/etc/ssl/certs/11450182.pem --> /usr/share/ca-certificates/11450182.pem (1708 bytes)
I0819 21:16:35.987046 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0819 21:16:36.032647 1361984 ssh_runner.go:362] scp /home/jenkins/minikube-integration/19423-1139612/.minikube/certs/1145018.pem --> /usr/share/ca-certificates/1145018.pem (1338 bytes)
I0819 21:16:36.062154 1361984 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0819 21:16:36.084930 1361984 ssh_runner.go:195] Run: openssl version
I0819 21:16:36.091304 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/11450182.pem && ln -fs /usr/share/ca-certificates/11450182.pem /etc/ssl/certs/11450182.pem"
I0819 21:16:36.101460 1361984 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/11450182.pem
I0819 21:16:36.105496 1361984 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 Aug 19 20:31 /usr/share/ca-certificates/11450182.pem
I0819 21:16:36.105564 1361984 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/11450182.pem
I0819 21:16:36.113117 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/11450182.pem /etc/ssl/certs/3ec20f2e.0"
I0819 21:16:36.122712 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0819 21:16:36.132093 1361984 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0819 21:16:36.135726 1361984 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 Aug 19 20:21 /usr/share/ca-certificates/minikubeCA.pem
I0819 21:16:36.135798 1361984 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0819 21:16:36.143270 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0819 21:16:36.152684 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/1145018.pem && ln -fs /usr/share/ca-certificates/1145018.pem /etc/ssl/certs/1145018.pem"
I0819 21:16:36.161883 1361984 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/1145018.pem
I0819 21:16:36.165733 1361984 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 Aug 19 20:31 /usr/share/ca-certificates/1145018.pem
I0819 21:16:36.165801 1361984 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/1145018.pem
I0819 21:16:36.172715 1361984 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/1145018.pem /etc/ssl/certs/51391683.0"
I0819 21:16:36.181903 1361984 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0819 21:16:36.185775 1361984 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0819 21:16:36.185874 1361984 kubeadm.go:392] StartCluster: {Name:embed-certs-249735 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1723740748-19452@sha256:2211a6931895d2d502e957e9667096db10734a96767d670cb4dbffdd37397b0d Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.31.0 ClusterName:embed-certs-249735 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:containerd CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.31.0 ContainerRuntime:containerd ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:fal
se CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0819 21:16:36.185957 1361984 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:paused Name: Namespaces:[kube-system]}
I0819 21:16:36.186011 1361984 ssh_runner.go:195] Run: sudo -s eval "crictl ps -a --quiet --label io.kubernetes.pod.namespace=kube-system"
I0819 21:16:36.230146 1361984 cri.go:89] found id: ""
I0819 21:16:36.230289 1361984 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0819 21:16:36.239386 1361984 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0819 21:16:36.248681 1361984 kubeadm.go:214] ignoring SystemVerification for kubeadm because of docker driver
I0819 21:16:36.248745 1361984 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0819 21:16:36.259827 1361984 kubeadm.go:155] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0819 21:16:36.259849 1361984 kubeadm.go:157] found existing configuration files:
I0819 21:16:36.259908 1361984 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0819 21:16:36.268962 1361984 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0819 21:16:36.269033 1361984 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0819 21:16:36.277134 1361984 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0819 21:16:36.286066 1361984 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0819 21:16:36.286170 1361984 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0819 21:16:36.295013 1361984 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0819 21:16:36.304121 1361984 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0819 21:16:36.304209 1361984 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0819 21:16:36.313415 1361984 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0819 21:16:36.322583 1361984 kubeadm.go:163] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0819 21:16:36.322647 1361984 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0819 21:16:36.333751 1361984 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.31.0:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0819 21:16:36.378906 1361984 kubeadm.go:310] [init] Using Kubernetes version: v1.31.0
I0819 21:16:36.379117 1361984 kubeadm.go:310] [preflight] Running pre-flight checks
I0819 21:16:36.398061 1361984 kubeadm.go:310] [preflight] The system verification failed. Printing the output from the verification:
I0819 21:16:36.398134 1361984 kubeadm.go:310] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1067-aws[0m
I0819 21:16:36.398175 1361984 kubeadm.go:310] [0;37mOS[0m: [0;32mLinux[0m
I0819 21:16:36.398224 1361984 kubeadm.go:310] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0819 21:16:36.398292 1361984 kubeadm.go:310] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0819 21:16:36.398341 1361984 kubeadm.go:310] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0819 21:16:36.398391 1361984 kubeadm.go:310] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0819 21:16:36.398446 1361984 kubeadm.go:310] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0819 21:16:36.398500 1361984 kubeadm.go:310] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0819 21:16:36.398547 1361984 kubeadm.go:310] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0819 21:16:36.398596 1361984 kubeadm.go:310] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0819 21:16:36.398646 1361984 kubeadm.go:310] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0819 21:16:36.457112 1361984 kubeadm.go:310] [preflight] Pulling images required for setting up a Kubernetes cluster
I0819 21:16:36.457222 1361984 kubeadm.go:310] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0819 21:16:36.457314 1361984 kubeadm.go:310] [preflight] You can also perform this action beforehand using 'kubeadm config images pull'
I0819 21:16:36.464754 1361984 kubeadm.go:310] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0819 21:16:36.469307 1361984 out.go:235] - Generating certificates and keys ...
I0819 21:16:36.469499 1361984 kubeadm.go:310] [certs] Using existing ca certificate authority
I0819 21:16:36.469623 1361984 kubeadm.go:310] [certs] Using existing apiserver certificate and key on disk
I0819 21:16:37.295435 1361984 kubeadm.go:310] [certs] Generating "apiserver-kubelet-client" certificate and key
I0819 21:16:37.844864 1361984 kubeadm.go:310] [certs] Generating "front-proxy-ca" certificate and key
I0819 21:16:38.101956 1361984 kubeadm.go:310] [certs] Generating "front-proxy-client" certificate and key
I0819 21:16:38.980698 1351451 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0819 21:16:39.000948 1351451 api_server.go:72] duration metric: took 5m58.287822906s to wait for apiserver process to appear ...
I0819 21:16:39.000980 1351451 api_server.go:88] waiting for apiserver healthz status ...
I0819 21:16:39.001038 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-apiserver Namespaces:[]}
I0819 21:16:39.001135 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-apiserver
I0819 21:16:39.096959 1351451 cri.go:89] found id: "863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:39.096986 1351451 cri.go:89] found id: "ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:39.096998 1351451 cri.go:89] found id: ""
I0819 21:16:39.097007 1351451 logs.go:276] 2 containers: [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce]
I0819 21:16:39.097082 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.103146 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.108342 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:etcd Namespaces:[]}
I0819 21:16:39.108415 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=etcd
I0819 21:16:39.181901 1351451 cri.go:89] found id: "fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:39.181922 1351451 cri.go:89] found id: "de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:39.181931 1351451 cri.go:89] found id: ""
I0819 21:16:39.181942 1351451 logs.go:276] 2 containers: [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5]
I0819 21:16:39.182019 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.187650 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.195541 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:coredns Namespaces:[]}
I0819 21:16:39.195721 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=coredns
I0819 21:16:39.279675 1351451 cri.go:89] found id: "1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:39.279764 1351451 cri.go:89] found id: "f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:39.279788 1351451 cri.go:89] found id: ""
I0819 21:16:39.279817 1351451 logs.go:276] 2 containers: [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657]
I0819 21:16:39.280034 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.285431 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.290629 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-scheduler Namespaces:[]}
I0819 21:16:39.290859 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-scheduler
I0819 21:16:39.371561 1351451 cri.go:89] found id: "c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:39.371650 1351451 cri.go:89] found id: "62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:39.371676 1351451 cri.go:89] found id: ""
I0819 21:16:39.371728 1351451 logs.go:276] 2 containers: [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a]
I0819 21:16:39.371851 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.376912 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.383459 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-proxy Namespaces:[]}
I0819 21:16:39.383627 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-proxy
I0819 21:16:39.448903 1351451 cri.go:89] found id: "3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:39.448982 1351451 cri.go:89] found id: "63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:39.449012 1351451 cri.go:89] found id: ""
I0819 21:16:39.449034 1351451 logs.go:276] 2 containers: [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e]
I0819 21:16:39.449149 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.454509 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.459726 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kube-controller-manager Namespaces:[]}
I0819 21:16:39.459909 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kube-controller-manager
I0819 21:16:39.545859 1351451 cri.go:89] found id: "a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:39.545936 1351451 cri.go:89] found id: "7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:39.545961 1351451 cri.go:89] found id: ""
I0819 21:16:39.545981 1351451 logs.go:276] 2 containers: [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525]
I0819 21:16:39.546083 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.561290 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.565853 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kindnet Namespaces:[]}
I0819 21:16:39.566023 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kindnet
I0819 21:16:39.630565 1351451 cri.go:89] found id: "2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:39.630648 1351451 cri.go:89] found id: "aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:39.630667 1351451 cri.go:89] found id: ""
I0819 21:16:39.630691 1351451 logs.go:276] 2 containers: [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e]
I0819 21:16:39.630799 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.636088 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.640406 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:kubernetes-dashboard Namespaces:[]}
I0819 21:16:39.640539 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=kubernetes-dashboard
I0819 21:16:39.705002 1351451 cri.go:89] found id: "18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:39.705082 1351451 cri.go:89] found id: ""
I0819 21:16:39.705109 1351451 logs.go:276] 1 containers: [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1]
I0819 21:16:39.705209 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.713612 1351451 cri.go:54] listing CRI containers in root /run/containerd/runc/k8s.io: {State:all Name:storage-provisioner Namespaces:[]}
I0819 21:16:39.713752 1351451 ssh_runner.go:195] Run: sudo crictl ps -a --quiet --name=storage-provisioner
I0819 21:16:39.771883 1351451 cri.go:89] found id: "3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:39.771963 1351451 cri.go:89] found id: "556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:39.771982 1351451 cri.go:89] found id: ""
I0819 21:16:39.772007 1351451 logs.go:276] 2 containers: [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd]
I0819 21:16:39.772133 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.777471 1351451 ssh_runner.go:195] Run: which crictl
I0819 21:16:39.782219 1351451 logs.go:123] Gathering logs for dmesg ...
I0819 21:16:39.782306 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0819 21:16:39.803506 1351451 logs.go:123] Gathering logs for kube-apiserver [ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce] ...
I0819 21:16:39.803589 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce"
I0819 21:16:39.902538 1351451 logs.go:123] Gathering logs for kube-controller-manager [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363] ...
I0819 21:16:39.902629 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363"
I0819 21:16:39.988902 1351451 logs.go:123] Gathering logs for storage-provisioner [556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd] ...
I0819 21:16:39.988936 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd"
I0819 21:16:40.051419 1351451 logs.go:123] Gathering logs for container status ...
I0819 21:16:40.051502 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0819 21:16:40.130468 1351451 logs.go:123] Gathering logs for describe nodes ...
I0819 21:16:40.130553 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0819 21:16:40.440578 1351451 logs.go:123] Gathering logs for kube-apiserver [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e] ...
I0819 21:16:40.440661 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e"
I0819 21:16:40.552968 1351451 logs.go:123] Gathering logs for etcd [de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5] ...
I0819 21:16:40.553065 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5"
I0819 21:16:40.613652 1351451 logs.go:123] Gathering logs for coredns [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e] ...
I0819 21:16:40.613737 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e"
I0819 21:16:40.704428 1351451 logs.go:123] Gathering logs for coredns [f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657] ...
I0819 21:16:40.704502 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657"
I0819 21:16:40.762919 1351451 logs.go:123] Gathering logs for storage-provisioner [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e] ...
I0819 21:16:40.762995 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e"
I0819 21:16:40.818047 1351451 logs.go:123] Gathering logs for kubernetes-dashboard [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1] ...
I0819 21:16:40.818117 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1"
I0819 21:16:40.872667 1351451 logs.go:123] Gathering logs for kube-scheduler [62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a] ...
I0819 21:16:40.872739 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a"
I0819 21:16:40.943174 1351451 logs.go:123] Gathering logs for kube-proxy [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a] ...
I0819 21:16:40.943243 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a"
I0819 21:16:41.028487 1351451 logs.go:123] Gathering logs for kube-proxy [63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e] ...
I0819 21:16:41.028563 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e"
I0819 21:16:41.099036 1351451 logs.go:123] Gathering logs for kube-controller-manager [7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525] ...
I0819 21:16:41.099118 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525"
I0819 21:16:41.217328 1351451 logs.go:123] Gathering logs for kindnet [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20] ...
I0819 21:16:41.217417 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20"
I0819 21:16:41.421061 1351451 logs.go:123] Gathering logs for kindnet [aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e] ...
I0819 21:16:41.421169 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e"
I0819 21:16:41.500448 1351451 logs.go:123] Gathering logs for kubelet ...
I0819 21:16:41.500533 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0819 21:16:41.586682 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.621751 665 reflector.go:138] object-"kube-system"/"kindnet-token-xptkw": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kindnet-token-xptkw" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.586985 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622026 665 reflector.go:138] object-"kube-system"/"coredns-token-jj86v": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-jj86v" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587223 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622171 665 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587487 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622316 665 reflector.go:138] object-"kube-system"/"storage-provisioner-token-ssctg": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-ssctg" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587731 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.622453 665 reflector.go:138] object-"default"/"default-token-vbtl7": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-vbtl7" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.587965 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671400 665 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.588235 1351451 logs.go:138] Found kubelet problem: Aug 19 21:10:57 old-k8s-version-127648 kubelet[665]: E0819 21:10:57.671660 665 reflector.go:138] object-"kube-system"/"metrics-server-token-x764s": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "metrics-server-token-x764s" is forbidden: User "system:node:old-k8s-version-127648" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-127648' and this object
W0819 21:16:41.597320 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.574230 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.597559 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:00 old-k8s-version-127648 kubelet[665]: E0819 21:11:00.789676 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.600560 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:15 old-k8s-version-127648 kubelet[665]: E0819 21:11:15.561064 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.602849 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:22 old-k8s-version-127648 kubelet[665]: E0819 21:11:22.882357 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.603229 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:23 old-k8s-version-127648 kubelet[665]: E0819 21:11:23.880032 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.603468 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:27 old-k8s-version-127648 kubelet[665]: E0819 21:11:27.549335 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.604309 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:30 old-k8s-version-127648 kubelet[665]: E0819 21:11:30.903604 665 pod_workers.go:191] Error syncing pod 74e4d116-4e4e-4dc5-af07-3013282e840a ("storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(74e4d116-4e4e-4dc5-af07-3013282e840a)"
W0819 21:16:41.604680 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:31 old-k8s-version-127648 kubelet[665]: E0819 21:11:31.736669 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 10s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.607638 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:41 old-k8s-version-127648 kubelet[665]: E0819 21:11:41.559000 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.608385 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:45 old-k8s-version-127648 kubelet[665]: E0819 21:11:45.947294 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.608989 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:51 old-k8s-version-127648 kubelet[665]: E0819 21:11:51.747256 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.609218 1351451 logs.go:138] Found kubelet problem: Aug 19 21:11:54 old-k8s-version-127648 kubelet[665]: E0819 21:11:54.549415 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.609955 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:06 old-k8s-version-127648 kubelet[665]: E0819 21:12:06.549632 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.610632 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:08 old-k8s-version-127648 kubelet[665]: E0819 21:12:08.101007 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.611019 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:11 old-k8s-version-127648 kubelet[665]: E0819 21:12:11.736495 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.611240 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:18 old-k8s-version-127648 kubelet[665]: E0819 21:12:18.550570 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.611614 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:22 old-k8s-version-127648 kubelet[665]: E0819 21:12:22.548984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.614201 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:29 old-k8s-version-127648 kubelet[665]: E0819 21:12:29.557879 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.614589 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:35 old-k8s-version-127648 kubelet[665]: E0819 21:12:35.549004 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.614826 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:41 old-k8s-version-127648 kubelet[665]: E0819 21:12:41.549476 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.615210 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:47 old-k8s-version-127648 kubelet[665]: E0819 21:12:47.548954 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.615441 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:56 old-k8s-version-127648 kubelet[665]: E0819 21:12:56.550012 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.616091 1351451 logs.go:138] Found kubelet problem: Aug 19 21:12:59 old-k8s-version-127648 kubelet[665]: E0819 21:12:59.243984 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.616472 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:01 old-k8s-version-127648 kubelet[665]: E0819 21:13:01.736702 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.616702 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:08 old-k8s-version-127648 kubelet[665]: E0819 21:13:08.550071 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.618906 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:15 old-k8s-version-127648 kubelet[665]: E0819 21:13:15.549079 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.619148 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:21 old-k8s-version-127648 kubelet[665]: E0819 21:13:21.549597 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.619523 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:27 old-k8s-version-127648 kubelet[665]: E0819 21:13:27.549537 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.619742 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:36 old-k8s-version-127648 kubelet[665]: E0819 21:13:36.549526 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.620148 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:39 old-k8s-version-127648 kubelet[665]: E0819 21:13:39.549198 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.622878 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:50 old-k8s-version-127648 kubelet[665]: E0819 21:13:50.557660 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
W0819 21:16:41.623251 1351451 logs.go:138] Found kubelet problem: Aug 19 21:13:52 old-k8s-version-127648 kubelet[665]: E0819 21:13:52.549116 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.623473 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:01 old-k8s-version-127648 kubelet[665]: E0819 21:14:01.549487 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.623831 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:06 old-k8s-version-127648 kubelet[665]: E0819 21:14:06.548964 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 1m20s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.624043 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:16 old-k8s-version-127648 kubelet[665]: E0819 21:14:16.553951 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.624685 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:20 old-k8s-version-127648 kubelet[665]: E0819 21:14:20.469865 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625055 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:21 old-k8s-version-127648 kubelet[665]: E0819 21:14:21.736453 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625267 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:31 old-k8s-version-127648 kubelet[665]: E0819 21:14:31.549363 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.625624 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:36 old-k8s-version-127648 kubelet[665]: E0819 21:14:36.549968 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.625840 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:46 old-k8s-version-127648 kubelet[665]: E0819 21:14:46.549614 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.626210 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:48 old-k8s-version-127648 kubelet[665]: E0819 21:14:48.551815 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.626434 1351451 logs.go:138] Found kubelet problem: Aug 19 21:14:57 old-k8s-version-127648 kubelet[665]: E0819 21:14:57.549314 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.626812 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:03 old-k8s-version-127648 kubelet[665]: E0819 21:15:03.548948 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.627047 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:08 old-k8s-version-127648 kubelet[665]: E0819 21:15:08.550121 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.627407 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: E0819 21:15:18.549558 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.627629 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:19 old-k8s-version-127648 kubelet[665]: E0819 21:15:19.549582 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.627992 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: E0819 21:15:31.549167 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.628208 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:34 old-k8s-version-127648 kubelet[665]: E0819 21:15:34.549642 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.628441 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.549479 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.628833 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.632353 1351451 logs.go:138] Found kubelet problem: Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.632616 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.632858 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.633216 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.633561 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.633784 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.636275 1351451 logs.go:138] Found kubelet problem: Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
I0819 21:16:41.636305 1351451 logs.go:123] Gathering logs for etcd [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267] ...
I0819 21:16:41.636337 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267"
I0819 21:16:41.736897 1351451 logs.go:123] Gathering logs for kube-scheduler [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4] ...
I0819 21:16:41.736971 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo /usr/bin/crictl logs --tail 400 c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4"
I0819 21:16:39.003265 1361984 kubeadm.go:310] [certs] Generating "etcd/ca" certificate and key
I0819 21:16:39.805980 1361984 kubeadm.go:310] [certs] Generating "etcd/server" certificate and key
I0819 21:16:39.806294 1361984 kubeadm.go:310] [certs] etcd/server serving cert is signed for DNS names [embed-certs-249735 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0819 21:16:40.260154 1361984 kubeadm.go:310] [certs] Generating "etcd/peer" certificate and key
I0819 21:16:40.260471 1361984 kubeadm.go:310] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-249735 localhost] and IPs [192.168.76.2 127.0.0.1 ::1]
I0819 21:16:41.300185 1361984 kubeadm.go:310] [certs] Generating "etcd/healthcheck-client" certificate and key
I0819 21:16:42.039736 1361984 kubeadm.go:310] [certs] Generating "apiserver-etcd-client" certificate and key
I0819 21:16:42.510620 1361984 kubeadm.go:310] [certs] Generating "sa" key and public key
I0819 21:16:42.510701 1361984 kubeadm.go:310] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0819 21:16:43.779577 1361984 kubeadm.go:310] [kubeconfig] Writing "admin.conf" kubeconfig file
I0819 21:16:44.370712 1361984 kubeadm.go:310] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I0819 21:16:44.612011 1361984 kubeadm.go:310] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0819 21:16:45.387392 1361984 kubeadm.go:310] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0819 21:16:45.648944 1361984 kubeadm.go:310] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0819 21:16:45.649601 1361984 kubeadm.go:310] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0819 21:16:45.652620 1361984 kubeadm.go:310] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0819 21:16:41.824699 1351451 logs.go:123] Gathering logs for containerd ...
I0819 21:16:41.824726 1351451 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u containerd -n 400"
I0819 21:16:41.905377 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:41.905452 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
W0819 21:16:41.905554 1351451 out.go:270] X Problems detected in kubelet:
W0819 21:16:41.905600 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.905771 1351451 out.go:270] Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.905806 1351451 out.go:270] Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0819 21:16:41.905859 1351451 out.go:270] Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
W0819 21:16:41.905906 1351451 out.go:270] Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
I0819 21:16:41.905968 1351451 out.go:358] Setting ErrFile to fd 2...
I0819 21:16:41.905993 1351451 out.go:392] TERM=,COLORTERM=, which probably does not support color
I0819 21:16:45.654802 1361984 out.go:235] - Booting up control plane ...
I0819 21:16:45.654895 1361984 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0819 21:16:45.654976 1361984 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0819 21:16:45.655045 1361984 kubeadm.go:310] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0819 21:16:45.668882 1361984 kubeadm.go:310] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0819 21:16:45.676020 1361984 kubeadm.go:310] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0819 21:16:45.676448 1361984 kubeadm.go:310] [kubelet-start] Starting the kubelet
I0819 21:16:45.783003 1361984 kubeadm.go:310] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I0819 21:16:45.783121 1361984 kubeadm.go:310] [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
I0819 21:16:47.785318 1361984 kubeadm.go:310] [kubelet-check] The kubelet is healthy after 2.002328001s
I0819 21:16:47.785419 1361984 kubeadm.go:310] [api-check] Waiting for a healthy API server. This can take up to 4m0s
I0819 21:16:51.906826 1351451 api_server.go:253] Checking apiserver healthz at https://192.168.85.2:8443/healthz ...
I0819 21:16:51.922052 1351451 api_server.go:279] https://192.168.85.2:8443/healthz returned 200:
ok
I0819 21:16:51.923612 1351451 out.go:201]
W0819 21:16:51.924862 1351451 out.go:270] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0819 21:16:51.925047 1351451 out.go:270] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0819 21:16:51.925129 1351451 out.go:270] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0819 21:16:51.925214 1351451 out.go:270] *
W0819 21:16:51.926428 1351451 out.go:293] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0819 21:16:51.927283 1351451 out.go:201]
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
90ffd31b3114d 523cad1a4df73 2 minutes ago Exited dashboard-metrics-scraper 5 f9e7ec35c46b5 dashboard-metrics-scraper-8d5bb5db8-lrvtv
3b3340693ab7f ba04bb24b9575 5 minutes ago Running storage-provisioner 2 a7fea58283917 storage-provisioner
18b6c4b9755bc 20b332c9a70d8 5 minutes ago Running kubernetes-dashboard 0 63d9a43425611 kubernetes-dashboard-cd95d586-45zvk
3eaea7a4d5646 25a5233254979 5 minutes ago Running kube-proxy 1 3c703ae455332 kube-proxy-l9jdt
1d52bf16b4b31 1611cd07b61d5 5 minutes ago Running busybox 1 55a0b3195bf4c busybox
556145ddf3521 ba04bb24b9575 5 minutes ago Exited storage-provisioner 1 a7fea58283917 storage-provisioner
1c4204d908305 db91994f4ee8f 5 minutes ago Running coredns 1 ef491a1d3b3fe coredns-74ff55c5b-fj4wf
2b97b1c764dc6 6a23fa8fd2b78 5 minutes ago Running kindnet-cni 1 b0e5da863d938 kindnet-bmcmx
c4fa68e3e489b e7605f88f17d6 6 minutes ago Running kube-scheduler 1 00655c236d420 kube-scheduler-old-k8s-version-127648
fe816949c9d21 05b738aa1bc63 6 minutes ago Running etcd 1 0d61b315278df etcd-old-k8s-version-127648
863a642d161e0 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 6d1c6c7613e2a kube-apiserver-old-k8s-version-127648
a93b5c5a814c8 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 8abb8077c6ed2 kube-controller-manager-old-k8s-version-127648
cd480a382e3b5 1611cd07b61d5 6 minutes ago Exited busybox 0 f2c6f94cae4b1 busybox
f881846bec02a db91994f4ee8f 8 minutes ago Exited coredns 0 2af76c44ae9bc coredns-74ff55c5b-fj4wf
aa6e1bccf3b9e 6a23fa8fd2b78 8 minutes ago Exited kindnet-cni 0 6df4aa804107e kindnet-bmcmx
63fbaf1d278cf 25a5233254979 8 minutes ago Exited kube-proxy 0 62f7f6ea5d35c kube-proxy-l9jdt
de6ad75777dfe 05b738aa1bc63 9 minutes ago Exited etcd 0 818fe65da9741 etcd-old-k8s-version-127648
ddd7fbd4fc4b1 2c08bbbc02d3a 9 minutes ago Exited kube-apiserver 0 0b5486c0d7170 kube-apiserver-old-k8s-version-127648
62242d9fb5d71 e7605f88f17d6 9 minutes ago Exited kube-scheduler 0 d43ca1ea08068 kube-scheduler-old-k8s-version-127648
7708e0950e37b 1df8a2b116bd1 9 minutes ago Exited kube-controller-manager 0 b11647aff870d kube-controller-manager-old-k8s-version-127648
==> containerd <==
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.576368174Z" level=info msg="CreateContainer within sandbox \"f9e7ec35c46b59d7bfb95d83e73bc014ac3da08d8d937d44497be6926a5b71f4\" for name:\"dashboard-metrics-scraper\" attempt:4 returns container id \"98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72\""
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.577098347Z" level=info msg="StartContainer for \"98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72\""
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.642349661Z" level=info msg="StartContainer for \"98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72\" returns successfully"
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.678189633Z" level=info msg="shim disconnected" id=98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72 namespace=k8s.io
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.678260015Z" level=warning msg="cleaning up after shim disconnected" id=98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72 namespace=k8s.io
Aug 19 21:12:58 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:58.678269647Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Aug 19 21:12:59 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:59.264012258Z" level=info msg="RemoveContainer for \"9c9f120d3b8c5c3120bc430c1904f7faf3099f6d257b5ad287df21e88ebb8e1d\""
Aug 19 21:12:59 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:12:59.270166965Z" level=info msg="RemoveContainer for \"9c9f120d3b8c5c3120bc430c1904f7faf3099f6d257b5ad287df21e88ebb8e1d\" returns successfully"
Aug 19 21:13:50 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:13:50.550335788Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:13:50 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:13:50.555324295Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Aug 19 21:13:50 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:13:50.557219199Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 19 21:13:50 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:13:50.557311349Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.550942549Z" level=info msg="CreateContainer within sandbox \"f9e7ec35c46b59d7bfb95d83e73bc014ac3da08d8d937d44497be6926a5b71f4\" for container name:\"dashboard-metrics-scraper\" attempt:5"
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.570508682Z" level=info msg="CreateContainer within sandbox \"f9e7ec35c46b59d7bfb95d83e73bc014ac3da08d8d937d44497be6926a5b71f4\" for name:\"dashboard-metrics-scraper\" attempt:5 returns container id \"90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc\""
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.571042133Z" level=info msg="StartContainer for \"90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc\""
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.653774379Z" level=info msg="StartContainer for \"90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc\" returns successfully"
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.687092401Z" level=info msg="shim disconnected" id=90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc namespace=k8s.io
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.687155677Z" level=warning msg="cleaning up after shim disconnected" id=90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc namespace=k8s.io
Aug 19 21:14:19 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:19.687167156Z" level=info msg="cleaning up dead shim" namespace=k8s.io
Aug 19 21:14:20 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:20.475867088Z" level=info msg="RemoveContainer for \"98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72\""
Aug 19 21:14:20 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:14:20.492350817Z" level=info msg="RemoveContainer for \"98808d57df0038ffeaa125bfff2b878ccd2ab753519c4e3d7dc0fdfb695cce72\" returns successfully"
Aug 19 21:16:39 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:16:39.550330462Z" level=info msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:39 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:16:39.555270100Z" level=info msg="trying next host" error="failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host" host=fake.domain
Aug 19 21:16:39 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:16:39.556303782Z" level=error msg="PullImage \"fake.domain/registry.k8s.io/echoserver:1.4\" failed" error="failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 19 21:16:39 old-k8s-version-127648 containerd[567]: time="2024-08-19T21:16:39.556425510Z" level=info msg="stop pulling image fake.domain/registry.k8s.io/echoserver:1.4: active requests=0, bytes read=0"
==> coredns [1c4204d908305e49b9bc8e0a713f3a60c3701a9edfadbf50bdaf6f12d055498e] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:57858 - 31612 "HINFO IN 5207757024897347302.6221884481298753466. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.019630763s
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
I0819 21:11:29.966002 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-19 21:10:59.965352446 +0000 UTC m=+0.027452277) (total time: 30.00054843s):
Trace[2019727887]: [30.00054843s] [30.00054843s] END
E0819 21:11:29.966040 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0819 21:11:29.966365 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-19 21:10:59.965925962 +0000 UTC m=+0.028025802) (total time: 30.000416109s):
Trace[939984059]: [30.000416109s] [30.000416109s] END
E0819 21:11:29.966427 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0819 21:11:29.966534 1 trace.go:116] Trace[911902081]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-08-19 21:10:59.966191211 +0000 UTC m=+0.028291051) (total time: 30.000331705s):
Trace[911902081]: [30.000331705s] [30.000331705s] END
E0819 21:11:29.966618 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
==> coredns [f881846bec02adc45467aa1bcde798e06c205d2f213869c24d08f2277be24657] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = 093a0bf1423dd8c4eee62372bb216168
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:35444 - 40715 "HINFO IN 4304486271716434029.2850386008944031993. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.046241969s
==> describe nodes <==
Name: old-k8s-version-127648
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-127648
kubernetes.io/os=linux
minikube.k8s.io/commit=7253360125032c7e2214e25ff4b5c894ae5844e8
minikube.k8s.io/name=old-k8s-version-127648
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_08_19T21_08_00_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 19 Aug 2024 21:07:56 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-127648
AcquireTime: <unset>
RenewTime: Mon, 19 Aug 2024 21:16:50 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 19 Aug 2024 21:11:58 +0000 Mon, 19 Aug 2024 21:07:50 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 19 Aug 2024 21:11:58 +0000 Mon, 19 Aug 2024 21:07:50 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 19 Aug 2024 21:11:58 +0000 Mon, 19 Aug 2024 21:07:50 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 19 Aug 2024 21:11:58 +0000 Mon, 19 Aug 2024 21:08:15 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.85.2
Hostname: old-k8s-version-127648
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022360Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022360Ki
pods: 110
System Info:
Machine ID: 8a06a41ecef14bd78834cf9d4770a35f
System UUID: cd5a7492-44e3-4e58-8a1a-93535ca2aa9b
Boot ID: b7846bbc-2ca5-4e44-8ea6-94e5c03d25fd
Kernel Version: 5.15.0-1067-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: containerd://1.7.20
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (12 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6m46s
kube-system coredns-74ff55c5b-fj4wf 100m (5%) 0 (0%) 70Mi (0%) 170Mi (2%) 8m39s
kube-system etcd-old-k8s-version-127648 100m (5%) 0 (0%) 100Mi (1%) 0 (0%) 8m46s
kube-system kindnet-bmcmx 100m (5%) 100m (5%) 50Mi (0%) 50Mi (0%) 8m39s
kube-system kube-apiserver-old-k8s-version-127648 250m (12%) 0 (0%) 0 (0%) 0 (0%) 8m46s
kube-system kube-controller-manager-old-k8s-version-127648 200m (10%) 0 (0%) 0 (0%) 0 (0%) 8m46s
kube-system kube-proxy-l9jdt 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m39s
kube-system kube-scheduler-old-k8s-version-127648 100m (5%) 0 (0%) 0 (0%) 0 (0%) 8m46s
kube-system metrics-server-9975d5f86-4glzw 100m (5%) 0 (0%) 200Mi (2%) 0 (0%) 6m36s
kube-system storage-provisioner 0 (0%) 0 (0%) 0 (0%) 0 (0%) 8m37s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-lrvtv 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m39s
kubernetes-dashboard kubernetes-dashboard-cd95d586-45zvk 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5m39s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 950m (47%) 100m (5%)
memory 420Mi (5%) 220Mi (2%)
ephemeral-storage 100Mi (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
hugepages-32Mi 0 (0%) 0 (0%)
hugepages-64Ki 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 9m6s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 9m6s (x4 over 9m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 9m6s (x3 over 9m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 9m6s (x3 over 9m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 9m6s kubelet Updated Node Allocatable limit across pods
Normal Starting 8m46s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m46s kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m46s kubelet Node old-k8s-version-127648 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m46s kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m46s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m39s kubelet Node old-k8s-version-127648 status is now: NodeReady
Normal Starting 8m37s kube-proxy Starting kube-proxy.
Normal Starting 6m6s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m6s (x7 over 6m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m6s (x8 over 6m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m6s (x8 over 6m6s) kubelet Node old-k8s-version-127648 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m6s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m53s kube-proxy Starting kube-proxy.
==> dmesg <==
==> etcd [de6ad75777dfe1d5d57ea2a85334e48c13ce3eefa7d1e083b8c7d11680d27ee5] <==
raft2024/08/19 21:07:50 INFO: 9f0758e1c58a86ed became candidate at term 2
raft2024/08/19 21:07:50 INFO: 9f0758e1c58a86ed received MsgVoteResp from 9f0758e1c58a86ed at term 2
raft2024/08/19 21:07:50 INFO: 9f0758e1c58a86ed became leader at term 2
raft2024/08/19 21:07:50 INFO: raft.node: 9f0758e1c58a86ed elected leader 9f0758e1c58a86ed at term 2
2024-08-19 21:07:50.560309 I | etcdserver: setting up the initial cluster version to 3.4
2024-08-19 21:07:50.562948 N | etcdserver/membership: set the initial cluster version to 3.4
2024-08-19 21:07:50.563122 I | etcdserver/api: enabled capabilities for version 3.4
2024-08-19 21:07:50.563212 I | etcdserver: published {Name:old-k8s-version-127648 ClientURLs:[https://192.168.85.2:2379]} to cluster 68eaea490fab4e05
2024-08-19 21:07:50.563292 I | embed: ready to serve client requests
2024-08-19 21:07:50.566952 I | embed: serving client requests on 127.0.0.1:2379
2024-08-19 21:07:50.567181 I | embed: ready to serve client requests
2024-08-19 21:07:50.579663 I | embed: serving client requests on 192.168.85.2:2379
2024-08-19 21:08:14.784950 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:08:23.479691 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:08:33.479699 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:08:43.479874 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:08:53.479690 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:03.479948 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:13.479716 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:23.479750 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:33.479570 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:43.479749 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:09:53.479893 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:10:03.479694 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:10:13.479685 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [fe816949c9d21133dfa290d66d4a672f9934aa3758c37c6b52633143ce54f267] <==
2024-08-19 21:12:44.993663 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:12:54.993582 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:04.993517 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:14.993569 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:24.993550 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:34.993455 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:44.993713 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:13:54.993526 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:04.993742 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:14.993527 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:24.993586 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:34.993550 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:44.993992 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:14:54.993504 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:04.993549 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:14.993538 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:24.993813 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:34.993557 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:44.993650 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:15:54.993671 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:16:04.993600 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:16:14.993932 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:16:24.993727 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:16:34.993729 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-08-19 21:16:44.997223 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> kernel <==
21:16:54 up 4:59, 0 users, load average: 1.75, 1.71, 2.31
Linux old-k8s-version-127648 5.15.0-1067-aws #73~20.04.1-Ubuntu SMP Wed Jul 24 17:31:05 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kindnet [2b97b1c764dc6a9ef65fd39fe731d38828a1720efb0d703155aa021f2e1cfa20] <==
I0819 21:15:30.400212 1 main.go:299] handling current node
I0819 21:15:40.399149 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:15:40.399187 1 main.go:299] handling current node
W0819 21:15:45.601116 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0819 21:15:45.601178 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0819 21:15:50.400010 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:15:50.400044 1 main.go:299] handling current node
I0819 21:16:00.399086 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:00.399126 1 main.go:299] handling current node
I0819 21:16:10.399243 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:10.399279 1 main.go:299] handling current node
W0819 21:16:12.794344 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0819 21:16:12.794394 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
I0819 21:16:20.399644 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:20.399699 1 main.go:299] handling current node
W0819 21:16:25.229635 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0819 21:16:25.229858 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0819 21:16:30.399966 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:30.400072 1 main.go:299] handling current node
W0819 21:16:34.848289 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0819 21:16:34.848332 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0819 21:16:40.402808 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:40.403030 1 main.go:299] handling current node
I0819 21:16:50.399263 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:16:50.399306 1 main.go:299] handling current node
==> kindnet [aa6e1bccf3b9e02915f7d77581c43d4973f5450dee4cbe799ab617b0d312978e] <==
E0819 21:09:01.150139 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0819 21:09:08.918239 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:08.918279 1 main.go:299] handling current node
I0819 21:09:18.917625 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:18.917668 1 main.go:299] handling current node
W0819 21:09:21.817271 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0819 21:09:21.817307 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
W0819 21:09:27.941506 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0819 21:09:27.941543 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
I0819 21:09:28.918073 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:28.918147 1 main.go:299] handling current node
I0819 21:09:38.917616 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:38.917658 1 main.go:299] handling current node
W0819 21:09:48.290292 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
E0819 21:09:48.290350 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.NetworkPolicy: failed to list *v1.NetworkPolicy: networkpolicies.networking.k8s.io is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "networkpolicies" in API group "networking.k8s.io" at the cluster scope
I0819 21:09:48.918124 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:48.918172 1 main.go:299] handling current node
I0819 21:09:58.917787 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:09:58.917821 1 main.go:299] handling current node
W0819 21:10:01.006124 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
E0819 21:10:01.006169 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "pods" in API group "" at the cluster scope
I0819 21:10:08.917471 1 main.go:295] Handling node with IPs: map[192.168.85.2:{}]
I0819 21:10:08.917509 1 main.go:299] handling current node
W0819 21:10:10.515149 1 reflector.go:547] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
E0819 21:10:10.515183 1 reflector.go:150] pkg/mod/k8s.io/client-go@v0.30.3/tools/cache/reflector.go:232: Failed to watch *v1.Namespace: failed to list *v1.Namespace: namespaces is forbidden: User "system:serviceaccount:kube-system:kindnet" cannot list resource "namespaces" in API group "" at the cluster scope
==> kube-apiserver [863a642d161e02c4a9c3c27a6f9246c0a91afa231d64297aa4c31f2bf085af9e] <==
I0819 21:13:35.245547 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:13:35.245578 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0819 21:14:00.445033 1 handler_proxy.go:102] no RequestInfo found in the context
E0819 21:14:00.445309 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0819 21:14:00.445325 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0819 21:14:19.624367 1 client.go:360] parsed scheme: "passthrough"
I0819 21:14:19.624409 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:14:19.624433 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:14:53.796882 1 client.go:360] parsed scheme: "passthrough"
I0819 21:14:53.796929 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:14:53.796938 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:15:27.257877 1 client.go:360] parsed scheme: "passthrough"
I0819 21:15:27.257921 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:15:27.257931 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0819 21:15:58.650104 1 handler_proxy.go:102] no RequestInfo found in the context
E0819 21:15:58.650423 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0819 21:15:58.650443 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0819 21:16:02.875576 1 client.go:360] parsed scheme: "passthrough"
I0819 21:16:02.875640 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:16:02.875650 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:16:41.242299 1 client.go:360] parsed scheme: "passthrough"
I0819 21:16:41.242355 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:16:41.242365 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-apiserver [ddd7fbd4fc4b1fc85931d27f22ce6544aefafe553e7ad058a444cb1240d045ce] <==
I0819 21:07:57.652777 1 storage_scheduling.go:132] created PriorityClass system-cluster-critical with value 2000000000
I0819 21:07:57.653544 1 storage_scheduling.go:148] all system priority classes are created successfully or already exist.
I0819 21:07:58.126088 1 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0819 21:07:58.171557 1 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
W0819 21:07:58.269714 1 lease.go:233] Resetting endpoints for master service "kubernetes" to [192.168.85.2]
I0819 21:07:58.270694 1 controller.go:606] quota admission added evaluator for: endpoints
I0819 21:07:58.275353 1 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
I0819 21:07:59.313026 1 controller.go:606] quota admission added evaluator for: serviceaccounts
I0819 21:07:59.951199 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0819 21:08:00.173007 1 controller.go:606] quota admission added evaluator for: daemonsets.apps
I0819 21:08:08.596742 1 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
I0819 21:08:15.346859 1 controller.go:606] quota admission added evaluator for: controllerrevisions.apps
I0819 21:08:15.362639 1 controller.go:606] quota admission added evaluator for: replicasets.apps
I0819 21:08:21.396840 1 client.go:360] parsed scheme: "passthrough"
I0819 21:08:21.396883 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:08:21.396891 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:09:04.689686 1 client.go:360] parsed scheme: "passthrough"
I0819 21:09:04.689726 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:09:04.689775 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:09:39.845014 1 client.go:360] parsed scheme: "passthrough"
I0819 21:09:39.845062 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:09:39.845071 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0819 21:10:14.353885 1 client.go:360] parsed scheme: "passthrough"
I0819 21:10:14.353950 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0819 21:10:14.353960 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [7708e0950e37b20add16fe5d8f9b34d5f3c928b13167f2ce6017087538b63525] <==
I0819 21:08:15.332668 1 taint_manager.go:187] Starting NoExecuteTaintManager
I0819 21:08:15.332941 1 event.go:291] "Event occurred" object="old-k8s-version-127648" kind="Node" apiVersion="v1" type="Normal" reason="RegisteredNode" message="Node old-k8s-version-127648 event: Registered Node old-k8s-version-127648 in Controller"
I0819 21:08:15.355063 1 shared_informer.go:247] Caches are synced for endpoint_slice_mirroring
I0819 21:08:15.414073 1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-l9jdt"
I0819 21:08:15.438282 1 shared_informer.go:247] Caches are synced for endpoint_slice
I0819 21:08:15.441232 1 event.go:291] "Event occurred" object="kube-system/kindnet" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kindnet-bmcmx"
I0819 21:08:15.439391 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-74ff55c5b to 2"
I0819 21:08:15.471004 1 shared_informer.go:247] Caches are synced for resource quota
I0819 21:08:15.480463 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-24vqz"
I0819 21:08:15.497907 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-74ff55c5b-fj4wf"
I0819 21:08:15.505849 1 shared_informer.go:247] Caches are synced for attach detach
I0819 21:08:15.509714 1 shared_informer.go:247] Caches are synced for resource quota
I0819 21:08:15.644863 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0819 21:08:15.945082 1 shared_informer.go:247] Caches are synced for garbage collector
I0819 21:08:15.954290 1 shared_informer.go:247] Caches are synced for garbage collector
I0819 21:08:15.954328 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0819 21:08:16.680448 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0819 21:08:16.711889 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-24vqz"
I0819 21:08:20.332640 1 node_lifecycle_controller.go:1222] Controller detected that some Nodes are Ready. Exiting master disruption mode.
I0819 21:10:17.736776 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
I0819 21:10:17.824803 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: pods \"metrics-server-9975d5f86-\" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount \"metrics-server\" not found"
E0819 21:10:17.849379 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with pods "metrics-server-9975d5f86-" is forbidden: error looking up service account kube-system/metrics-server: serviceaccount "metrics-server" not found
E0819 21:10:18.030693 1 clusterroleaggregation_controller.go:181] edit failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "edit": the object has been modified; please apply your changes to the latest version and try again
E0819 21:10:18.051546 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
I0819 21:10:18.882772 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: metrics-server-9975d5f86-4glzw"
==> kube-controller-manager [a93b5c5a814c88adf2a58e19336d5018f346f39f3866b52c57d7df973b978363] <==
E0819 21:12:47.085409 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:12:55.052911 1 request.go:655] Throttling request took 1.043835654s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1?timeout=32s
W0819 21:12:55.904561 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:13:17.590688 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:13:27.555145 1 request.go:655] Throttling request took 1.048227965s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0819 21:13:28.406616 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:13:48.092653 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:14:00.057261 1 request.go:655] Throttling request took 1.048423718s, request: GET:https://192.168.85.2:8443/apis/policy/v1beta1?timeout=32s
W0819 21:14:00.908378 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:14:18.594429 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:14:32.558917 1 request.go:655] Throttling request took 1.048281502s, request: GET:https://192.168.85.2:8443/apis/scheduling.k8s.io/v1?timeout=32s
W0819 21:14:33.410396 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:14:49.095238 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:15:05.060848 1 request.go:655] Throttling request took 1.048208338s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1?timeout=32s
W0819 21:15:05.912356 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:15:19.602413 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:15:37.562839 1 request.go:655] Throttling request took 1.04850511s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1?timeout=32s
W0819 21:15:38.414352 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:15:50.104492 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:16:10.064956 1 request.go:655] Throttling request took 1.048215287s, request: GET:https://192.168.85.2:8443/apis/coordination.k8s.io/v1beta1?timeout=32s
W0819 21:16:10.916497 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:16:20.644621 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0819 21:16:42.566859 1 request.go:655] Throttling request took 1.048380714s, request: GET:https://192.168.85.2:8443/apis/extensions/v1beta1?timeout=32s
W0819 21:16:43.418434 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0819 21:16:51.146905 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
==> kube-proxy [3eaea7a4d564682eae94a2b48007416b88761d90abe2579a6ba16eff34bdba9a] <==
I0819 21:11:01.839659 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0819 21:11:01.839950 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0819 21:11:01.863917 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0819 21:11:01.864135 1 server_others.go:185] Using iptables Proxier.
I0819 21:11:01.864679 1 server.go:650] Version: v1.20.0
I0819 21:11:01.865602 1 config.go:224] Starting endpoint slice config controller
I0819 21:11:01.865772 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0819 21:11:01.865948 1 config.go:315] Starting service config controller
I0819 21:11:01.866042 1 shared_informer.go:240] Waiting for caches to sync for service config
I0819 21:11:01.966357 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0819 21:11:01.966661 1 shared_informer.go:247] Caches are synced for service config
==> kube-proxy [63fbaf1d278cf5491ca7dd5ad4fdb07c3e8a90f6f4dfbcf5e073c12bf517db0e] <==
I0819 21:08:17.635555 1 node.go:172] Successfully retrieved node IP: 192.168.85.2
I0819 21:08:17.635641 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.85.2), assume IPv4 operation
W0819 21:08:17.659167 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0819 21:08:17.659250 1 server_others.go:185] Using iptables Proxier.
I0819 21:08:17.659451 1 server.go:650] Version: v1.20.0
I0819 21:08:17.660277 1 config.go:315] Starting service config controller
I0819 21:08:17.660286 1 shared_informer.go:240] Waiting for caches to sync for service config
I0819 21:08:17.660303 1 config.go:224] Starting endpoint slice config controller
I0819 21:08:17.660307 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0819 21:08:17.761449 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0819 21:08:17.761523 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [62242d9fb5d7108b3cbff9c68fdf6693fa30bbe1a57a81165d6adf95d2f9a15a] <==
I0819 21:07:50.855590 1 serving.go:331] Generated self-signed cert in-memory
W0819 21:07:56.792780 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0819 21:07:56.792819 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0819 21:07:56.792829 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0819 21:07:56.792834 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0819 21:07:56.884146 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0819 21:07:56.887113 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0819 21:07:56.887728 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0819 21:07:56.887890 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0819 21:07:56.895115 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0819 21:07:56.898979 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0819 21:07:56.899365 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0819 21:07:56.900932 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0819 21:07:56.901742 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0819 21:07:56.902004 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0819 21:07:56.903337 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0819 21:07:56.904890 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0819 21:07:56.905859 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0819 21:07:56.910629 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0819 21:07:56.911047 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0819 21:07:56.911796 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0819 21:07:57.733164 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0819 21:07:57.775931 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0819 21:07:57.807718 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
I0819 21:07:59.887934 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [c4fa68e3e489b8ed0f15e0a99bcf04281f96ae3328ebc705a20531880e4d77b4] <==
I0819 21:10:51.210123 1 serving.go:331] Generated self-signed cert in-memory
W0819 21:10:57.474711 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0819 21:10:57.474748 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0819 21:10:57.474770 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0819 21:10:57.474776 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0819 21:10:57.753548 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0819 21:10:57.759137 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0819 21:10:57.759210 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0819 21:10:57.759231 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0819 21:10:57.864076 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: I0819 21:15:18.549161 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:15:18 old-k8s-version-127648 kubelet[665]: E0819 21:15:18.549558 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:15:19 old-k8s-version-127648 kubelet[665]: E0819 21:15:19.549582 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: I0819 21:15:31.548813 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:15:31 old-k8s-version-127648 kubelet[665]: E0819 21:15:31.549167 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:15:34 old-k8s-version-127648 kubelet[665]: E0819 21:15:34.549642 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.549479 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: I0819 21:15:46.550894 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:15:46 old-k8s-version-127648 kubelet[665]: E0819 21:15:46.551366 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: I0819 21:15:59.548729 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:15:59 old-k8s-version-127648 kubelet[665]: E0819 21:15:59.549085 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:00 old-k8s-version-127648 kubelet[665]: E0819 21:16:00.554355 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.549495 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: I0819 21:16:14.550464 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:16:14 old-k8s-version-127648 kubelet[665]: E0819 21:16:14.550809 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: I0819 21:16:28.549487 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.551255 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
Aug 19 21:16:28 old-k8s-version-127648 kubelet[665]: E0819 21:16:28.549823 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556742 665 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556786 665 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556913 665 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-x764s,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exec
:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-4glzw_kube-system(611ff47
f-2896-429d-b32c-b9fbf62a64f3): ErrImagePull: rpc error: code = Unknown desc = failed to pull and unpack image "fake.domain/registry.k8s.io/echoserver:1.4": failed to resolve reference "fake.domain/registry.k8s.io/echoserver:1.4": failed to do request: Head "https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host
Aug 19 21:16:39 old-k8s-version-127648 kubelet[665]: E0819 21:16:39.556947 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = failed to pull and unpack image \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to resolve reference \"fake.domain/registry.k8s.io/echoserver:1.4\": failed to do request: Head \"https://fake.domain/v2/registry.k8s.io/echoserver/manifests/1.4\": dial tcp: lookup fake.domain on 192.168.85.1:53: no such host"
Aug 19 21:16:43 old-k8s-version-127648 kubelet[665]: I0819 21:16:43.548646 665 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90ffd31b3114d75eda8948c5e9153728e6ff7db21822b6f58bafd791e62e15bc
Aug 19 21:16:43 old-k8s-version-127648 kubelet[665]: E0819 21:16:43.549476 665 pod_workers.go:191] Error syncing pod 838001cf-190e-416b-a50f-c260a8b8a946 ("dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with CrashLoopBackOff: "back-off 2m40s restarting failed container=dashboard-metrics-scraper pod=dashboard-metrics-scraper-8d5bb5db8-lrvtv_kubernetes-dashboard(838001cf-190e-416b-a50f-c260a8b8a946)"
Aug 19 21:16:51 old-k8s-version-127648 kubelet[665]: E0819 21:16:51.555869 665 pod_workers.go:191] Error syncing pod 611ff47f-2896-429d-b32c-b9fbf62a64f3 ("metrics-server-9975d5f86-4glzw_kube-system(611ff47f-2896-429d-b32c-b9fbf62a64f3)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
==> kubernetes-dashboard [18b6c4b9755bc4f8f489bd6d79488572db886134c98fa78b6b1ebbbaf13e78a1] <==
2024/08/19 21:11:24 Using namespace: kubernetes-dashboard
2024/08/19 21:11:24 Using in-cluster config to connect to apiserver
2024/08/19 21:11:24 Using secret token for csrf signing
2024/08/19 21:11:24 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/08/19 21:11:24 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/08/19 21:11:24 Successful initial request to the apiserver, version: v1.20.0
2024/08/19 21:11:24 Generating JWE encryption key
2024/08/19 21:11:24 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/08/19 21:11:24 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/08/19 21:11:24 Initializing JWE encryption key from synchronized object
2024/08/19 21:11:24 Creating in-cluster Sidecar client
2024/08/19 21:11:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:11:24 Serving insecurely on HTTP port: 9090
2024/08/19 21:11:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:12:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:12:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:13:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:13:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:14:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:14:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:15:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:15:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:16:24 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:16:54 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/08/19 21:11:24 Starting overwatch
==> storage-provisioner [3b3340693ab7f6759c32d8665585f8d968c8d552cf6180284c4eaeed124e3d8e] <==
I0819 21:11:46.856654 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0819 21:11:46.910086 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0819 21:11:46.910303 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0819 21:12:04.420366 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0819 21:12:04.420600 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-127648_d54e4fd2-f404-4b49-9985-07c6db5272fd!
I0819 21:12:04.421779 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"75385448-a290-4473-9ab6-ed14a3aaa903", APIVersion:"v1", ResourceVersion:"882", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-127648_d54e4fd2-f404-4b49-9985-07c6db5272fd became leader
I0819 21:12:04.520737 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-127648_d54e4fd2-f404-4b49-9985-07c6db5272fd!
==> storage-provisioner [556145ddf35210e0aed6253010261fc026d520ff7cf4f070c664092c877a33dd] <==
I0819 21:11:00.452711 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0819 21:11:30.455286 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-127648 -n old-k8s-version-127648
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-127648 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-4glzw
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-127648 describe pod metrics-server-9975d5f86-4glzw
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-127648 describe pod metrics-server-9975d5f86-4glzw: exit status 1 (121.740772ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-4glzw" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-127648 describe pod metrics-server-9975d5f86-4glzw: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (384.83s)