=== RUN TestStartStop/group/old-k8s-version/serial/SecondStart
start_stop_delete_test.go:256: (dbg) Run: out/minikube-linux-arm64 start -p old-k8s-version-185009 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0
start_stop_delete_test.go:256: (dbg) Non-zero exit: out/minikube-linux-arm64 start -p old-k8s-version-185009 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0: exit status 102 (6m11.278587607s)
-- stdout --
* [old-k8s-version-185009] minikube v1.33.1 on Ubuntu 20.04 (arm64)
- MINIKUBE_LOCATION=18929
- MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
- KUBECONFIG=/home/jenkins/minikube-integration/18929-2395474/kubeconfig
- MINIKUBE_HOME=/home/jenkins/minikube-integration/18929-2395474/.minikube
- MINIKUBE_BIN=out/minikube-linux-arm64
- MINIKUBE_FORCE_SYSTEMD=
* Kubernetes 1.30.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.1
* Using the docker driver based on existing profile
* Starting "old-k8s-version-185009" primary control-plane node in "old-k8s-version-185009" cluster
* Pulling base image v0.0.44-1715707529-18887 ...
* Restarting existing docker container for "old-k8s-version-185009" ...
* Preparing Kubernetes v1.20.0 on Docker 26.1.2 ...
* Verifying Kubernetes components...
- Using image gcr.io/k8s-minikube/storage-provisioner:v5
- Using image docker.io/kubernetesui/dashboard:v2.7.0
- Using image registry.k8s.io/echoserver:1.4
- Using image fake.domain/registry.k8s.io/echoserver:1.4
* Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-185009 addons enable metrics-server
* Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
-- /stdout --
** stderr **
I0520 15:12:21.659166 2687927 out.go:291] Setting OutFile to fd 1 ...
I0520 15:12:21.659351 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:12:21.659363 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:12:21.659368 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:12:21.659637 2687927 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18929-2395474/.minikube/bin
I0520 15:12:21.660043 2687927 out.go:298] Setting JSON to false
I0520 15:12:21.661128 2687927 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":168889,"bootTime":1716049053,"procs":212,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1058-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I0520 15:12:21.662163 2687927 start.go:139] virtualization:
I0520 15:12:21.665851 2687927 out.go:177] * [old-k8s-version-185009] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0520 15:12:21.668613 2687927 out.go:177] - MINIKUBE_LOCATION=18929
I0520 15:12:21.671339 2687927 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0520 15:12:21.668670 2687927 notify.go:220] Checking for updates...
I0520 15:12:21.674161 2687927 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18929-2395474/kubeconfig
I0520 15:12:21.677762 2687927 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18929-2395474/.minikube
I0520 15:12:21.680475 2687927 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0520 15:12:21.682986 2687927 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0520 15:12:21.686202 2687927 config.go:182] Loaded profile config "old-k8s-version-185009": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0520 15:12:21.689357 2687927 out.go:177] * Kubernetes 1.30.1 is now available. If you would like to upgrade, specify: --kubernetes-version=v1.30.1
I0520 15:12:21.692378 2687927 driver.go:392] Setting default libvirt URI to qemu:///system
I0520 15:12:21.725124 2687927 docker.go:122] docker version: linux-26.1.3:Docker Engine - Community
I0520 15:12:21.725265 2687927 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0520 15:12:21.837273 2687927 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:57 OomKillDisable:true NGoroutines:69 SystemTime:2024-05-20 15:12:21.817083217 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1058-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0520 15:12:21.837388 2687927 docker.go:295] overlay module found
I0520 15:12:21.840253 2687927 out.go:177] * Using the docker driver based on existing profile
I0520 15:12:21.842803 2687927 start.go:297] selected driver: docker
I0520 15:12:21.842825 2687927 start.go:901] validating driver "docker" against &{Name:old-k8s-version-185009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-185009 Namespace:default APIServerHAVIP: AP
IServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountSt
ring:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0520 15:12:21.842951 2687927 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0520 15:12:21.843651 2687927 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0520 15:12:21.957112 2687927 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:2 ContainersRunning:1 ContainersPaused:0 ContainersStopped:1 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:57 OomKillDisable:true NGoroutines:69 SystemTime:2024-05-20 15:12:21.944840558 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1058-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0520 15:12:21.957465 2687927 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0520 15:12:21.957486 2687927 cni.go:84] Creating CNI manager for ""
I0520 15:12:21.957499 2687927 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I0520 15:12:21.957533 2687927 start.go:340] cluster config:
{Name:old-k8s-version-185009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-185009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local
ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountI
P: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0520 15:12:21.960581 2687927 out.go:177] * Starting "old-k8s-version-185009" primary control-plane node in "old-k8s-version-185009" cluster
I0520 15:12:21.963136 2687927 cache.go:121] Beginning downloading kic base image for docker with docker
I0520 15:12:21.965949 2687927 out.go:177] * Pulling base image v0.0.44-1715707529-18887 ...
I0520 15:12:21.968633 2687927 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker
I0520 15:12:21.968691 2687927 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4
I0520 15:12:21.968704 2687927 cache.go:56] Caching tarball of preloaded images
I0520 15:12:21.968711 2687927 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a in local docker daemon
I0520 15:12:21.968811 2687927 preload.go:173] Found /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.20.0-docker-overlay2-arm64.tar.lz4 in cache, skipping download
I0520 15:12:21.968826 2687927 cache.go:59] Finished verifying existence of preloaded tar for v1.20.0 on docker
I0520 15:12:21.968936 2687927 profile.go:143] Saving config to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/config.json ...
I0520 15:12:21.986425 2687927 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a in local docker daemon, skipping pull
I0520 15:12:21.986449 2687927 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a exists in daemon, skipping load
I0520 15:12:21.986469 2687927 cache.go:194] Successfully downloaded all kic artifacts
I0520 15:12:21.986513 2687927 start.go:360] acquireMachinesLock for old-k8s-version-185009: {Name:mk84701f4a7a8f9a38ef64ea5be7a3b336b58466 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 15:12:21.986593 2687927 start.go:364] duration metric: took 52.684µs to acquireMachinesLock for "old-k8s-version-185009"
I0520 15:12:21.986614 2687927 start.go:96] Skipping create...Using existing machine configuration
I0520 15:12:21.986620 2687927 fix.go:54] fixHost starting:
I0520 15:12:21.986894 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:22.005172 2687927 fix.go:112] recreateIfNeeded on old-k8s-version-185009: state=Stopped err=<nil>
W0520 15:12:22.005207 2687927 fix.go:138] unexpected machine state, will restart: <nil>
I0520 15:12:22.010318 2687927 out.go:177] * Restarting existing docker container for "old-k8s-version-185009" ...
I0520 15:12:22.013029 2687927 cli_runner.go:164] Run: docker start old-k8s-version-185009
I0520 15:12:22.427172 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:22.454819 2687927 kic.go:430] container "old-k8s-version-185009" state is running.
I0520 15:12:22.455217 2687927 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-185009
I0520 15:12:22.483966 2687927 profile.go:143] Saving config to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/config.json ...
I0520 15:12:22.484233 2687927 machine.go:94] provisionDockerMachine start ...
I0520 15:12:22.484297 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:22.517400 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:22.517835 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:22.517850 2687927 main.go:141] libmachine: About to run SSH command:
hostname
I0520 15:12:22.518547 2687927 main.go:141] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:59956->127.0.0.1:41933: read: connection reset by peer
I0520 15:12:25.657327 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-185009
I0520 15:12:25.657353 2687927 ubuntu.go:169] provisioning hostname "old-k8s-version-185009"
I0520 15:12:25.657421 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:25.675370 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:25.675638 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:25.675656 2687927 main.go:141] libmachine: About to run SSH command:
sudo hostname old-k8s-version-185009 && echo "old-k8s-version-185009" | sudo tee /etc/hostname
I0520 15:12:25.815374 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>: old-k8s-version-185009
I0520 15:12:25.815463 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:25.833155 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:25.833401 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:25.833419 2687927 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sold-k8s-version-185009' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 old-k8s-version-185009/g' /etc/hosts;
else
echo '127.0.1.1 old-k8s-version-185009' | sudo tee -a /etc/hosts;
fi
fi
I0520 15:12:25.963152 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0520 15:12:25.963180 2687927 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18929-2395474/.minikube CaCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18929-2395474/.minikube}
I0520 15:12:25.963203 2687927 ubuntu.go:177] setting up certificates
I0520 15:12:25.963212 2687927 provision.go:84] configureAuth start
I0520 15:12:25.963293 2687927 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-185009
I0520 15:12:25.981300 2687927 provision.go:143] copyHostCerts
I0520 15:12:25.981380 2687927 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem, removing ...
I0520 15:12:25.981410 2687927 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem
I0520 15:12:25.981500 2687927 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem (1679 bytes)
I0520 15:12:25.981609 2687927 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem, removing ...
I0520 15:12:25.981621 2687927 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem
I0520 15:12:25.981810 2687927 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem (1082 bytes)
I0520 15:12:25.981897 2687927 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem, removing ...
I0520 15:12:25.981908 2687927 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem
I0520 15:12:25.981938 2687927 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem (1123 bytes)
I0520 15:12:25.982011 2687927 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem org=jenkins.old-k8s-version-185009 san=[127.0.0.1 192.168.76.2 localhost minikube old-k8s-version-185009]
I0520 15:12:26.145713 2687927 provision.go:177] copyRemoteCerts
I0520 15:12:26.145811 2687927 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0520 15:12:26.145874 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:26.164612 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:26.263071 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0520 15:12:26.289177 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem --> /etc/docker/server.pem (1233 bytes)
I0520 15:12:26.314729 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1675 bytes)
I0520 15:12:26.340513 2687927 provision.go:87] duration metric: took 377.287818ms to configureAuth
I0520 15:12:26.340539 2687927 ubuntu.go:193] setting minikube options for container-runtime
I0520 15:12:26.340743 2687927 config.go:182] Loaded profile config "old-k8s-version-185009": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0520 15:12:26.340811 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:26.358149 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:26.358407 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:26.358426 2687927 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0520 15:12:26.487383 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0520 15:12:26.487413 2687927 ubuntu.go:71] root file system type: overlay
I0520 15:12:26.487525 2687927 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0520 15:12:26.487608 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:26.506065 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:26.506335 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:26.506425 2687927 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0520 15:12:26.664783 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0520 15:12:26.664899 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:26.681703 2687927 main.go:141] libmachine: Using SSH client type: native
I0520 15:12:26.681966 2687927 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41933 <nil> <nil>}
I0520 15:12:26.681991 2687927 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0520 15:12:26.817149 2687927 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0520 15:12:26.817242 2687927 machine.go:97] duration metric: took 4.332997775s to provisionDockerMachine
I0520 15:12:26.817272 2687927 start.go:293] postStartSetup for "old-k8s-version-185009" (driver="docker")
I0520 15:12:26.817319 2687927 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0520 15:12:26.817441 2687927 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0520 15:12:26.817530 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:26.842675 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:26.935676 2687927 ssh_runner.go:195] Run: cat /etc/os-release
I0520 15:12:26.939519 2687927 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0520 15:12:26.939551 2687927 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0520 15:12:26.939560 2687927 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0520 15:12:26.939576 2687927 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0520 15:12:26.939588 2687927 filesync.go:126] Scanning /home/jenkins/minikube-integration/18929-2395474/.minikube/addons for local assets ...
I0520 15:12:26.939649 2687927 filesync.go:126] Scanning /home/jenkins/minikube-integration/18929-2395474/.minikube/files for local assets ...
I0520 15:12:26.939754 2687927 filesync.go:149] local asset: /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem -> 24009082.pem in /etc/ssl/certs
I0520 15:12:26.939860 2687927 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0520 15:12:26.949305 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem --> /etc/ssl/certs/24009082.pem (1708 bytes)
I0520 15:12:26.979436 2687927 start.go:296] duration metric: took 162.131691ms for postStartSetup
I0520 15:12:26.979535 2687927 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0520 15:12:26.979606 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:27.005106 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:27.095483 2687927 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0520 15:12:27.102700 2687927 fix.go:56] duration metric: took 5.116072033s for fixHost
I0520 15:12:27.102724 2687927 start.go:83] releasing machines lock for "old-k8s-version-185009", held for 5.116122288s
I0520 15:12:27.102803 2687927 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" old-k8s-version-185009
I0520 15:12:27.127787 2687927 ssh_runner.go:195] Run: cat /version.json
I0520 15:12:27.127850 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:27.127859 2687927 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0520 15:12:27.127919 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:27.152542 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:27.161862 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:27.249536 2687927 ssh_runner.go:195] Run: systemctl --version
I0520 15:12:27.411476 2687927 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0520 15:12:27.416783 2687927 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0520 15:12:27.454020 2687927 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0520 15:12:27.454115 2687927 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *bridge* -not -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e '/"dst": ".*:.*"/d' -e 's|^(.*)"dst": (.*)[,*]$|\1"dst": \2|g' -e '/"subnet": ".*:.*"/d' -e 's|^(.*)"subnet": ".*"(.*)[,*]$|\1"subnet": "10.244.0.0/16"\2|g' {}" ;
I0520 15:12:27.475230 2687927 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *podman* -not -name *.mk_disabled -printf "%p, " -exec sh -c "sudo sed -i -r -e 's|^(.*)"subnet": ".*"(.*)$|\1"subnet": "10.244.0.0/16"\2|g' -e 's|^(.*)"gateway": ".*"(.*)$|\1"gateway": "10.244.0.1"\2|g' {}" ;
I0520 15:12:27.498549 2687927 cni.go:308] configured [/etc/cni/net.d/100-crio-bridge.conf, /etc/cni/net.d/87-podman-bridge.conflist] bridge cni config(s)
I0520 15:12:27.498632 2687927 start.go:494] detecting cgroup driver to use...
I0520 15:12:27.498675 2687927 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0520 15:12:27.498828 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0520 15:12:27.517229 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.2"|' /etc/containerd/config.toml"
I0520 15:12:27.528641 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0520 15:12:27.543253 2687927 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0520 15:12:27.543389 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0520 15:12:27.556935 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 15:12:27.569273 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0520 15:12:27.583006 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 15:12:27.597355 2687927 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0520 15:12:27.610146 2687927 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0520 15:12:27.621478 2687927 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0520 15:12:27.634398 2687927 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0520 15:12:27.645798 2687927 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:12:27.741560 2687927 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0520 15:12:27.859079 2687927 start.go:494] detecting cgroup driver to use...
I0520 15:12:27.859134 2687927 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0520 15:12:27.859191 2687927 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0520 15:12:27.873619 2687927 cruntime.go:279] skipping containerd shutdown because we are bound to it
I0520 15:12:27.873709 2687927 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0520 15:12:27.903824 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %s "runtime-endpoint: unix:///var/run/dockershim.sock
" | sudo tee /etc/crictl.yaml"
I0520 15:12:27.929048 2687927 ssh_runner.go:195] Run: which cri-dockerd
I0520 15:12:27.933266 2687927 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0520 15:12:27.945405 2687927 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0520 15:12:27.976872 2687927 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0520 15:12:28.141378 2687927 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0520 15:12:28.268216 2687927 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0520 15:12:28.268463 2687927 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0520 15:12:28.294129 2687927 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:12:28.402751 2687927 ssh_runner.go:195] Run: sudo systemctl restart docker
I0520 15:12:28.876008 2687927 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0520 15:12:28.898608 2687927 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0520 15:12:28.923085 2687927 out.go:204] * Preparing Kubernetes v1.20.0 on Docker 26.1.2 ...
I0520 15:12:28.923213 2687927 cli_runner.go:164] Run: docker network inspect old-k8s-version-185009 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0520 15:12:28.937194 2687927 ssh_runner.go:195] Run: grep 192.168.76.1 host.minikube.internal$ /etc/hosts
I0520 15:12:28.941180 2687927 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.76.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0520 15:12:28.952574 2687927 kubeadm.go:877] updating cluster {Name:old-k8s-version-185009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-185009 Namespace:default APIServerHAVIP: APIServerName:minik
ubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkin
s:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0520 15:12:28.952714 2687927 preload.go:132] Checking if preload exists for k8s version v1.20.0 and runtime docker
I0520 15:12:28.952774 2687927 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0520 15:12:28.972917 2687927 docker.go:685] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.0
registry.k8s.io/kube-proxy:v1.20.0
k8s.gcr.io/kube-controller-manager:v1.20.0
registry.k8s.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-apiserver:v1.20.0
registry.k8s.io/kube-apiserver:v1.20.0
k8s.gcr.io/kube-scheduler:v1.20.0
registry.k8s.io/kube-scheduler:v1.20.0
k8s.gcr.io/etcd:3.4.13-0
registry.k8s.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
registry.k8s.io/coredns:1.7.0
k8s.gcr.io/pause:3.2
registry.k8s.io/pause:3.2
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0520 15:12:28.972948 2687927 docker.go:615] Images already preloaded, skipping extraction
I0520 15:12:28.973027 2687927 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0520 15:12:28.990323 2687927 docker.go:685] Got preloaded images: -- stdout --
gcr.io/k8s-minikube/storage-provisioner:v5
k8s.gcr.io/kube-proxy:v1.20.0
registry.k8s.io/kube-proxy:v1.20.0
registry.k8s.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-controller-manager:v1.20.0
k8s.gcr.io/kube-apiserver:v1.20.0
registry.k8s.io/kube-apiserver:v1.20.0
k8s.gcr.io/kube-scheduler:v1.20.0
registry.k8s.io/kube-scheduler:v1.20.0
k8s.gcr.io/etcd:3.4.13-0
registry.k8s.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
registry.k8s.io/coredns:1.7.0
k8s.gcr.io/pause:3.2
registry.k8s.io/pause:3.2
gcr.io/k8s-minikube/busybox:1.28.4-glibc
-- /stdout --
I0520 15:12:28.990345 2687927 cache_images.go:84] Images are preloaded, skipping loading
I0520 15:12:28.990355 2687927 kubeadm.go:928] updating node { 192.168.76.2 8443 v1.20.0 docker true true} ...
I0520 15:12:28.990467 2687927 kubeadm.go:940] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.20.0/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=docker --hostname-override=old-k8s-version-185009 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.76.2
[Install]
config:
{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-185009 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0520 15:12:28.990539 2687927 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0520 15:12:29.063431 2687927 cni.go:84] Creating CNI manager for ""
I0520 15:12:29.063473 2687927 cni.go:162] CNI unnecessary in this configuration, recommending no CNI
I0520 15:12:29.063482 2687927 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0520 15:12:29.063499 2687927 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.76.2 APIServerPort:8443 KubernetesVersion:v1.20.0 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:old-k8s-version-185009 NodeName:old-k8s-version-185009 DNSDomain:cluster.local CRISocket:/var/run/dockershim.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.76.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.76.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticP
odPath:/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:false}
I0520 15:12:29.063653 2687927 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.76.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: "old-k8s-version-185009"
kubeletExtraArgs:
node-ip: 192.168.76.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.76.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0520 15:12:29.063734 2687927 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.20.0
I0520 15:12:29.076896 2687927 binaries.go:44] Found k8s binaries, skipping transfer
I0520 15:12:29.076979 2687927 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0520 15:12:29.090006 2687927 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (348 bytes)
I0520 15:12:29.116040 2687927 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0520 15:12:29.139646 2687927 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2118 bytes)
I0520 15:12:29.159171 2687927 ssh_runner.go:195] Run: grep 192.168.76.2 control-plane.minikube.internal$ /etc/hosts
I0520 15:12:29.162767 2687927 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.76.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0520 15:12:29.174069 2687927 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:12:29.263641 2687927 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0520 15:12:29.283926 2687927 certs.go:68] Setting up /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009 for IP: 192.168.76.2
I0520 15:12:29.283950 2687927 certs.go:194] generating shared ca certs ...
I0520 15:12:29.283966 2687927 certs.go:226] acquiring lock for ca certs: {Name:mkb3ccf41a209540ed3793a4984e70a3a08868cc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:12:29.284159 2687927 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.key
I0520 15:12:29.284241 2687927 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.key
I0520 15:12:29.284253 2687927 certs.go:256] generating profile certs ...
I0520 15:12:29.284365 2687927 certs.go:359] skipping valid signed profile cert regeneration for "minikube-user": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/client.key
I0520 15:12:29.284494 2687927 certs.go:359] skipping valid signed profile cert regeneration for "minikube": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/apiserver.key.41c876e9
I0520 15:12:29.284586 2687927 certs.go:359] skipping valid signed profile cert regeneration for "aggregator": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/proxy-client.key
I0520 15:12:29.284750 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908.pem (1338 bytes)
W0520 15:12:29.284805 2687927 certs.go:480] ignoring /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908_empty.pem, impossibly tiny 0 bytes
I0520 15:12:29.284821 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem (1675 bytes)
I0520 15:12:29.284849 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem (1082 bytes)
I0520 15:12:29.284902 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem (1123 bytes)
I0520 15:12:29.284932 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem (1679 bytes)
I0520 15:12:29.284993 2687927 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem (1708 bytes)
I0520 15:12:29.285727 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0520 15:12:29.314783 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0520 15:12:29.344061 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0520 15:12:29.373139 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0520 15:12:29.407716 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1432 bytes)
I0520 15:12:29.448974 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1675 bytes)
I0520 15:12:29.480128 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0520 15:12:29.520602 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/old-k8s-version-185009/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0520 15:12:29.591939 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908.pem --> /usr/share/ca-certificates/2400908.pem (1338 bytes)
I0520 15:12:29.644122 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem --> /usr/share/ca-certificates/24009082.pem (1708 bytes)
I0520 15:12:29.672621 2687927 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0520 15:12:29.698616 2687927 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0520 15:12:29.718697 2687927 ssh_runner.go:195] Run: openssl version
I0520 15:12:29.725734 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2400908.pem && ln -fs /usr/share/ca-certificates/2400908.pem /etc/ssl/certs/2400908.pem"
I0520 15:12:29.736342 2687927 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2400908.pem
I0520 15:12:29.740418 2687927 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 May 20 14:24 /usr/share/ca-certificates/2400908.pem
I0520 15:12:29.740502 2687927 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2400908.pem
I0520 15:12:29.747874 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2400908.pem /etc/ssl/certs/51391683.0"
I0520 15:12:29.757586 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/24009082.pem && ln -fs /usr/share/ca-certificates/24009082.pem /etc/ssl/certs/24009082.pem"
I0520 15:12:29.767230 2687927 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/24009082.pem
I0520 15:12:29.770709 2687927 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 May 20 14:24 /usr/share/ca-certificates/24009082.pem
I0520 15:12:29.770792 2687927 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/24009082.pem
I0520 15:12:29.777808 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/24009082.pem /etc/ssl/certs/3ec20f2e.0"
I0520 15:12:29.786953 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0520 15:12:29.796572 2687927 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0520 15:12:29.800070 2687927 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 May 20 14:18 /usr/share/ca-certificates/minikubeCA.pem
I0520 15:12:29.800182 2687927 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0520 15:12:29.807443 2687927 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0520 15:12:29.816865 2687927 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0520 15:12:29.820507 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-etcd-client.crt -checkend 86400
I0520 15:12:29.827414 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/apiserver-kubelet-client.crt -checkend 86400
I0520 15:12:29.834630 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/server.crt -checkend 86400
I0520 15:12:29.841670 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/healthcheck-client.crt -checkend 86400
I0520 15:12:29.848456 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/etcd/peer.crt -checkend 86400
I0520 15:12:29.855682 2687927 ssh_runner.go:195] Run: openssl x509 -noout -in /var/lib/minikube/certs/front-proxy-client.crt -checkend 86400
I0520 15:12:29.862700 2687927 kubeadm.go:391] StartCluster: {Name:old-k8s-version-185009 KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:true NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.0 ClusterName:old-k8s-version-185009 Namespace:default APIServerHAVIP: APIServerName:minikube
CA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[dashboard:true default-storageclass:true metrics-server:true storage-provisioner:true] CustomAddonImages:map[MetricsScraper:registry.k8s.io/echoserver:1.4 MetricsServer:registry.k8s.io/echoserver:1.4] CustomAddonRegistries:map[MetricsServer:fake.domain] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/
minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0520 15:12:29.862876 2687927 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0520 15:12:29.880381 2687927 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
W0520 15:12:29.889784 2687927 kubeadm.go:404] apiserver tunnel failed: apiserver port not set
I0520 15:12:29.889847 2687927 kubeadm.go:407] found existing configuration files, will attempt cluster restart
I0520 15:12:29.889876 2687927 kubeadm.go:587] restartPrimaryControlPlane start ...
I0520 15:12:29.889947 2687927 ssh_runner.go:195] Run: sudo test -d /data/minikube
I0520 15:12:29.899155 2687927 kubeadm.go:129] /data/minikube skipping compat symlinks: sudo test -d /data/minikube: Process exited with status 1
stdout:
stderr:
I0520 15:12:29.899637 2687927 kubeconfig.go:47] verify endpoint returned: get endpoint: "old-k8s-version-185009" does not appear in /home/jenkins/minikube-integration/18929-2395474/kubeconfig
I0520 15:12:29.899785 2687927 kubeconfig.go:62] /home/jenkins/minikube-integration/18929-2395474/kubeconfig needs updating (will repair): [kubeconfig missing "old-k8s-version-185009" cluster setting kubeconfig missing "old-k8s-version-185009" context setting]
I0520 15:12:29.900106 2687927 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/kubeconfig: {Name:mkc0c2fdd3c0dd0a3e30fb2ecc4be30e877e5c13 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:12:29.901370 2687927 ssh_runner.go:195] Run: sudo diff -u /var/tmp/minikube/kubeadm.yaml /var/tmp/minikube/kubeadm.yaml.new
I0520 15:12:29.910593 2687927 kubeadm.go:624] The running cluster does not require reconfiguration: 192.168.76.2
I0520 15:12:29.910672 2687927 kubeadm.go:591] duration metric: took 20.776352ms to restartPrimaryControlPlane
I0520 15:12:29.910701 2687927 kubeadm.go:393] duration metric: took 48.011258ms to StartCluster
I0520 15:12:29.910740 2687927 settings.go:142] acquiring lock: {Name:mkab8535b4b7ab061e854bdfa7125db1dcae808a Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:12:29.910820 2687927 settings.go:150] Updating kubeconfig: /home/jenkins/minikube-integration/18929-2395474/kubeconfig
I0520 15:12:29.911468 2687927 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/kubeconfig: {Name:mkc0c2fdd3c0dd0a3e30fb2ecc4be30e877e5c13 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:12:29.911714 2687927 start.go:234] Will wait 6m0s for node &{Name: IP:192.168.76.2 Port:8443 KubernetesVersion:v1.20.0 ContainerRuntime:docker ControlPlane:true Worker:true}
I0520 15:12:29.915267 2687927 out.go:177] * Verifying Kubernetes components...
I0520 15:12:29.912022 2687927 config.go:182] Loaded profile config "old-k8s-version-185009": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0520 15:12:29.912034 2687927 addons.go:502] enable addons start: toEnable=map[ambassador:false auto-pause:false cloud-spanner:false csi-hostpath-driver:false dashboard:true default-storageclass:true efk:false freshpod:false gcp-auth:false gvisor:false headlamp:false helm-tiller:false inaccel:false ingress:false ingress-dns:false inspektor-gadget:false istio:false istio-provisioner:false kong:false kubeflow:false kubevirt:false logviewer:false metallb:false metrics-server:true nvidia-device-plugin:false nvidia-driver-installer:false nvidia-gpu-device-plugin:false olm:false pod-security-policy:false portainer:false registry:false registry-aliases:false registry-creds:false storage-provisioner:true storage-provisioner-gluster:false storage-provisioner-rancher:false volumesnapshots:false yakd:false]
I0520 15:12:29.915447 2687927 addons.go:69] Setting storage-provisioner=true in profile "old-k8s-version-185009"
I0520 15:12:29.915482 2687927 addons.go:234] Setting addon storage-provisioner=true in "old-k8s-version-185009"
W0520 15:12:29.915492 2687927 addons.go:243] addon storage-provisioner should already be in state true
I0520 15:12:29.915526 2687927 host.go:66] Checking if "old-k8s-version-185009" exists ...
I0520 15:12:29.915963 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:29.916100 2687927 addons.go:69] Setting dashboard=true in profile "old-k8s-version-185009"
I0520 15:12:29.916125 2687927 addons.go:234] Setting addon dashboard=true in "old-k8s-version-185009"
W0520 15:12:29.916132 2687927 addons.go:243] addon dashboard should already be in state true
I0520 15:12:29.916153 2687927 host.go:66] Checking if "old-k8s-version-185009" exists ...
I0520 15:12:29.916499 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:29.916886 2687927 addons.go:69] Setting default-storageclass=true in profile "old-k8s-version-185009"
I0520 15:12:29.916945 2687927 addons_storage_classes.go:33] enableOrDisableStorageClasses default-storageclass=true on "old-k8s-version-185009"
I0520 15:12:29.920633 2687927 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:12:29.917134 2687927 addons.go:69] Setting metrics-server=true in profile "old-k8s-version-185009"
I0520 15:12:29.917393 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:29.921205 2687927 addons.go:234] Setting addon metrics-server=true in "old-k8s-version-185009"
W0520 15:12:29.921222 2687927 addons.go:243] addon metrics-server should already be in state true
I0520 15:12:29.921257 2687927 host.go:66] Checking if "old-k8s-version-185009" exists ...
I0520 15:12:29.922039 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:29.966028 2687927 out.go:177] - Using image gcr.io/k8s-minikube/storage-provisioner:v5
I0520 15:12:29.968245 2687927 out.go:177] - Using image docker.io/kubernetesui/dashboard:v2.7.0
I0520 15:12:29.972765 2687927 out.go:177] - Using image registry.k8s.io/echoserver:1.4
I0520 15:12:29.974816 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-ns.yaml
I0520 15:12:29.974847 2687927 ssh_runner.go:362] scp dashboard/dashboard-ns.yaml --> /etc/kubernetes/addons/dashboard-ns.yaml (759 bytes)
I0520 15:12:29.974923 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:29.968364 2687927 addons.go:426] installing /etc/kubernetes/addons/storage-provisioner.yaml
I0520 15:12:29.976848 2687927 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/storage-provisioner.yaml (2676 bytes)
I0520 15:12:29.976927 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:29.985777 2687927 addons.go:234] Setting addon default-storageclass=true in "old-k8s-version-185009"
W0520 15:12:29.985798 2687927 addons.go:243] addon default-storageclass should already be in state true
I0520 15:12:29.985826 2687927 host.go:66] Checking if "old-k8s-version-185009" exists ...
I0520 15:12:29.986238 2687927 cli_runner.go:164] Run: docker container inspect old-k8s-version-185009 --format={{.State.Status}}
I0520 15:12:29.990372 2687927 out.go:177] - Using image fake.domain/registry.k8s.io/echoserver:1.4
I0520 15:12:29.991996 2687927 addons.go:426] installing /etc/kubernetes/addons/metrics-apiservice.yaml
I0520 15:12:29.992025 2687927 ssh_runner.go:362] scp metrics-server/metrics-apiservice.yaml --> /etc/kubernetes/addons/metrics-apiservice.yaml (424 bytes)
I0520 15:12:29.992092 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:30.060863 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:30.067382 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:30.077691 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:30.087477 2687927 addons.go:426] installing /etc/kubernetes/addons/storageclass.yaml
I0520 15:12:30.087503 2687927 ssh_runner.go:362] scp storageclass/storageclass.yaml --> /etc/kubernetes/addons/storageclass.yaml (271 bytes)
I0520 15:12:30.087583 2687927 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" old-k8s-version-185009
I0520 15:12:30.114727 2687927 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41933 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/old-k8s-version-185009/id_rsa Username:docker}
I0520 15:12:30.139067 2687927 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0520 15:12:30.204869 2687927 node_ready.go:35] waiting up to 6m0s for node "old-k8s-version-185009" to be "Ready" ...
I0520 15:12:30.255170 2687927 addons.go:426] installing /etc/kubernetes/addons/metrics-server-deployment.yaml
I0520 15:12:30.255195 2687927 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/metrics-server-deployment.yaml (1825 bytes)
I0520 15:12:30.281178 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml
I0520 15:12:30.282170 2687927 addons.go:426] installing /etc/kubernetes/addons/metrics-server-rbac.yaml
I0520 15:12:30.282191 2687927 ssh_runner.go:362] scp metrics-server/metrics-server-rbac.yaml --> /etc/kubernetes/addons/metrics-server-rbac.yaml (2175 bytes)
I0520 15:12:30.318130 2687927 addons.go:426] installing /etc/kubernetes/addons/metrics-server-service.yaml
I0520 15:12:30.318156 2687927 ssh_runner.go:362] scp metrics-server/metrics-server-service.yaml --> /etc/kubernetes/addons/metrics-server-service.yaml (446 bytes)
I0520 15:12:30.324355 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrole.yaml
I0520 15:12:30.324381 2687927 ssh_runner.go:362] scp dashboard/dashboard-clusterrole.yaml --> /etc/kubernetes/addons/dashboard-clusterrole.yaml (1001 bytes)
I0520 15:12:30.326702 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml
I0520 15:12:30.365587 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0520 15:12:30.384916 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml
I0520 15:12:30.384993 2687927 ssh_runner.go:362] scp dashboard/dashboard-clusterrolebinding.yaml --> /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml (1018 bytes)
I0520 15:12:30.478647 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-configmap.yaml
I0520 15:12:30.478721 2687927 ssh_runner.go:362] scp dashboard/dashboard-configmap.yaml --> /etc/kubernetes/addons/dashboard-configmap.yaml (837 bytes)
W0520 15:12:30.483425 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.483511 2687927 retry.go:31] will retry after 232.135555ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.507538 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-dp.yaml
I0520 15:12:30.507566 2687927 ssh_runner.go:362] scp memory --> /etc/kubernetes/addons/dashboard-dp.yaml (4201 bytes)
W0520 15:12:30.525515 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.525548 2687927 retry.go:31] will retry after 144.173184ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.538485 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-role.yaml
I0520 15:12:30.538514 2687927 ssh_runner.go:362] scp dashboard/dashboard-role.yaml --> /etc/kubernetes/addons/dashboard-role.yaml (1724 bytes)
W0520 15:12:30.578733 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.578766 2687927 retry.go:31] will retry after 359.071965ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.591056 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-rolebinding.yaml
I0520 15:12:30.591086 2687927 ssh_runner.go:362] scp dashboard/dashboard-rolebinding.yaml --> /etc/kubernetes/addons/dashboard-rolebinding.yaml (1046 bytes)
I0520 15:12:30.611785 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-sa.yaml
I0520 15:12:30.611810 2687927 ssh_runner.go:362] scp dashboard/dashboard-sa.yaml --> /etc/kubernetes/addons/dashboard-sa.yaml (837 bytes)
I0520 15:12:30.635303 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-secret.yaml
I0520 15:12:30.635330 2687927 ssh_runner.go:362] scp dashboard/dashboard-secret.yaml --> /etc/kubernetes/addons/dashboard-secret.yaml (1389 bytes)
I0520 15:12:30.658618 2687927 addons.go:426] installing /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:30.658642 2687927 ssh_runner.go:362] scp dashboard/dashboard-svc.yaml --> /etc/kubernetes/addons/dashboard-svc.yaml (1294 bytes)
I0520 15:12:30.669860 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0520 15:12:30.678726 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:30.716165 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0520 15:12:30.785173 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.785205 2687927 retry.go:31] will retry after 311.61008ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:30.789078 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.789113 2687927 retry.go:31] will retry after 260.852483ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:30.843684 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.843770 2687927 retry.go:31] will retry after 496.839817ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:30.937972 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0520 15:12:31.018079 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.018121 2687927 retry.go:31] will retry after 340.643865ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.050264 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:31.097688 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0520 15:12:31.177416 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.177461 2687927 retry.go:31] will retry after 559.631475ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:31.214175 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.214208 2687927 retry.go:31] will retry after 631.445522ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.341455 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0520 15:12:31.359832 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0520 15:12:31.458831 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.458920 2687927 retry.go:31] will retry after 661.741384ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:31.481231 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.481313 2687927 retry.go:31] will retry after 652.560189ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.737788 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
W0520 15:12:31.816051 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.816123 2687927 retry.go:31] will retry after 527.754965ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.846299 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0520 15:12:31.924299 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:31.924329 2687927 retry.go:31] will retry after 424.083248ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.121695 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0520 15:12:32.135010 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0520 15:12:32.205894 2687927 node_ready.go:53] error getting node "old-k8s-version-185009": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-185009": dial tcp 192.168.76.2:8443: connect: connection refused
W0520 15:12:32.228073 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.228157 2687927 retry.go:31] will retry after 1.010674814s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:32.246063 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.246099 2687927 retry.go:31] will retry after 1.199146352s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.344359 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:32.348948 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
W0520 15:12:32.449926 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.450005 2687927 retry.go:31] will retry after 706.380112ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:32.466549 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:32.466583 2687927 retry.go:31] will retry after 1.572198082s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:33.156612 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:33.239921 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
W0520 15:12:33.356217 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:33.356253 2687927 retry.go:31] will retry after 1.507162811s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
W0520 15:12:33.384090 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:33.384124 2687927 retry.go:31] will retry after 743.675718ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:33.446246 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
W0520 15:12:33.688642 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:33.688727 2687927 retry.go:31] will retry after 729.22517ms: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:34.039783 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0520 15:12:34.128230 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml
I0520 15:12:34.206287 2687927 node_ready.go:53] error getting node "old-k8s-version-185009": Get "https://192.168.76.2:8443/api/v1/nodes/old-k8s-version-185009": dial tcp 192.168.76.2:8443: connect: connection refused
W0520 15:12:34.357546 2687927 addons.go:452] apply failed, will retry: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:34.357575 2687927 retry.go:31] will retry after 2.632971404s: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: Process exited with status 1
stdout:
stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
I0520 15:12:34.418814 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml
I0520 15:12:34.867921 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml
I0520 15:12:36.991416 2687927 ssh_runner.go:195] Run: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml
I0520 15:12:45.112920 2687927 node_ready.go:49] node "old-k8s-version-185009" has status "Ready":"True"
I0520 15:12:45.112957 2687927 node_ready.go:38] duration metric: took 14.908038182s for node "old-k8s-version-185009" to be "Ready" ...
I0520 15:12:45.112973 2687927 pod_ready.go:35] extra waiting up to 6m0s for all system-critical pods including labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0520 15:12:45.368512 2687927 pod_ready.go:78] waiting up to 6m0s for pod "coredns-74ff55c5b-frwjt" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.594541 2687927 pod_ready.go:92] pod "coredns-74ff55c5b-frwjt" in "kube-system" namespace has status "Ready":"True"
I0520 15:12:45.594623 2687927 pod_ready.go:81] duration metric: took 225.997816ms for pod "coredns-74ff55c5b-frwjt" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.594665 2687927 pod_ready.go:78] waiting up to 6m0s for pod "etcd-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.690380 2687927 pod_ready.go:92] pod "etcd-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"True"
I0520 15:12:45.690452 2687927 pod_ready.go:81] duration metric: took 95.746393ms for pod "etcd-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.690495 2687927 pod_ready.go:78] waiting up to 6m0s for pod "kube-apiserver-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.723538 2687927 pod_ready.go:92] pod "kube-apiserver-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"True"
I0520 15:12:45.723637 2687927 pod_ready.go:81] duration metric: took 33.092251ms for pod "kube-apiserver-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:12:45.723670 2687927 pod_ready.go:78] waiting up to 6m0s for pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:12:46.975195 2687927 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storage-provisioner.yaml: (12.846924286s)
I0520 15:12:47.032191 2687927 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/metrics-apiservice.yaml -f /etc/kubernetes/addons/metrics-server-deployment.yaml -f /etc/kubernetes/addons/metrics-server-rbac.yaml -f /etc/kubernetes/addons/metrics-server-service.yaml: (12.613337618s)
I0520 15:12:47.032282 2687927 addons.go:470] Verifying addon metrics-server=true in "old-k8s-version-185009"
I0520 15:12:47.113234 2687927 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: (12.245266581s)
I0520 15:12:47.115121 2687927 out.go:177] * Some dashboard features require the metrics-server addon. To enable all features please run:
minikube -p old-k8s-version-185009 addons enable metrics-server
I0520 15:12:47.113327 2687927 ssh_runner.go:235] Completed: sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.20.0/kubectl apply --force -f /etc/kubernetes/addons/storageclass.yaml: (10.121885378s)
I0520 15:12:47.127864 2687927 out.go:177] * Enabled addons: storage-provisioner, metrics-server, dashboard, default-storageclass
I0520 15:12:47.129966 2687927 addons.go:505] duration metric: took 17.217923106s for enable addons: enabled=[storage-provisioner metrics-server dashboard default-storageclass]
I0520 15:12:47.729225 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:12:49.729413 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:12:51.730309 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:12:53.730437 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:12:55.730966 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:12:57.731060 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:00.300772 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:02.732119 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:04.735248 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:06.740494 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:09.233945 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:11.237582 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:13.731661 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:16.230802 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:18.731177 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:20.731608 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:23.230316 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:25.234930 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:27.729950 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:29.730946 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:31.732330 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:34.255200 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:36.730717 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:38.734043 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:41.230606 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:43.232179 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:45.728870 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:47.729876 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:49.730475 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:51.730852 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:54.230258 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:56.730384 2687927 pod_ready.go:102] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:13:59.230711 2687927 pod_ready.go:92] pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"True"
I0520 15:13:59.230738 2687927 pod_ready.go:81] duration metric: took 1m13.507013013s for pod "kube-controller-manager-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:13:59.230750 2687927 pod_ready.go:78] waiting up to 6m0s for pod "kube-proxy-p9rc9" in "kube-system" namespace to be "Ready" ...
I0520 15:13:59.236957 2687927 pod_ready.go:92] pod "kube-proxy-p9rc9" in "kube-system" namespace has status "Ready":"True"
I0520 15:13:59.236988 2687927 pod_ready.go:81] duration metric: took 6.228733ms for pod "kube-proxy-p9rc9" in "kube-system" namespace to be "Ready" ...
I0520 15:13:59.237000 2687927 pod_ready.go:78] waiting up to 6m0s for pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:14:01.251978 2687927 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:03.744251 2687927 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:05.747369 2687927 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:08.245174 2687927 pod_ready.go:102] pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:10.249265 2687927 pod_ready.go:92] pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace has status "Ready":"True"
I0520 15:14:10.249302 2687927 pod_ready.go:81] duration metric: took 11.012293244s for pod "kube-scheduler-old-k8s-version-185009" in "kube-system" namespace to be "Ready" ...
I0520 15:14:10.249331 2687927 pod_ready.go:78] waiting up to 6m0s for pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace to be "Ready" ...
I0520 15:14:12.256054 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:14.256162 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:16.756366 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:18.756657 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:21.256192 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:23.755988 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:25.756743 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:28.255703 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:30.755238 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:32.755860 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:35.255498 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:37.755555 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:40.255751 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:42.256679 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:44.755835 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:46.757542 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:49.255257 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:51.757413 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:54.255319 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:56.255927 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:14:58.755689 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:00.757484 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:02.759364 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:05.255986 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:07.755599 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:09.755652 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:11.756013 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:13.756715 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:16.255979 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:18.256141 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:20.755646 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:22.756064 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:24.756437 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:27.255665 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:29.256652 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:31.756480 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:34.255419 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:36.256138 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:38.755894 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:41.256199 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:43.755538 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:46.275141 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:48.756561 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:51.257601 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:53.755367 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:55.756477 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:15:57.756525 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:00.273600 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:02.755162 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:04.758678 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:07.256925 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:09.755675 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:11.755757 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:13.755826 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:16.256412 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:18.755617 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:20.755890 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:22.756834 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:25.255187 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:27.256016 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:29.755809 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:31.756265 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:33.756681 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:36.255925 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:38.756045 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:41.256077 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:43.256159 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:45.259663 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:47.755757 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:50.256588 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:52.755855 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:54.756096 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:56.756203 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:16:58.757145 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:01.288977 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:03.757226 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:05.758746 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:08.257064 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:10.755953 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:12.756264 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:15.255875 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:17.756088 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:20.263365 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:22.756480 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:24.759925 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:27.256522 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:29.756124 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:32.255005 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:34.256596 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:36.755943 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:38.756383 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:41.255888 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:43.260533 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:45.766730 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:48.259944 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:50.266967 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:52.757742 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:54.762524 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:57.256085 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:59.256773 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:01.257463 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:03.757342 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:06.257208 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:08.258452 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:10.256495 2687927 pod_ready.go:81] duration metric: took 4m0.007137082s for pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace to be "Ready" ...
E0520 15:18:10.256524 2687927 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0520 15:18:10.256535 2687927 pod_ready.go:38] duration metric: took 5m25.143545959s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0520 15:18:10.256554 2687927 api_server.go:52] waiting for apiserver process to appear ...
I0520 15:18:10.256637 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0520 15:18:10.283268 2687927 logs.go:276] 2 containers: [7b906f9c7e42 21cda61304ec]
I0520 15:18:10.283354 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0520 15:18:10.302345 2687927 logs.go:276] 2 containers: [4183e6237564 78716dd9f5e6]
I0520 15:18:10.302441 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0520 15:18:10.329708 2687927 logs.go:276] 2 containers: [57e027c8e0da 5876807206c9]
I0520 15:18:10.329805 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0520 15:18:10.356545 2687927 logs.go:276] 2 containers: [20ae03428247 798f34afc547]
I0520 15:18:10.356646 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0520 15:18:10.378250 2687927 logs.go:276] 2 containers: [75eb680f6808 36b9b4fb305d]
I0520 15:18:10.378352 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0520 15:18:10.406769 2687927 logs.go:276] 2 containers: [18ca6dabc240 7c325beb8903]
I0520 15:18:10.406878 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0520 15:18:10.424347 2687927 logs.go:276] 0 containers: []
W0520 15:18:10.424373 2687927 logs.go:278] No container was found matching "kindnet"
I0520 15:18:10.424462 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0520 15:18:10.446113 2687927 logs.go:276] 2 containers: [ee9aab3e1ca5 623598d9bb43]
I0520 15:18:10.446197 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0520 15:18:10.465324 2687927 logs.go:276] 1 containers: [e9d80f29a80f]
I0520 15:18:10.465359 2687927 logs.go:123] Gathering logs for kubelet ...
I0520 15:18:10.465373 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0520 15:18:10.547614 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076509 1198 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.547881 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076589 1198 reflector.go:138] object-"kube-system"/"kube-proxy-token-4znn8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-4znn8" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.548103 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076649 1198 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.548320 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076901 1198 reflector.go:138] object-"kube-system"/"coredns-token-rk862": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-rk862" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.549299 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.252268 1198 reflector.go:138] object-"default"/"default-token-kj425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-kj425" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.549534 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.263194 1198 reflector.go:138] object-"kube-system"/"storage-provisioner-token-rpfqn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-rpfqn" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.559239 2687927 logs.go:138] Found kubelet problem: May 20 15:12:46 old-k8s-version-185009 kubelet[1198]: E0520 15:12:46.633160 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.559629 2687927 logs.go:138] Found kubelet problem: May 20 15:12:47 old-k8s-version-185009 kubelet[1198]: E0520 15:12:47.647957 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.560622 2687927 logs.go:138] Found kubelet problem: May 20 15:12:48 old-k8s-version-185009 kubelet[1198]: E0520 15:12:48.926924 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.563115 2687927 logs.go:138] Found kubelet problem: May 20 15:13:00 old-k8s-version-185009 kubelet[1198]: E0520 15:13:00.334303 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.567886 2687927 logs.go:138] Found kubelet problem: May 20 15:13:05 old-k8s-version-185009 kubelet[1198]: E0520 15:13:05.302095 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.568104 2687927 logs.go:138] Found kubelet problem: May 20 15:13:06 old-k8s-version-185009 kubelet[1198]: E0520 15:13:06.152746 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.568296 2687927 logs.go:138] Found kubelet problem: May 20 15:13:11 old-k8s-version-185009 kubelet[1198]: E0520 15:13:11.129523 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.569266 2687927 logs.go:138] Found kubelet problem: May 20 15:13:18 old-k8s-version-185009 kubelet[1198]: E0520 15:13:18.383550 1198 pod_workers.go:191] Error syncing pod 55c63b6d-7e3d-4491-9a29-4bc83a4d54fa ("storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"
W0520 15:18:10.571561 2687927 logs.go:138] Found kubelet problem: May 20 15:13:19 old-k8s-version-185009 kubelet[1198]: E0520 15:13:19.635684 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.573759 2687927 logs.go:138] Found kubelet problem: May 20 15:13:23 old-k8s-version-185009 kubelet[1198]: E0520 15:13:23.158433 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.574313 2687927 logs.go:138] Found kubelet problem: May 20 15:13:32 old-k8s-version-185009 kubelet[1198]: E0520 15:13:32.119072 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.574640 2687927 logs.go:138] Found kubelet problem: May 20 15:13:35 old-k8s-version-185009 kubelet[1198]: E0520 15:13:35.124065 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577236 2687927 logs.go:138] Found kubelet problem: May 20 15:13:43 old-k8s-version-185009 kubelet[1198]: E0520 15:13:43.578964 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.577454 2687927 logs.go:138] Found kubelet problem: May 20 15:13:47 old-k8s-version-185009 kubelet[1198]: E0520 15:13:47.118959 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577738 2687927 logs.go:138] Found kubelet problem: May 20 15:13:58 old-k8s-version-185009 kubelet[1198]: E0520 15:13:58.119337 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577940 2687927 logs.go:138] Found kubelet problem: May 20 15:14:01 old-k8s-version-185009 kubelet[1198]: E0520 15:14:01.118790 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.578455 2687927 logs.go:138] Found kubelet problem: May 20 15:14:10 old-k8s-version-185009 kubelet[1198]: E0520 15:14:10.148705 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.580731 2687927 logs.go:138] Found kubelet problem: May 20 15:14:14 old-k8s-version-185009 kubelet[1198]: E0520 15:14:14.139015 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.583477 2687927 logs.go:138] Found kubelet problem: May 20 15:14:25 old-k8s-version-185009 kubelet[1198]: E0520 15:14:25.575698 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.583771 2687927 logs.go:138] Found kubelet problem: May 20 15:14:28 old-k8s-version-185009 kubelet[1198]: E0520 15:14:28.126943 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584099 2687927 logs.go:138] Found kubelet problem: May 20 15:14:38 old-k8s-version-185009 kubelet[1198]: E0520 15:14:38.118687 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584312 2687927 logs.go:138] Found kubelet problem: May 20 15:14:42 old-k8s-version-185009 kubelet[1198]: E0520 15:14:42.119209 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584520 2687927 logs.go:138] Found kubelet problem: May 20 15:14:53 old-k8s-version-185009 kubelet[1198]: E0520 15:14:53.121794 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584720 2687927 logs.go:138] Found kubelet problem: May 20 15:14:56 old-k8s-version-185009 kubelet[1198]: E0520 15:14:56.118640 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584936 2687927 logs.go:138] Found kubelet problem: May 20 15:15:08 old-k8s-version-185009 kubelet[1198]: E0520 15:15:08.118744 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585137 2687927 logs.go:138] Found kubelet problem: May 20 15:15:11 old-k8s-version-185009 kubelet[1198]: E0520 15:15:11.125539 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585333 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.119290 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585557 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.129546 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.586072 2687927 logs.go:138] Found kubelet problem: May 20 15:15:34 old-k8s-version-185009 kubelet[1198]: E0520 15:15:34.118580 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.588388 2687927 logs.go:138] Found kubelet problem: May 20 15:15:35 old-k8s-version-185009 kubelet[1198]: E0520 15:15:35.142356 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.588591 2687927 logs.go:138] Found kubelet problem: May 20 15:15:47 old-k8s-version-185009 kubelet[1198]: E0520 15:15:47.118833 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591221 2687927 logs.go:138] Found kubelet problem: May 20 15:15:49 old-k8s-version-185009 kubelet[1198]: E0520 15:15:49.579168 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.591436 2687927 logs.go:138] Found kubelet problem: May 20 15:15:58 old-k8s-version-185009 kubelet[1198]: E0520 15:15:58.119002 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591673 2687927 logs.go:138] Found kubelet problem: May 20 15:16:03 old-k8s-version-185009 kubelet[1198]: E0520 15:16:03.129847 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591871 2687927 logs.go:138] Found kubelet problem: May 20 15:16:09 old-k8s-version-185009 kubelet[1198]: E0520 15:16:09.118709 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592081 2687927 logs.go:138] Found kubelet problem: May 20 15:16:15 old-k8s-version-185009 kubelet[1198]: E0520 15:16:15.140673 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592292 2687927 logs.go:138] Found kubelet problem: May 20 15:16:22 old-k8s-version-185009 kubelet[1198]: E0520 15:16:22.118599 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592517 2687927 logs.go:138] Found kubelet problem: May 20 15:16:26 old-k8s-version-185009 kubelet[1198]: E0520 15:16:26.125485 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592724 2687927 logs.go:138] Found kubelet problem: May 20 15:16:35 old-k8s-version-185009 kubelet[1198]: E0520 15:16:35.118669 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592937 2687927 logs.go:138] Found kubelet problem: May 20 15:16:37 old-k8s-version-185009 kubelet[1198]: E0520 15:16:37.119240 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593148 2687927 logs.go:138] Found kubelet problem: May 20 15:16:49 old-k8s-version-185009 kubelet[1198]: E0520 15:16:49.118660 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593456 2687927 logs.go:138] Found kubelet problem: May 20 15:16:51 old-k8s-version-185009 kubelet[1198]: E0520 15:16:51.118889 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593781 2687927 logs.go:138] Found kubelet problem: May 20 15:17:01 old-k8s-version-185009 kubelet[1198]: E0520 15:17:01.131117 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594012 2687927 logs.go:138] Found kubelet problem: May 20 15:17:02 old-k8s-version-185009 kubelet[1198]: E0520 15:17:02.119112 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594271 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118969 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594481 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118971 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594687 2687927 logs.go:138] Found kubelet problem: May 20 15:17:25 old-k8s-version-185009 kubelet[1198]: E0520 15:17:25.119102 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594902 2687927 logs.go:138] Found kubelet problem: May 20 15:17:27 old-k8s-version-185009 kubelet[1198]: E0520 15:17:27.141000 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595111 2687927 logs.go:138] Found kubelet problem: May 20 15:17:37 old-k8s-version-185009 kubelet[1198]: E0520 15:17:37.118917 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595323 2687927 logs.go:138] Found kubelet problem: May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595526 2687927 logs.go:138] Found kubelet problem: May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595751 2687927 logs.go:138] Found kubelet problem: May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595966 2687927 logs.go:138] Found kubelet problem: May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.596194 2687927 logs.go:138] Found kubelet problem: May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:10.596208 2687927 logs.go:123] Gathering logs for kube-apiserver [7b906f9c7e42] ...
I0520 15:18:10.596221 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b906f9c7e42"
I0520 15:18:10.656241 2687927 logs.go:123] Gathering logs for kube-scheduler [798f34afc547] ...
I0520 15:18:10.656300 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 798f34afc547"
I0520 15:18:10.702047 2687927 logs.go:123] Gathering logs for kube-controller-manager [18ca6dabc240] ...
I0520 15:18:10.702088 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 18ca6dabc240"
I0520 15:18:10.778491 2687927 logs.go:123] Gathering logs for storage-provisioner [ee9aab3e1ca5] ...
I0520 15:18:10.778536 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 ee9aab3e1ca5"
I0520 15:18:10.810518 2687927 logs.go:123] Gathering logs for storage-provisioner [623598d9bb43] ...
I0520 15:18:10.810557 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 623598d9bb43"
I0520 15:18:10.842415 2687927 logs.go:123] Gathering logs for dmesg ...
I0520 15:18:10.842455 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0520 15:18:10.874520 2687927 logs.go:123] Gathering logs for etcd [4183e6237564] ...
I0520 15:18:10.874572 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4183e6237564"
I0520 15:18:10.918596 2687927 logs.go:123] Gathering logs for kubernetes-dashboard [e9d80f29a80f] ...
I0520 15:18:10.918666 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 e9d80f29a80f"
I0520 15:18:10.949484 2687927 logs.go:123] Gathering logs for kube-apiserver [21cda61304ec] ...
I0520 15:18:10.949565 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 21cda61304ec"
I0520 15:18:11.042227 2687927 logs.go:123] Gathering logs for coredns [57e027c8e0da] ...
I0520 15:18:11.042306 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 57e027c8e0da"
I0520 15:18:11.067073 2687927 logs.go:123] Gathering logs for coredns [5876807206c9] ...
I0520 15:18:11.067098 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 5876807206c9"
I0520 15:18:11.098711 2687927 logs.go:123] Gathering logs for container status ...
I0520 15:18:11.098814 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0520 15:18:11.167370 2687927 logs.go:123] Gathering logs for describe nodes ...
I0520 15:18:11.167517 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0520 15:18:11.349855 2687927 logs.go:123] Gathering logs for etcd [78716dd9f5e6] ...
I0520 15:18:11.349928 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 78716dd9f5e6"
I0520 15:18:11.395012 2687927 logs.go:123] Gathering logs for kube-scheduler [20ae03428247] ...
I0520 15:18:11.395091 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 20ae03428247"
I0520 15:18:11.436527 2687927 logs.go:123] Gathering logs for kube-proxy [75eb680f6808] ...
I0520 15:18:11.436579 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 75eb680f6808"
I0520 15:18:11.472843 2687927 logs.go:123] Gathering logs for kube-proxy [36b9b4fb305d] ...
I0520 15:18:11.472886 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 36b9b4fb305d"
I0520 15:18:11.500352 2687927 logs.go:123] Gathering logs for kube-controller-manager [7c325beb8903] ...
I0520 15:18:11.500394 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7c325beb8903"
I0520 15:18:11.565557 2687927 logs.go:123] Gathering logs for Docker ...
I0520 15:18:11.565612 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0520 15:18:11.597615 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:11.597661 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0520 15:18:11.597729 2687927 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0520 15:18:11.597744 2687927 out.go:239] May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597760 2687927 out.go:239] May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597767 2687927 out.go:239] May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597775 2687927 out.go:239] May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597929 2687927 out.go:239] May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:11.597943 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:11.597951 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:18:21.599316 2687927 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0520 15:18:21.614072 2687927 api_server.go:72] duration metric: took 5m51.702320208s to wait for apiserver process to appear ...
I0520 15:18:21.614097 2687927 api_server.go:88] waiting for apiserver healthz status ...
I0520 15:18:21.614176 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0520 15:18:21.640018 2687927 logs.go:276] 2 containers: [7b906f9c7e42 21cda61304ec]
I0520 15:18:21.640104 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0520 15:18:21.656535 2687927 logs.go:276] 2 containers: [4183e6237564 78716dd9f5e6]
I0520 15:18:21.656612 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0520 15:18:21.675372 2687927 logs.go:276] 2 containers: [57e027c8e0da 5876807206c9]
I0520 15:18:21.675454 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0520 15:18:21.694106 2687927 logs.go:276] 2 containers: [20ae03428247 798f34afc547]
I0520 15:18:21.694191 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0520 15:18:21.713309 2687927 logs.go:276] 2 containers: [75eb680f6808 36b9b4fb305d]
I0520 15:18:21.713391 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0520 15:18:21.736072 2687927 logs.go:276] 2 containers: [18ca6dabc240 7c325beb8903]
I0520 15:18:21.736157 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0520 15:18:21.751873 2687927 logs.go:276] 0 containers: []
W0520 15:18:21.751894 2687927 logs.go:278] No container was found matching "kindnet"
I0520 15:18:21.751953 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0520 15:18:21.769052 2687927 logs.go:276] 1 containers: [e9d80f29a80f]
I0520 15:18:21.769133 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0520 15:18:21.786563 2687927 logs.go:276] 2 containers: [ee9aab3e1ca5 623598d9bb43]
I0520 15:18:21.786597 2687927 logs.go:123] Gathering logs for kube-proxy [36b9b4fb305d] ...
I0520 15:18:21.786609 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 36b9b4fb305d"
I0520 15:18:21.809092 2687927 logs.go:123] Gathering logs for kube-controller-manager [7c325beb8903] ...
I0520 15:18:21.809118 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7c325beb8903"
I0520 15:18:21.860586 2687927 logs.go:123] Gathering logs for storage-provisioner [ee9aab3e1ca5] ...
I0520 15:18:21.860642 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 ee9aab3e1ca5"
I0520 15:18:21.896739 2687927 logs.go:123] Gathering logs for kubelet ...
I0520 15:18:21.896810 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0520 15:18:21.977326 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076509 1198 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.977623 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076589 1198 reflector.go:138] object-"kube-system"/"kube-proxy-token-4znn8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-4znn8" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.977905 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076649 1198 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.978156 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076901 1198 reflector.go:138] object-"kube-system"/"coredns-token-rk862": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-rk862" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.979172 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.252268 1198 reflector.go:138] object-"default"/"default-token-kj425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-kj425" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.979475 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.263194 1198 reflector.go:138] object-"kube-system"/"storage-provisioner-token-rpfqn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-rpfqn" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.988228 2687927 logs.go:138] Found kubelet problem: May 20 15:12:46 old-k8s-version-185009 kubelet[1198]: E0520 15:12:46.633160 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:21.988671 2687927 logs.go:138] Found kubelet problem: May 20 15:12:47 old-k8s-version-185009 kubelet[1198]: E0520 15:12:47.647957 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.989730 2687927 logs.go:138] Found kubelet problem: May 20 15:12:48 old-k8s-version-185009 kubelet[1198]: E0520 15:12:48.926924 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.992237 2687927 logs.go:138] Found kubelet problem: May 20 15:13:00 old-k8s-version-185009 kubelet[1198]: E0520 15:13:00.334303 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:21.997546 2687927 logs.go:138] Found kubelet problem: May 20 15:13:05 old-k8s-version-185009 kubelet[1198]: E0520 15:13:05.302095 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:21.997800 2687927 logs.go:138] Found kubelet problem: May 20 15:13:06 old-k8s-version-185009 kubelet[1198]: E0520 15:13:06.152746 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.998019 2687927 logs.go:138] Found kubelet problem: May 20 15:13:11 old-k8s-version-185009 kubelet[1198]: E0520 15:13:11.129523 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.999028 2687927 logs.go:138] Found kubelet problem: May 20 15:13:18 old-k8s-version-185009 kubelet[1198]: E0520 15:13:18.383550 1198 pod_workers.go:191] Error syncing pod 55c63b6d-7e3d-4491-9a29-4bc83a4d54fa ("storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"
W0520 15:18:22.001353 2687927 logs.go:138] Found kubelet problem: May 20 15:13:19 old-k8s-version-185009 kubelet[1198]: E0520 15:13:19.635684 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.003942 2687927 logs.go:138] Found kubelet problem: May 20 15:13:23 old-k8s-version-185009 kubelet[1198]: E0520 15:13:23.158433 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.006875 2687927 logs.go:138] Found kubelet problem: May 20 15:13:32 old-k8s-version-185009 kubelet[1198]: E0520 15:13:32.119072 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.007244 2687927 logs.go:138] Found kubelet problem: May 20 15:13:35 old-k8s-version-185009 kubelet[1198]: E0520 15:13:35.124065 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.009571 2687927 logs.go:138] Found kubelet problem: May 20 15:13:43 old-k8s-version-185009 kubelet[1198]: E0520 15:13:43.578964 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.009800 2687927 logs.go:138] Found kubelet problem: May 20 15:13:47 old-k8s-version-185009 kubelet[1198]: E0520 15:13:47.118959 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010034 2687927 logs.go:138] Found kubelet problem: May 20 15:13:58 old-k8s-version-185009 kubelet[1198]: E0520 15:13:58.119337 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010345 2687927 logs.go:138] Found kubelet problem: May 20 15:14:01 old-k8s-version-185009 kubelet[1198]: E0520 15:14:01.118790 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010635 2687927 logs.go:138] Found kubelet problem: May 20 15:14:10 old-k8s-version-185009 kubelet[1198]: E0520 15:14:10.148705 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.012895 2687927 logs.go:138] Found kubelet problem: May 20 15:14:14 old-k8s-version-185009 kubelet[1198]: E0520 15:14:14.139015 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.015404 2687927 logs.go:138] Found kubelet problem: May 20 15:14:25 old-k8s-version-185009 kubelet[1198]: E0520 15:14:25.575698 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.015647 2687927 logs.go:138] Found kubelet problem: May 20 15:14:28 old-k8s-version-185009 kubelet[1198]: E0520 15:14:28.126943 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.015879 2687927 logs.go:138] Found kubelet problem: May 20 15:14:38 old-k8s-version-185009 kubelet[1198]: E0520 15:14:38.118687 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016099 2687927 logs.go:138] Found kubelet problem: May 20 15:14:42 old-k8s-version-185009 kubelet[1198]: E0520 15:14:42.119209 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016340 2687927 logs.go:138] Found kubelet problem: May 20 15:14:53 old-k8s-version-185009 kubelet[1198]: E0520 15:14:53.121794 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016567 2687927 logs.go:138] Found kubelet problem: May 20 15:14:56 old-k8s-version-185009 kubelet[1198]: E0520 15:14:56.118640 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016803 2687927 logs.go:138] Found kubelet problem: May 20 15:15:08 old-k8s-version-185009 kubelet[1198]: E0520 15:15:08.118744 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017026 2687927 logs.go:138] Found kubelet problem: May 20 15:15:11 old-k8s-version-185009 kubelet[1198]: E0520 15:15:11.125539 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017249 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.119290 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017486 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.129546 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.019229 2687927 logs.go:138] Found kubelet problem: May 20 15:15:34 old-k8s-version-185009 kubelet[1198]: E0520 15:15:34.118580 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.021424 2687927 logs.go:138] Found kubelet problem: May 20 15:15:35 old-k8s-version-185009 kubelet[1198]: E0520 15:15:35.142356 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.021691 2687927 logs.go:138] Found kubelet problem: May 20 15:15:47 old-k8s-version-185009 kubelet[1198]: E0520 15:15:47.118833 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024033 2687927 logs.go:138] Found kubelet problem: May 20 15:15:49 old-k8s-version-185009 kubelet[1198]: E0520 15:15:49.579168 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.024251 2687927 logs.go:138] Found kubelet problem: May 20 15:15:58 old-k8s-version-185009 kubelet[1198]: E0520 15:15:58.119002 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024484 2687927 logs.go:138] Found kubelet problem: May 20 15:16:03 old-k8s-version-185009 kubelet[1198]: E0520 15:16:03.129847 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024703 2687927 logs.go:138] Found kubelet problem: May 20 15:16:09 old-k8s-version-185009 kubelet[1198]: E0520 15:16:09.118709 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024938 2687927 logs.go:138] Found kubelet problem: May 20 15:16:15 old-k8s-version-185009 kubelet[1198]: E0520 15:16:15.140673 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025166 2687927 logs.go:138] Found kubelet problem: May 20 15:16:22 old-k8s-version-185009 kubelet[1198]: E0520 15:16:22.118599 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025535 2687927 logs.go:138] Found kubelet problem: May 20 15:16:26 old-k8s-version-185009 kubelet[1198]: E0520 15:16:26.125485 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025809 2687927 logs.go:138] Found kubelet problem: May 20 15:16:35 old-k8s-version-185009 kubelet[1198]: E0520 15:16:35.118669 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026048 2687927 logs.go:138] Found kubelet problem: May 20 15:16:37 old-k8s-version-185009 kubelet[1198]: E0520 15:16:37.119240 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026311 2687927 logs.go:138] Found kubelet problem: May 20 15:16:49 old-k8s-version-185009 kubelet[1198]: E0520 15:16:49.118660 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026556 2687927 logs.go:138] Found kubelet problem: May 20 15:16:51 old-k8s-version-185009 kubelet[1198]: E0520 15:16:51.118889 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026776 2687927 logs.go:138] Found kubelet problem: May 20 15:17:01 old-k8s-version-185009 kubelet[1198]: E0520 15:17:01.131117 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027012 2687927 logs.go:138] Found kubelet problem: May 20 15:17:02 old-k8s-version-185009 kubelet[1198]: E0520 15:17:02.119112 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027240 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118969 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027488 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118971 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027714 2687927 logs.go:138] Found kubelet problem: May 20 15:17:25 old-k8s-version-185009 kubelet[1198]: E0520 15:17:25.119102 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027944 2687927 logs.go:138] Found kubelet problem: May 20 15:17:27 old-k8s-version-185009 kubelet[1198]: E0520 15:17:27.141000 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028163 2687927 logs.go:138] Found kubelet problem: May 20 15:17:37 old-k8s-version-185009 kubelet[1198]: E0520 15:17:37.118917 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028395 2687927 logs.go:138] Found kubelet problem: May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028633 2687927 logs.go:138] Found kubelet problem: May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028865 2687927 logs.go:138] Found kubelet problem: May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029088 2687927 logs.go:138] Found kubelet problem: May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029320 2687927 logs.go:138] Found kubelet problem: May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029540 2687927 logs.go:138] Found kubelet problem: May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029784 2687927 logs.go:138] Found kubelet problem: May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:22.029813 2687927 logs.go:123] Gathering logs for dmesg ...
I0520 15:18:22.029846 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0520 15:18:22.053942 2687927 logs.go:123] Gathering logs for describe nodes ...
I0520 15:18:22.054014 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0520 15:18:22.227606 2687927 logs.go:123] Gathering logs for coredns [57e027c8e0da] ...
I0520 15:18:22.227642 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 57e027c8e0da"
I0520 15:18:22.251970 2687927 logs.go:123] Gathering logs for coredns [5876807206c9] ...
I0520 15:18:22.251999 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 5876807206c9"
I0520 15:18:22.282757 2687927 logs.go:123] Gathering logs for storage-provisioner [623598d9bb43] ...
I0520 15:18:22.282789 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 623598d9bb43"
I0520 15:18:22.318436 2687927 logs.go:123] Gathering logs for Docker ...
I0520 15:18:22.318468 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0520 15:18:22.362088 2687927 logs.go:123] Gathering logs for container status ...
I0520 15:18:22.362124 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0520 15:18:22.428320 2687927 logs.go:123] Gathering logs for kube-apiserver [21cda61304ec] ...
I0520 15:18:22.428347 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 21cda61304ec"
I0520 15:18:22.512882 2687927 logs.go:123] Gathering logs for kubernetes-dashboard [e9d80f29a80f] ...
I0520 15:18:22.512921 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 e9d80f29a80f"
I0520 15:18:22.547849 2687927 logs.go:123] Gathering logs for kube-apiserver [7b906f9c7e42] ...
I0520 15:18:22.547961 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b906f9c7e42"
I0520 15:18:22.611472 2687927 logs.go:123] Gathering logs for etcd [4183e6237564] ...
I0520 15:18:22.611514 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4183e6237564"
I0520 15:18:22.640116 2687927 logs.go:123] Gathering logs for kube-scheduler [20ae03428247] ...
I0520 15:18:22.640149 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 20ae03428247"
I0520 15:18:22.669795 2687927 logs.go:123] Gathering logs for kube-proxy [75eb680f6808] ...
I0520 15:18:22.669823 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 75eb680f6808"
I0520 15:18:22.700074 2687927 logs.go:123] Gathering logs for etcd [78716dd9f5e6] ...
I0520 15:18:22.700103 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 78716dd9f5e6"
I0520 15:18:22.744444 2687927 logs.go:123] Gathering logs for kube-scheduler [798f34afc547] ...
I0520 15:18:22.744475 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 798f34afc547"
I0520 15:18:22.785309 2687927 logs.go:123] Gathering logs for kube-controller-manager [18ca6dabc240] ...
I0520 15:18:22.785340 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 18ca6dabc240"
I0520 15:18:22.849036 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:22.849070 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0520 15:18:22.849150 2687927 out.go:239] X Problems detected in kubelet:
X Problems detected in kubelet:
W0520 15:18:22.849163 2687927 out.go:239] May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849202 2687927 out.go:239] May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849215 2687927 out.go:239] May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849222 2687927 out.go:239] May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849241 2687927 out.go:239] May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:22.849252 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:22.849258 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:18:32.850044 2687927 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0520 15:18:32.868528 2687927 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0520 15:18:32.871090 2687927 out.go:177]
W0520 15:18:32.873480 2687927 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0520 15:18:32.873524 2687927 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
* Suggestion: Control Plane could not update, try minikube delete --all --purge
W0520 15:18:32.873552 2687927 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
* Related issue: https://github.com/kubernetes/minikube/issues/11417
W0520 15:18:32.873563 2687927 out.go:239] *
*
W0520 15:18:32.874427 2687927 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0520 15:18:32.877815 2687927 out.go:177]
** /stderr **
start_stop_delete_test.go:259: failed to start minikube post-stop. args "out/minikube-linux-arm64 start -p old-k8s-version-185009 --memory=2200 --alsologtostderr --wait=true --kvm-network=default --kvm-qemu-uri=qemu:///system --disable-driver-mounts --keep-context=false --driver=docker --container-runtime=docker --kubernetes-version=v1.20.0": exit status 102
helpers_test.go:222: -----------------------post-mortem--------------------------------
helpers_test.go:230: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: docker inspect <======
helpers_test.go:231: (dbg) Run: docker inspect old-k8s-version-185009
helpers_test.go:235: (dbg) docker inspect old-k8s-version-185009:
-- stdout --
[
{
"Id": "6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3",
"Created": "2024-05-20T15:09:18.544693824Z",
"Path": "/usr/local/bin/entrypoint",
"Args": [
"/sbin/init"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 2688136,
"ExitCode": 0,
"Error": "",
"StartedAt": "2024-05-20T15:12:22.41613542Z",
"FinishedAt": "2024-05-20T15:12:21.027429312Z"
},
"Image": "sha256:56620e18f2c2c9a0448fc43c42f840334bd2baea497ff8deae66477dd0dbfecf",
"ResolvConfPath": "/var/lib/docker/containers/6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3/hostname",
"HostsPath": "/var/lib/docker/containers/6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3/hosts",
"LogPath": "/var/lib/docker/containers/6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3/6fa9376b15eefe7100683cca9d5d2c5de6bf19afe6177f61cb9b8372406a9ce3-json.log",
"Name": "/old-k8s-version-185009",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "unconfined",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/lib/modules:/lib/modules:ro",
"old-k8s-version-185009:/var"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "old-k8s-version-185009",
"PortBindings": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": ""
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": true,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": [
"seccomp=unconfined",
"apparmor=unconfined",
"label=disable"
],
"Tmpfs": {
"/run": "",
"/tmp": ""
},
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 2306867200,
"NanoCpus": 2000000000,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": [],
"BlkioDeviceWriteBps": [],
"BlkioDeviceReadIOps": [],
"BlkioDeviceWriteIOps": [],
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 4613734400,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": null,
"ReadonlyPaths": null
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/b5a8710961384a1e64d4a6c2da195fc314052a6a339f1cdd0c23368e6fd16291-init/diff:/var/lib/docker/overlay2/eb58d5eec84b374be8fb18d2d95ec8703ff7962b0d68bcbb57666f474e2f61ed/diff",
"MergedDir": "/var/lib/docker/overlay2/b5a8710961384a1e64d4a6c2da195fc314052a6a339f1cdd0c23368e6fd16291/merged",
"UpperDir": "/var/lib/docker/overlay2/b5a8710961384a1e64d4a6c2da195fc314052a6a339f1cdd0c23368e6fd16291/diff",
"WorkDir": "/var/lib/docker/overlay2/b5a8710961384a1e64d4a6c2da195fc314052a6a339f1cdd0c23368e6fd16291/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "old-k8s-version-185009",
"Source": "/var/lib/docker/volumes/old-k8s-version-185009/_data",
"Destination": "/var",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
},
{
"Type": "bind",
"Source": "/lib/modules",
"Destination": "/lib/modules",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "old-k8s-version-185009",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"22/tcp": {},
"2376/tcp": {},
"32443/tcp": {},
"5000/tcp": {},
"8443/tcp": {}
},
"Tty": true,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"container=docker",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a",
"Volumes": null,
"WorkingDir": "/",
"Entrypoint": [
"/usr/local/bin/entrypoint",
"/sbin/init"
],
"OnBuild": null,
"Labels": {
"created_by.minikube.sigs.k8s.io": "true",
"mode.minikube.sigs.k8s.io": "old-k8s-version-185009",
"name.minikube.sigs.k8s.io": "old-k8s-version-185009",
"role.minikube.sigs.k8s.io": ""
},
"StopSignal": "SIGRTMIN+3"
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "86600a905d7ffe6fb6393d60e38dc227541a04d4862d601bac768922862453c9",
"SandboxKey": "/var/run/docker/netns/86600a905d7f",
"Ports": {
"22/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41933"
}
],
"2376/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41932"
}
],
"32443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41929"
}
],
"5000/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41931"
}
],
"8443/tcp": [
{
"HostIp": "127.0.0.1",
"HostPort": "41930"
}
]
},
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"old-k8s-version-185009": {
"IPAMConfig": {
"IPv4Address": "192.168.76.2"
},
"Links": null,
"Aliases": null,
"MacAddress": "02:42:c0:a8:4c:02",
"NetworkID": "fc74e77bcf9ce811ceb7f93010d4a84c90a46d82c2fe347db44bd269b6ccad3d",
"EndpointID": "0f0a51ef667586adb9bdbe869f8849cf4ff65a2f23d6c55b6083f0274da4c537",
"Gateway": "192.168.76.1",
"IPAddress": "192.168.76.2",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"old-k8s-version-185009",
"6fa9376b15ee"
]
}
}
}
}
]
-- /stdout --
helpers_test.go:239: (dbg) Run: out/minikube-linux-arm64 status --format={{.Host}} -p old-k8s-version-185009 -n old-k8s-version-185009
helpers_test.go:244: <<< TestStartStop/group/old-k8s-version/serial/SecondStart FAILED: start of post-mortem logs <<<
helpers_test.go:245: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: minikube logs <======
helpers_test.go:247: (dbg) Run: out/minikube-linux-arm64 -p old-k8s-version-185009 logs -n 25
helpers_test.go:247: (dbg) Done: out/minikube-linux-arm64 -p old-k8s-version-185009 logs -n 25: (1.56481241s)
helpers_test.go:252: TestStartStop/group/old-k8s-version/serial/SecondStart logs:
-- stdout --
==> Audit <==
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| Command | Args | Profile | User | Version | Start Time | End Time |
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
| ssh | docker-flags-127480 ssh | docker-flags-127480 | jenkins | v1.33.1 | 20 May 24 15:08 UTC | 20 May 24 15:08 UTC |
| | sudo systemctl show docker | | | | | |
| | --property=Environment | | | | | |
| | --no-pager | | | | | |
| ssh | docker-flags-127480 ssh | docker-flags-127480 | jenkins | v1.33.1 | 20 May 24 15:08 UTC | 20 May 24 15:08 UTC |
| | sudo systemctl show docker | | | | | |
| | --property=ExecStart | | | | | |
| | --no-pager | | | | | |
| delete | -p docker-flags-127480 | docker-flags-127480 | jenkins | v1.33.1 | 20 May 24 15:08 UTC | 20 May 24 15:08 UTC |
| start | -p cert-options-949820 | cert-options-949820 | jenkins | v1.33.1 | 20 May 24 15:08 UTC | 20 May 24 15:09 UTC |
| | --memory=2048 | | | | | |
| | --apiserver-ips=127.0.0.1 | | | | | |
| | --apiserver-ips=192.168.15.15 | | | | | |
| | --apiserver-names=localhost | | | | | |
| | --apiserver-names=www.google.com | | | | | |
| | --apiserver-port=8555 | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| ssh | cert-options-949820 ssh | cert-options-949820 | jenkins | v1.33.1 | 20 May 24 15:09 UTC | 20 May 24 15:09 UTC |
| | openssl x509 -text -noout -in | | | | | |
| | /var/lib/minikube/certs/apiserver.crt | | | | | |
| ssh | -p cert-options-949820 -- sudo | cert-options-949820 | jenkins | v1.33.1 | 20 May 24 15:09 UTC | 20 May 24 15:09 UTC |
| | cat /etc/kubernetes/admin.conf | | | | | |
| delete | -p cert-options-949820 | cert-options-949820 | jenkins | v1.33.1 | 20 May 24 15:09 UTC | 20 May 24 15:09 UTC |
| start | -p old-k8s-version-185009 | old-k8s-version-185009 | jenkins | v1.33.1 | 20 May 24 15:09 UTC | 20 May 24 15:11 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| start | -p cert-expiration-483756 | cert-expiration-483756 | jenkins | v1.33.1 | 20 May 24 15:11 UTC | 20 May 24 15:11 UTC |
| | --memory=2048 | | | | | |
| | --cert-expiration=8760h | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| delete | -p cert-expiration-483756 | cert-expiration-483756 | jenkins | v1.33.1 | 20 May 24 15:11 UTC | 20 May 24 15:11 UTC |
| start | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:11 UTC | 20 May 24 15:12 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
| addons | enable metrics-server -p old-k8s-version-185009 | old-k8s-version-185009 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | 20 May 24 15:12 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p old-k8s-version-185009 | old-k8s-version-185009 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | 20 May 24 15:12 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p old-k8s-version-185009 | old-k8s-version-185009 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | 20 May 24 15:12 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p old-k8s-version-185009 | old-k8s-version-185009 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --kvm-network=default | | | | | |
| | --kvm-qemu-uri=qemu:///system | | | | | |
| | --disable-driver-mounts | | | | | |
| | --keep-context=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.20.0 | | | | | |
| addons | enable metrics-server -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | 20 May 24 15:12 UTC |
| | --images=MetricsServer=registry.k8s.io/echoserver:1.4 | | | | | |
| | --registries=MetricsServer=fake.domain | | | | | |
| stop | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:12 UTC | 20 May 24 15:13 UTC |
| | --alsologtostderr -v=3 | | | | | |
| addons | enable dashboard -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:13 UTC | 20 May 24 15:13 UTC |
| | --images=MetricsScraper=registry.k8s.io/echoserver:1.4 | | | | | |
| start | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:13 UTC | 20 May 24 15:17 UTC |
| | --memory=2200 | | | | | |
| | --alsologtostderr | | | | | |
| | --wait=true --preload=false | | | | | |
| | --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
| image | no-preload-356893 image list | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | 20 May 24 15:17 UTC |
| | --format=json | | | | | |
| pause | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | 20 May 24 15:17 UTC |
| | --alsologtostderr -v=1 | | | | | |
| unpause | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | 20 May 24 15:17 UTC |
| | --alsologtostderr -v=1 | | | | | |
| delete | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | 20 May 24 15:17 UTC |
| delete | -p no-preload-356893 | no-preload-356893 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | 20 May 24 15:17 UTC |
| start | -p embed-certs-684040 | embed-certs-684040 | jenkins | v1.33.1 | 20 May 24 15:17 UTC | |
| | --memory=2200 | | | | | |
| | --alsologtostderr --wait=true | | | | | |
| | --embed-certs --driver=docker | | | | | |
| | --container-runtime=docker | | | | | |
| | --kubernetes-version=v1.30.1 | | | | | |
|---------|--------------------------------------------------------|------------------------|---------|---------|---------------------|---------------------|
==> Last Start <==
Log file created at: 2024/05/20 15:17:49
Running on machine: ip-172-31-21-244
Binary: Built with gc go1.22.3 for linux/arm64
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0520 15:17:49.241764 2701137 out.go:291] Setting OutFile to fd 1 ...
I0520 15:17:49.241914 2701137 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:17:49.241926 2701137 out.go:304] Setting ErrFile to fd 2...
I0520 15:17:49.241932 2701137 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:17:49.242188 2701137 root.go:338] Updating PATH: /home/jenkins/minikube-integration/18929-2395474/.minikube/bin
I0520 15:17:49.242592 2701137 out.go:298] Setting JSON to false
I0520 15:17:49.244109 2701137 start.go:129] hostinfo: {"hostname":"ip-172-31-21-244","uptime":169217,"bootTime":1716049053,"procs":234,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"20.04","kernelVersion":"5.15.0-1058-aws","kernelArch":"aarch64","virtualizationSystem":"","virtualizationRole":"","hostId":"da8ac1fd-6236-412a-a346-95873c98230d"}
I0520 15:17:49.244239 2701137 start.go:139] virtualization:
I0520 15:17:49.246751 2701137 out.go:177] * [embed-certs-684040] minikube v1.33.1 on Ubuntu 20.04 (arm64)
I0520 15:17:49.249105 2701137 out.go:177] - MINIKUBE_LOCATION=18929
I0520 15:17:49.251018 2701137 out.go:177] - MINIKUBE_SUPPRESS_DOCKER_PERFORMANCE=true
I0520 15:17:49.249183 2701137 notify.go:220] Checking for updates...
I0520 15:17:49.255996 2701137 out.go:177] - KUBECONFIG=/home/jenkins/minikube-integration/18929-2395474/kubeconfig
I0520 15:17:49.257571 2701137 out.go:177] - MINIKUBE_HOME=/home/jenkins/minikube-integration/18929-2395474/.minikube
I0520 15:17:49.259420 2701137 out.go:177] - MINIKUBE_BIN=out/minikube-linux-arm64
I0520 15:17:49.260862 2701137 out.go:177] - MINIKUBE_FORCE_SYSTEMD=
I0520 15:17:49.263062 2701137 config.go:182] Loaded profile config "old-k8s-version-185009": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.20.0
I0520 15:17:49.263208 2701137 driver.go:392] Setting default libvirt URI to qemu:///system
I0520 15:17:49.283615 2701137 docker.go:122] docker version: linux-26.1.3:Docker Engine - Community
I0520 15:17:49.283756 2701137 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0520 15:17:49.345186 2701137 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-05-20 15:17:49.335526742 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1058-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0520 15:17:49.345438 2701137 docker.go:295] overlay module found
I0520 15:17:49.347562 2701137 out.go:177] * Using the docker driver based on user configuration
I0520 15:17:49.349434 2701137 start.go:297] selected driver: docker
I0520 15:17:49.349454 2701137 start.go:901] validating driver "docker" against <nil>
I0520 15:17:49.349467 2701137 start.go:912] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc: Version:}
I0520 15:17:49.350301 2701137 cli_runner.go:164] Run: docker system info --format "{{json .}}"
I0520 15:17:49.411921 2701137 info.go:266] docker info: {ID:5FDH:SA5P:5GCT:NLAS:B73P:SGDQ:PBG5:UBVH:UZY3:RXGO:CI7S:WAIH Containers:1 ContainersRunning:1 ContainersPaused:0 ContainersStopped:0 Images:4 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Using metacopy false] [Native Overlay Diff true] [userxattr false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:false KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:33 OomKillDisable:true NGoroutines:52 SystemTime:2024-05-20 15:17:49.402653214 +0000 UTC LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.15.0-1058-aws OperatingSystem:Ubuntu 20.04.6 LTS OSType:linux Architecture:aar
ch64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:2 MemTotal:8215035904 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:ip-172-31-21-244 Labels:[] ExperimentalBuild:false ServerVersion:26.1.3 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:e377cd56a71523140ca6ae87e30244719194a521 Expected:e377cd56a71523140ca6ae87e30244719194a521} RuncCommit:{ID:v1.1.12-0-g51d5e94 Expected:v1.1.12-0-g51d5e94} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=apparmor name=seccomp,profile=builtin] ProductLicense: Warnings:<nil> ServerErro
rs:[] ClientInfo:{Debug:false Plugins:[map[Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Docker Buildx Vendor:Docker Inc. Version:v0.14.0] map[Name:compose Path:/usr/libexec/docker/cli-plugins/docker-compose SchemaVersion:0.1.0 ShortDescription:Docker Compose Vendor:Docker Inc. Version:v2.27.0]] Warnings:<nil>}}
I0520 15:17:49.412101 2701137 start_flags.go:310] no existing cluster config was found, will generate one from the flags
I0520 15:17:49.412336 2701137 start_flags.go:947] Waiting for all components: map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true]
I0520 15:17:49.414182 2701137 out.go:177] * Using Docker driver with root privileges
I0520 15:17:49.415877 2701137 cni.go:84] Creating CNI manager for ""
I0520 15:17:49.415910 2701137 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I0520 15:17:49.415920 2701137 start_flags.go:319] Found "bridge CNI" CNI - setting NetworkPlugin=cni
I0520 15:17:49.415997 2701137 start.go:340] cluster config:
{Name:embed-certs-684040 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:embed-certs-684040 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local Contain
erRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSH
AuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0520 15:17:49.419343 2701137 out.go:177] * Starting "embed-certs-684040" primary control-plane node in "embed-certs-684040" cluster
I0520 15:17:49.421157 2701137 cache.go:121] Beginning downloading kic base image for docker with docker
I0520 15:17:49.423306 2701137 out.go:177] * Pulling base image v0.0.44-1715707529-18887 ...
I0520 15:17:49.425301 2701137 preload.go:132] Checking if preload exists for k8s version v1.30.1 and runtime docker
I0520 15:17:49.425355 2701137 preload.go:147] Found local preload: /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.1-docker-overlay2-arm64.tar.lz4
I0520 15:17:49.425368 2701137 cache.go:56] Caching tarball of preloaded images
I0520 15:17:49.425387 2701137 image.go:79] Checking for gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a in local docker daemon
I0520 15:17:49.425450 2701137 preload.go:173] Found /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.1-docker-overlay2-arm64.tar.lz4 in cache, skipping download
I0520 15:17:49.425459 2701137 cache.go:59] Finished verifying existence of preloaded tar for v1.30.1 on docker
I0520 15:17:49.425565 2701137 profile.go:143] Saving config to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/config.json ...
I0520 15:17:49.425584 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/config.json: {Name:mk33ff875afcaad2ae5fc00acd958165a9bfd643 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:17:49.440968 2701137 image.go:83] Found gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a in local docker daemon, skipping pull
I0520 15:17:49.440992 2701137 cache.go:144] gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a exists in daemon, skipping load
I0520 15:17:49.441017 2701137 cache.go:194] Successfully downloaded all kic artifacts
I0520 15:17:49.441059 2701137 start.go:360] acquireMachinesLock for embed-certs-684040: {Name:mk2f896f0c36c7028f900d00d57f3dce7f505ad7 Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0520 15:17:49.441186 2701137 start.go:364] duration metric: took 108.116µs to acquireMachinesLock for "embed-certs-684040"
I0520 15:17:49.441213 2701137 start.go:93] Provisioning new machine with config: &{Name:embed-certs-684040 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:embed-certs-684040 Namespace:default APIServerHAVIP: APIServe
rName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:f
alse CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} &{Name: IP: Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:docker ControlPlane:true Worker:true}
I0520 15:17:49.441295 2701137 start.go:125] createHost starting for "" (driver="docker")
I0520 15:17:48.259944 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:50.266967 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:49.443876 2701137 out.go:204] * Creating docker container (CPUs=2, Memory=2200MB) ...
I0520 15:17:49.444318 2701137 start.go:159] libmachine.API.Create for "embed-certs-684040" (driver="docker")
I0520 15:17:49.444357 2701137 client.go:168] LocalClient.Create starting
I0520 15:17:49.444421 2701137 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem
I0520 15:17:49.444459 2701137 main.go:141] libmachine: Decoding PEM data...
I0520 15:17:49.444477 2701137 main.go:141] libmachine: Parsing certificate...
I0520 15:17:49.444528 2701137 main.go:141] libmachine: Reading certificate data from /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem
I0520 15:17:49.444552 2701137 main.go:141] libmachine: Decoding PEM data...
I0520 15:17:49.444566 2701137 main.go:141] libmachine: Parsing certificate...
I0520 15:17:49.444950 2701137 cli_runner.go:164] Run: docker network inspect embed-certs-684040 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
W0520 15:17:49.459144 2701137 cli_runner.go:211] docker network inspect embed-certs-684040 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}" returned with exit code 1
I0520 15:17:49.459224 2701137 network_create.go:281] running [docker network inspect embed-certs-684040] to gather additional debugging logs...
I0520 15:17:49.459246 2701137 cli_runner.go:164] Run: docker network inspect embed-certs-684040
W0520 15:17:49.473086 2701137 cli_runner.go:211] docker network inspect embed-certs-684040 returned with exit code 1
I0520 15:17:49.473128 2701137 network_create.go:284] error running [docker network inspect embed-certs-684040]: docker network inspect embed-certs-684040: exit status 1
stdout:
[]
stderr:
Error response from daemon: network embed-certs-684040 not found
I0520 15:17:49.473141 2701137 network_create.go:286] output of [docker network inspect embed-certs-684040]: -- stdout --
[]
-- /stdout --
** stderr **
Error response from daemon: network embed-certs-684040 not found
** /stderr **
I0520 15:17:49.473270 2701137 cli_runner.go:164] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0520 15:17:49.488344 2701137 network.go:211] skipping subnet 192.168.49.0/24 that is taken: &{IP:192.168.49.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.49.0/24 Gateway:192.168.49.1 ClientMin:192.168.49.2 ClientMax:192.168.49.254 Broadcast:192.168.49.255 IsPrivate:true Interface:{IfaceName:br-3733c4422ad6 IfaceIPv4:192.168.49.1 IfaceMTU:1500 IfaceMAC:02:42:25:1c:31:92} reservation:<nil>}
I0520 15:17:49.488677 2701137 network.go:211] skipping subnet 192.168.58.0/24 that is taken: &{IP:192.168.58.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.58.0/24 Gateway:192.168.58.1 ClientMin:192.168.58.2 ClientMax:192.168.58.254 Broadcast:192.168.58.255 IsPrivate:true Interface:{IfaceName:br-8e08f549db52 IfaceIPv4:192.168.58.1 IfaceMTU:1500 IfaceMAC:02:42:88:38:4c:5c} reservation:<nil>}
I0520 15:17:49.489145 2701137 network.go:211] skipping subnet 192.168.67.0/24 that is taken: &{IP:192.168.67.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.67.0/24 Gateway:192.168.67.1 ClientMin:192.168.67.2 ClientMax:192.168.67.254 Broadcast:192.168.67.255 IsPrivate:true Interface:{IfaceName:br-757ec3af1f14 IfaceIPv4:192.168.67.1 IfaceMTU:1500 IfaceMAC:02:42:9f:47:33:c6} reservation:<nil>}
I0520 15:17:49.489540 2701137 network.go:211] skipping subnet 192.168.76.0/24 that is taken: &{IP:192.168.76.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.76.0/24 Gateway:192.168.76.1 ClientMin:192.168.76.2 ClientMax:192.168.76.254 Broadcast:192.168.76.255 IsPrivate:true Interface:{IfaceName:br-fc74e77bcf9c IfaceIPv4:192.168.76.1 IfaceMTU:1500 IfaceMAC:02:42:4b:45:18:d8} reservation:<nil>}
I0520 15:17:49.490065 2701137 network.go:206] using free private subnet 192.168.85.0/24: &{IP:192.168.85.0 Netmask:255.255.255.0 Prefix:24 CIDR:192.168.85.0/24 Gateway:192.168.85.1 ClientMin:192.168.85.2 ClientMax:192.168.85.254 Broadcast:192.168.85.255 IsPrivate:true Interface:{IfaceName: IfaceIPv4: IfaceMTU:0 IfaceMAC:} reservation:0x40016af810}
I0520 15:17:49.490148 2701137 network_create.go:124] attempt to create docker network embed-certs-684040 192.168.85.0/24 with gateway 192.168.85.1 and MTU of 1500 ...
I0520 15:17:49.490231 2701137 cli_runner.go:164] Run: docker network create --driver=bridge --subnet=192.168.85.0/24 --gateway=192.168.85.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true --label=name.minikube.sigs.k8s.io=embed-certs-684040 embed-certs-684040
I0520 15:17:49.560874 2701137 network_create.go:108] docker network embed-certs-684040 192.168.85.0/24 created
I0520 15:17:49.560908 2701137 kic.go:121] calculated static IP "192.168.85.2" for the "embed-certs-684040" container
I0520 15:17:49.560980 2701137 cli_runner.go:164] Run: docker ps -a --format {{.Names}}
I0520 15:17:49.600019 2701137 cli_runner.go:164] Run: docker volume create embed-certs-684040 --label name.minikube.sigs.k8s.io=embed-certs-684040 --label created_by.minikube.sigs.k8s.io=true
I0520 15:17:49.624236 2701137 oci.go:103] Successfully created a docker volume embed-certs-684040
I0520 15:17:49.624339 2701137 cli_runner.go:164] Run: docker run --rm --name embed-certs-684040-preload-sidecar --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-684040 --entrypoint /usr/bin/test -v embed-certs-684040:/var gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a -d /var/lib
I0520 15:17:50.326759 2701137 oci.go:107] Successfully prepared a docker volume embed-certs-684040
I0520 15:17:50.326812 2701137 preload.go:132] Checking if preload exists for k8s version v1.30.1 and runtime docker
I0520 15:17:50.326833 2701137 kic.go:194] Starting extracting preloaded images to volume ...
I0520 15:17:50.326922 2701137 cli_runner.go:164] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.1-docker-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-684040:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a -I lz4 -xf /preloaded.tar -C /extractDir
I0520 15:17:52.757742 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:54.762524 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:54.627207 2701137 cli_runner.go:217] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/jenkins/minikube-integration/18929-2395474/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v18-v1.30.1-docker-overlay2-arm64.tar.lz4:/preloaded.tar:ro -v embed-certs-684040:/extractDir gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a -I lz4 -xf /preloaded.tar -C /extractDir: (4.300238597s)
I0520 15:17:54.627243 2701137 kic.go:203] duration metric: took 4.300406101s to extract preloaded images to volume ...
W0520 15:17:54.627395 2701137 cgroups_linux.go:77] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0520 15:17:54.627539 2701137 cli_runner.go:164] Run: docker info --format "'{{json .SecurityOptions}}'"
I0520 15:17:54.689291 2701137 cli_runner.go:164] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname embed-certs-684040 --name embed-certs-684040 --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=embed-certs-684040 --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=embed-certs-684040 --network embed-certs-684040 --ip 192.168.85.2 --volume embed-certs-684040:/var --security-opt apparmor=unconfined --memory=2200mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 --publish=127.0.0.1::32443 gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a
I0520 15:17:55.051324 2701137 cli_runner.go:164] Run: docker container inspect embed-certs-684040 --format={{.State.Running}}
I0520 15:17:55.076608 2701137 cli_runner.go:164] Run: docker container inspect embed-certs-684040 --format={{.State.Status}}
I0520 15:17:55.104055 2701137 cli_runner.go:164] Run: docker exec embed-certs-684040 stat /var/lib/dpkg/alternatives/iptables
I0520 15:17:55.183762 2701137 oci.go:144] the created container "embed-certs-684040" has a running status.
I0520 15:17:55.183791 2701137 kic.go:225] Creating ssh key for kic: /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa...
I0520 15:17:55.625973 2701137 kic_runner.go:191] docker (temp): /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa.pub --> /home/docker/.ssh/authorized_keys (381 bytes)
I0520 15:17:55.657849 2701137 cli_runner.go:164] Run: docker container inspect embed-certs-684040 --format={{.State.Status}}
I0520 15:17:55.678599 2701137 kic_runner.go:93] Run: chown docker:docker /home/docker/.ssh/authorized_keys
I0520 15:17:55.678620 2701137 kic_runner.go:114] Args: [docker exec --privileged embed-certs-684040 chown docker:docker /home/docker/.ssh/authorized_keys]
I0520 15:17:55.772240 2701137 cli_runner.go:164] Run: docker container inspect embed-certs-684040 --format={{.State.Status}}
I0520 15:17:55.792283 2701137 machine.go:94] provisionDockerMachine start ...
I0520 15:17:55.792369 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:55.815889 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:55.816170 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:55.816179 2701137 main.go:141] libmachine: About to run SSH command:
hostname
I0520 15:17:56.013878 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-684040
I0520 15:17:56.013905 2701137 ubuntu.go:169] provisioning hostname "embed-certs-684040"
I0520 15:17:56.013976 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:56.038561 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:56.038946 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:56.038999 2701137 main.go:141] libmachine: About to run SSH command:
sudo hostname embed-certs-684040 && echo "embed-certs-684040" | sudo tee /etc/hostname
I0520 15:17:56.199268 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>: embed-certs-684040
I0520 15:17:56.199405 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:56.221732 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:56.222010 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:56.222029 2701137 main.go:141] libmachine: About to run SSH command:
if ! grep -xq '.*\sembed-certs-684040' /etc/hosts; then
if grep -xq '127.0.1.1\s.*' /etc/hosts; then
sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 embed-certs-684040/g' /etc/hosts;
else
echo '127.0.1.1 embed-certs-684040' | sudo tee -a /etc/hosts;
fi
fi
I0520 15:17:56.369858 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>:
I0520 15:17:56.369895 2701137 ubuntu.go:175] set auth options {CertDir:/home/jenkins/minikube-integration/18929-2395474/.minikube CaCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/18929-2395474/.minikube}
I0520 15:17:56.369926 2701137 ubuntu.go:177] setting up certificates
I0520 15:17:56.369939 2701137 provision.go:84] configureAuth start
I0520 15:17:56.370006 2701137 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-684040
I0520 15:17:56.387357 2701137 provision.go:143] copyHostCerts
I0520 15:17:56.387424 2701137 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem, removing ...
I0520 15:17:56.387438 2701137 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem
I0520 15:17:56.387550 2701137 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/key.pem (1679 bytes)
I0520 15:17:56.387670 2701137 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem, removing ...
I0520 15:17:56.387683 2701137 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem
I0520 15:17:56.387713 2701137 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.pem (1082 bytes)
I0520 15:17:56.387778 2701137 exec_runner.go:144] found /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem, removing ...
I0520 15:17:56.387792 2701137 exec_runner.go:203] rm: /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem
I0520 15:17:56.387819 2701137 exec_runner.go:151] cp: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem --> /home/jenkins/minikube-integration/18929-2395474/.minikube/cert.pem (1123 bytes)
I0520 15:17:56.387875 2701137 provision.go:117] generating server cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem org=jenkins.embed-certs-684040 san=[127.0.0.1 192.168.85.2 embed-certs-684040 localhost minikube]
I0520 15:17:56.831498 2701137 provision.go:177] copyRemoteCerts
I0520 15:17:56.831572 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/docker /etc/docker /etc/docker
I0520 15:17:56.831613 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:56.850421 2701137 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41943 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa Username:docker}
I0520 15:17:56.946937 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem --> /etc/docker/ca.pem (1082 bytes)
I0520 15:17:56.977208 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server.pem --> /etc/docker/server.pem (1220 bytes)
I0520 15:17:57.012850 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/machines/server-key.pem --> /etc/docker/server-key.pem (1679 bytes)
I0520 15:17:57.042012 2701137 provision.go:87] duration metric: took 672.053083ms to configureAuth
I0520 15:17:57.042051 2701137 ubuntu.go:193] setting minikube options for container-runtime
I0520 15:17:57.042261 2701137 config.go:182] Loaded profile config "embed-certs-684040": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.30.1
I0520 15:17:57.042332 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:57.064844 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:57.065095 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:57.065112 2701137 main.go:141] libmachine: About to run SSH command:
df --output=fstype / | tail -n 1
I0520 15:17:57.198741 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>: overlay
I0520 15:17:57.198764 2701137 ubuntu.go:71] root file system type: overlay
I0520 15:17:57.198887 2701137 provision.go:314] Updating docker unit: /lib/systemd/system/docker.service ...
I0520 15:17:57.198953 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:57.217543 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:57.218018 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:57.218111 2701137 main.go:141] libmachine: About to run SSH command:
sudo mkdir -p /lib/systemd/system && printf %!s(MISSING) "[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP \$MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
" | sudo tee /lib/systemd/system/docker.service.new
I0520 15:17:57.355934 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>: [Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
StartLimitBurst=3
StartLimitIntervalSec=60
[Service]
Type=notify
Restart=on-failure
# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
# The base configuration already specifies an 'ExecStart=...' command. The first directive
# here is to clear out that command inherited from the base configuration. Without this,
# the command from the base configuration and the command specified here are treated as
# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
# will catch this invalid input and refuse to start the service with an error like:
# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
I0520 15:17:57.356035 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:57.373774 2701137 main.go:141] libmachine: Using SSH client type: native
I0520 15:17:57.374058 2701137 main.go:141] libmachine: &{{{<nil> 0 [] [] []} docker [0x3e2b30] 0x3e5390 <nil> [] 0s} 127.0.0.1 41943 <nil> <nil>}
I0520 15:17:57.374085 2701137 main.go:141] libmachine: About to run SSH command:
sudo diff -u /lib/systemd/system/docker.service /lib/systemd/system/docker.service.new || { sudo mv /lib/systemd/system/docker.service.new /lib/systemd/system/docker.service; sudo systemctl -f daemon-reload && sudo systemctl -f enable docker && sudo systemctl -f restart docker; }
I0520 15:17:58.205109 2701137 main.go:141] libmachine: SSH cmd err, output: <nil>: --- /lib/systemd/system/docker.service 2024-05-08 13:59:39.000000000 +0000
+++ /lib/systemd/system/docker.service.new 2024-05-20 15:17:57.350949358 +0000
@@ -1,46 +1,49 @@
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
-After=network-online.target docker.socket firewalld.service containerd.service time-set.target
-Wants=network-online.target containerd.service
+BindsTo=containerd.service
+After=network-online.target firewalld.service containerd.service
+Wants=network-online.target
Requires=docker.socket
+StartLimitBurst=3
+StartLimitIntervalSec=60
[Service]
Type=notify
-# the default is not to use systemd for cgroups because the delegate issues still
-# exists and systemd currently does not support the cgroup feature set required
-# for containers run by docker
-ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-ExecReload=/bin/kill -s HUP $MAINPID
-TimeoutStartSec=0
-RestartSec=2
-Restart=always
+Restart=on-failure
-# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
-# Both the old, and new location are accepted by systemd 229 and up, so using the old location
-# to make them work for either version of systemd.
-StartLimitBurst=3
-# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
-# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
-# this option work for either version of systemd.
-StartLimitInterval=60s
+
+# This file is a systemd drop-in unit that inherits from the base dockerd configuration.
+# The base configuration already specifies an 'ExecStart=...' command. The first directive
+# here is to clear out that command inherited from the base configuration. Without this,
+# the command from the base configuration and the command specified here are treated as
+# a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
+# will catch this invalid input and refuse to start the service with an error like:
+# Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
+
+# NOTE: default-ulimit=nofile is set to an arbitrary number for consistency with other
+# container runtimes. If left unlimited, it may result in OOM issues with MySQL.
+ExecStart=
+ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=docker --insecure-registry 10.96.0.0/12
+ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
+LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
-# Comment TasksMax if your systemd version does not support it.
-# Only systemd 226 and above support this option.
+# Uncomment TasksMax if your systemd version supports it.
+# Only systemd 226 and above support this version.
TasksMax=infinity
+TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
-OOMScoreAdjust=-500
[Install]
WantedBy=multi-user.target
Synchronizing state of docker.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable docker
I0520 15:17:58.205142 2701137 machine.go:97] duration metric: took 2.41284163s to provisionDockerMachine
I0520 15:17:58.205155 2701137 client.go:171] duration metric: took 8.760789457s to LocalClient.Create
I0520 15:17:58.205217 2701137 start.go:167] duration metric: took 8.760900446s to libmachine.API.Create "embed-certs-684040"
I0520 15:17:58.205229 2701137 start.go:293] postStartSetup for "embed-certs-684040" (driver="docker")
I0520 15:17:58.205241 2701137 start.go:322] creating required directories: [/etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs]
I0520 15:17:58.205317 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/kubernetes/addons /etc/kubernetes/manifests /var/tmp/minikube /var/lib/minikube /var/lib/minikube/certs /var/lib/minikube/images /var/lib/minikube/binaries /tmp/gvisor /usr/share/ca-certificates /etc/ssl/certs
I0520 15:17:58.205370 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:58.222895 2701137 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41943 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa Username:docker}
I0520 15:17:58.318665 2701137 ssh_runner.go:195] Run: cat /etc/os-release
I0520 15:17:58.322138 2701137 main.go:141] libmachine: Couldn't set key VERSION_CODENAME, no corresponding struct field found
I0520 15:17:58.322176 2701137 main.go:141] libmachine: Couldn't set key PRIVACY_POLICY_URL, no corresponding struct field found
I0520 15:17:58.322189 2701137 main.go:141] libmachine: Couldn't set key UBUNTU_CODENAME, no corresponding struct field found
I0520 15:17:58.322200 2701137 info.go:137] Remote host: Ubuntu 22.04.4 LTS
I0520 15:17:58.322212 2701137 filesync.go:126] Scanning /home/jenkins/minikube-integration/18929-2395474/.minikube/addons for local assets ...
I0520 15:17:58.322282 2701137 filesync.go:126] Scanning /home/jenkins/minikube-integration/18929-2395474/.minikube/files for local assets ...
I0520 15:17:58.322377 2701137 filesync.go:149] local asset: /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem -> 24009082.pem in /etc/ssl/certs
I0520 15:17:58.322492 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/ssl/certs
I0520 15:17:58.331635 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem --> /etc/ssl/certs/24009082.pem (1708 bytes)
I0520 15:17:58.361898 2701137 start.go:296] duration metric: took 156.653106ms for postStartSetup
I0520 15:17:58.362332 2701137 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-684040
I0520 15:17:58.376365 2701137 profile.go:143] Saving config to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/config.json ...
I0520 15:17:58.376653 2701137 ssh_runner.go:195] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0520 15:17:58.376707 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:58.395007 2701137 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41943 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa Username:docker}
I0520 15:17:58.483041 2701137 ssh_runner.go:195] Run: sh -c "df -BG /var | awk 'NR==2{print $4}'"
I0520 15:17:58.487691 2701137 start.go:128] duration metric: took 9.046379848s to createHost
I0520 15:17:58.487718 2701137 start.go:83] releasing machines lock for "embed-certs-684040", held for 9.046522491s
I0520 15:17:58.487788 2701137 cli_runner.go:164] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" embed-certs-684040
I0520 15:17:58.504690 2701137 ssh_runner.go:195] Run: cat /version.json
I0520 15:17:58.504748 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:58.504992 2701137 ssh_runner.go:195] Run: curl -sS -m 2 https://registry.k8s.io/
I0520 15:17:58.505028 2701137 cli_runner.go:164] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" embed-certs-684040
I0520 15:17:58.522697 2701137 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41943 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa Username:docker}
I0520 15:17:58.527246 2701137 sshutil.go:53] new ssh client: &{IP:127.0.0.1 Port:41943 SSHKeyPath:/home/jenkins/minikube-integration/18929-2395474/.minikube/machines/embed-certs-684040/id_rsa Username:docker}
I0520 15:17:58.609521 2701137 ssh_runner.go:195] Run: systemctl --version
I0520 15:17:58.727216 2701137 ssh_runner.go:195] Run: sh -c "stat /etc/cni/net.d/*loopback.conf*"
I0520 15:17:58.732237 2701137 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f -name *loopback.conf* -not -name *.mk_disabled -exec sh -c "grep -q loopback {} && ( grep -q name {} || sudo sed -i '/"type": "loopback"/i \ \ \ \ "name": "loopback",' {} ) && sudo sed -i 's|"cniVersion": ".*"|"cniVersion": "1.0.0"|g' {}" ;
I0520 15:17:58.764244 2701137 cni.go:230] loopback cni configuration patched: "/etc/cni/net.d/*loopback.conf*" found
I0520 15:17:58.764330 2701137 ssh_runner.go:195] Run: sudo find /etc/cni/net.d -maxdepth 1 -type f ( ( -name *bridge* -or -name *podman* ) -and -not -name *.mk_disabled ) -printf "%!p(MISSING), " -exec sh -c "sudo mv {} {}.mk_disabled" ;
I0520 15:17:58.796228 2701137 cni.go:262] disabled [/etc/cni/net.d/87-podman-bridge.conflist, /etc/cni/net.d/100-crio-bridge.conf] bridge cni config(s)
I0520 15:17:58.796256 2701137 start.go:494] detecting cgroup driver to use...
I0520 15:17:58.796290 2701137 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0520 15:17:58.796394 2701137 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///run/containerd/containerd.sock
" | sudo tee /etc/crictl.yaml"
I0520 15:17:58.813415 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)sandbox_image = .*$|\1sandbox_image = "registry.k8s.io/pause:3.9"|' /etc/containerd/config.toml"
I0520 15:17:58.823887 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)restrict_oom_score_adj = .*$|\1restrict_oom_score_adj = false|' /etc/containerd/config.toml"
I0520 15:17:58.835725 2701137 containerd.go:146] configuring containerd to use "cgroupfs" as cgroup driver...
I0520 15:17:58.835903 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)SystemdCgroup = .*$|\1SystemdCgroup = false|g' /etc/containerd/config.toml"
I0520 15:17:58.847826 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runtime.v1.linux"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 15:17:58.858845 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i '/systemd_cgroup/d' /etc/containerd/config.toml"
I0520 15:17:58.870108 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i 's|"io.containerd.runc.v1"|"io.containerd.runc.v2"|g' /etc/containerd/config.toml"
I0520 15:17:58.884160 2701137 ssh_runner.go:195] Run: sh -c "sudo rm -rf /etc/cni/net.mk"
I0520 15:17:58.894458 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)conf_dir = .*$|\1conf_dir = "/etc/cni/net.d"|g' /etc/containerd/config.toml"
I0520 15:17:58.904723 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i '/^ *enable_unprivileged_ports = .*/d' /etc/containerd/config.toml"
I0520 15:17:58.915434 2701137 ssh_runner.go:195] Run: sh -c "sudo sed -i -r 's|^( *)\[plugins."io.containerd.grpc.v1.cri"\]|&\n\1 enable_unprivileged_ports = true|' /etc/containerd/config.toml"
I0520 15:17:58.926201 2701137 ssh_runner.go:195] Run: sudo sysctl net.bridge.bridge-nf-call-iptables
I0520 15:17:58.935155 2701137 ssh_runner.go:195] Run: sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
I0520 15:17:58.943809 2701137 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:17:59.041356 2701137 ssh_runner.go:195] Run: sudo systemctl restart containerd
I0520 15:17:59.193351 2701137 start.go:494] detecting cgroup driver to use...
I0520 15:17:59.193400 2701137 detect.go:196] detected "cgroupfs" cgroup driver on host os
I0520 15:17:59.193469 2701137 ssh_runner.go:195] Run: sudo systemctl cat docker.service
I0520 15:17:59.209105 2701137 cruntime.go:279] skipping containerd shutdown because we are bound to it
I0520 15:17:59.209180 2701137 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service crio
I0520 15:17:59.223200 2701137 ssh_runner.go:195] Run: /bin/bash -c "sudo mkdir -p /etc && printf %!s(MISSING) "runtime-endpoint: unix:///var/run/cri-dockerd.sock
" | sudo tee /etc/crictl.yaml"
I0520 15:17:59.244598 2701137 ssh_runner.go:195] Run: which cri-dockerd
I0520 15:17:59.249381 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/cri-docker.service.d
I0520 15:17:59.264508 2701137 ssh_runner.go:362] scp memory --> /etc/systemd/system/cri-docker.service.d/10-cni.conf (189 bytes)
I0520 15:17:59.287285 2701137 ssh_runner.go:195] Run: sudo systemctl unmask docker.service
I0520 15:17:59.398859 2701137 ssh_runner.go:195] Run: sudo systemctl enable docker.socket
I0520 15:17:59.497077 2701137 docker.go:574] configuring docker to use "cgroupfs" as cgroup driver...
I0520 15:17:59.497249 2701137 ssh_runner.go:362] scp memory --> /etc/docker/daemon.json (130 bytes)
I0520 15:17:59.520291 2701137 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:17:59.631443 2701137 ssh_runner.go:195] Run: sudo systemctl restart docker
I0520 15:17:59.915035 2701137 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.socket
I0520 15:17:59.928825 2701137 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0520 15:17:59.944130 2701137 ssh_runner.go:195] Run: sudo systemctl unmask cri-docker.socket
I0520 15:18:00.083566 2701137 ssh_runner.go:195] Run: sudo systemctl enable cri-docker.socket
I0520 15:18:00.337372 2701137 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:18:00.464982 2701137 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.socket
I0520 15:18:00.484164 2701137 ssh_runner.go:195] Run: sudo systemctl is-active --quiet service cri-docker.service
I0520 15:18:00.497995 2701137 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:18:00.604615 2701137 ssh_runner.go:195] Run: sudo systemctl restart cri-docker.service
I0520 15:18:00.680070 2701137 start.go:541] Will wait 60s for socket path /var/run/cri-dockerd.sock
I0520 15:18:00.680173 2701137 ssh_runner.go:195] Run: stat /var/run/cri-dockerd.sock
I0520 15:18:00.685461 2701137 start.go:562] Will wait 60s for crictl version
I0520 15:18:00.685557 2701137 ssh_runner.go:195] Run: which crictl
I0520 15:18:00.689988 2701137 ssh_runner.go:195] Run: sudo /usr/bin/crictl version
I0520 15:18:00.734513 2701137 start.go:578] Version: 0.1.0
RuntimeName: docker
RuntimeVersion: 26.1.2
RuntimeApiVersion: v1
I0520 15:18:00.734630 2701137 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0520 15:18:00.761117 2701137 ssh_runner.go:195] Run: docker version --format {{.Server.Version}}
I0520 15:17:57.256085 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:17:59.256773 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:01.257463 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:00.786788 2701137 out.go:204] * Preparing Kubernetes v1.30.1 on Docker 26.1.2 ...
I0520 15:18:00.786912 2701137 cli_runner.go:164] Run: docker network inspect embed-certs-684040 --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}}, "ContainerIPs": [{{range $k,$v := .Containers }}"{{$v.IPv4Address}}",{{end}}]}"
I0520 15:18:00.801402 2701137 ssh_runner.go:195] Run: grep 192.168.85.1 host.minikube.internal$ /etc/hosts
I0520 15:18:00.805371 2701137 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\thost.minikube.internal$' "/etc/hosts"; echo "192.168.85.1 host.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0520 15:18:00.816614 2701137 kubeadm.go:877] updating cluster {Name:embed-certs-684040 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:embed-certs-684040 Namespace:default APIServerHAVIP: APIServerName:minikubeCA AP
IServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false Cu
stomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s} ...
I0520 15:18:00.816741 2701137 preload.go:132] Checking if preload exists for k8s version v1.30.1 and runtime docker
I0520 15:18:00.816802 2701137 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0520 15:18:00.833367 2701137 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.30.1
registry.k8s.io/kube-scheduler:v1.30.1
registry.k8s.io/kube-controller-manager:v1.30.1
registry.k8s.io/kube-proxy:v1.30.1
registry.k8s.io/etcd:3.5.12-0
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/pause:3.9
gcr.io/k8s-minikube/storage-provisioner:v5
-- /stdout --
I0520 15:18:00.833387 2701137 docker.go:615] Images already preloaded, skipping extraction
I0520 15:18:00.833451 2701137 ssh_runner.go:195] Run: docker images --format {{.Repository}}:{{.Tag}}
I0520 15:18:00.851866 2701137 docker.go:685] Got preloaded images: -- stdout --
registry.k8s.io/kube-apiserver:v1.30.1
registry.k8s.io/kube-scheduler:v1.30.1
registry.k8s.io/kube-controller-manager:v1.30.1
registry.k8s.io/kube-proxy:v1.30.1
registry.k8s.io/etcd:3.5.12-0
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/pause:3.9
gcr.io/k8s-minikube/storage-provisioner:v5
-- /stdout --
I0520 15:18:00.851892 2701137 cache_images.go:84] Images are preloaded, skipping loading
I0520 15:18:00.851912 2701137 kubeadm.go:928] updating node { 192.168.85.2 8443 v1.30.1 docker true true} ...
I0520 15:18:00.852017 2701137 kubeadm.go:940] kubelet [Unit]
Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.30.1/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --config=/var/lib/kubelet/config.yaml --hostname-override=embed-certs-684040 --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.85.2
[Install]
config:
{KubernetesVersion:v1.30.1 ClusterName:embed-certs-684040 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:}
I0520 15:18:00.852098 2701137 ssh_runner.go:195] Run: docker info --format {{.CgroupDriver}}
I0520 15:18:00.904130 2701137 cni.go:84] Creating CNI manager for ""
I0520 15:18:00.904160 2701137 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I0520 15:18:00.904171 2701137 kubeadm.go:84] Using pod CIDR: 10.244.0.0/16
I0520 15:18:00.904191 2701137 kubeadm.go:181] kubeadm options: {CertDir:/var/lib/minikube/certs ServiceCIDR:10.96.0.0/12 PodSubnet:10.244.0.0/16 AdvertiseAddress:192.168.85.2 APIServerPort:8443 KubernetesVersion:v1.30.1 EtcdDataDir:/var/lib/minikube/etcd EtcdExtraArgs:map[] ClusterName:embed-certs-684040 NodeName:embed-certs-684040 DNSDomain:cluster.local CRISocket:/var/run/cri-dockerd.sock ImageRepository: ComponentOptions:[{Component:apiServer ExtraArgs:map[enable-admission-plugins:NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota] Pairs:map[certSANs:["127.0.0.1", "localhost", "192.168.85.2"]]} {Component:controllerManager ExtraArgs:map[allocate-node-cidrs:true leader-elect:false] Pairs:map[]} {Component:scheduler ExtraArgs:map[leader-elect:false] Pairs:map[]}] FeatureArgs:map[] NodeIP:192.168.85.2 CgroupDriver:cgroupfs ClientCAFile:/var/lib/minikube/certs/ca.crt StaticPodPath:
/etc/kubernetes/manifests ControlPlaneAddress:control-plane.minikube.internal KubeProxyOptions:map[] ResolvConfSearchRegression:false KubeletConfigOpts:map[containerRuntimeEndpoint:unix:///var/run/cri-dockerd.sock hairpinMode:hairpin-veth runtimeRequestTimeout:15m] PrependCriSocketUnix:true}
I0520 15:18:00.904340 2701137 kubeadm.go:187] kubeadm config:
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.85.2
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: unix:///var/run/cri-dockerd.sock
name: "embed-certs-684040"
kubeletExtraArgs:
node-ip: 192.168.85.2
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
certSANs: ["127.0.0.1", "localhost", "192.168.85.2"]
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
controllerManager:
extraArgs:
allocate-node-cidrs: "true"
leader-elect: "false"
scheduler:
extraArgs:
leader-elect: "false"
certificatesDir: /var/lib/minikube/certs
clusterName: mk
controlPlaneEndpoint: control-plane.minikube.internal:8443
etcd:
local:
dataDir: /var/lib/minikube/etcd
extraArgs:
proxy-refresh-interval: "70000"
kubernetesVersion: v1.30.1
networking:
dnsDomain: cluster.local
podSubnet: "10.244.0.0/16"
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
x509:
clientCAFile: /var/lib/minikube/certs/ca.crt
cgroupDriver: cgroupfs
containerRuntimeEndpoint: unix:///var/run/cri-dockerd.sock
hairpinMode: hairpin-veth
runtimeRequestTimeout: 15m
clusterDomain: "cluster.local"
# disable disk resource management by default
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%!"(MISSING)
nodefs.inodesFree: "0%!"(MISSING)
imagefs.available: "0%!"(MISSING)
failSwapOn: false
staticPodPath: /etc/kubernetes/manifests
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
clusterCIDR: "10.244.0.0/16"
metricsBindAddress: 0.0.0.0:10249
conntrack:
maxPerCore: 0
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_established"
tcpEstablishedTimeout: 0s
# Skip setting "net.netfilter.nf_conntrack_tcp_timeout_close"
tcpCloseWaitTimeout: 0s
I0520 15:18:00.904417 2701137 ssh_runner.go:195] Run: sudo ls /var/lib/minikube/binaries/v1.30.1
I0520 15:18:00.913826 2701137 binaries.go:44] Found k8s binaries, skipping transfer
I0520 15:18:00.913899 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/systemd/system/kubelet.service.d /lib/systemd/system /var/tmp/minikube
I0520 15:18:00.922992 2701137 ssh_runner.go:362] scp memory --> /etc/systemd/system/kubelet.service.d/10-kubeadm.conf (317 bytes)
I0520 15:18:00.943407 2701137 ssh_runner.go:362] scp memory --> /lib/systemd/system/kubelet.service (352 bytes)
I0520 15:18:00.963340 2701137 ssh_runner.go:362] scp memory --> /var/tmp/minikube/kubeadm.yaml.new (2160 bytes)
I0520 15:18:00.982837 2701137 ssh_runner.go:195] Run: grep 192.168.85.2 control-plane.minikube.internal$ /etc/hosts
I0520 15:18:00.986379 2701137 ssh_runner.go:195] Run: /bin/bash -c "{ grep -v $'\tcontrol-plane.minikube.internal$' "/etc/hosts"; echo "192.168.85.2 control-plane.minikube.internal"; } > /tmp/h.$$; sudo cp /tmp/h.$$ "/etc/hosts""
I0520 15:18:00.997808 2701137 ssh_runner.go:195] Run: sudo systemctl daemon-reload
I0520 15:18:01.093356 2701137 ssh_runner.go:195] Run: sudo systemctl start kubelet
I0520 15:18:01.110953 2701137 certs.go:68] Setting up /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040 for IP: 192.168.85.2
I0520 15:18:01.110976 2701137 certs.go:194] generating shared ca certs ...
I0520 15:18:01.111014 2701137 certs.go:226] acquiring lock for ca certs: {Name:mkb3ccf41a209540ed3793a4984e70a3a08868cc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:01.111173 2701137 certs.go:235] skipping valid "minikubeCA" ca cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.key
I0520 15:18:01.111239 2701137 certs.go:235] skipping valid "proxyClientCA" ca cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.key
I0520 15:18:01.111254 2701137 certs.go:256] generating profile certs ...
I0520 15:18:01.111329 2701137 certs.go:363] generating signed profile cert for "minikube-user": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.key
I0520 15:18:01.111351 2701137 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.crt with IP's: []
I0520 15:18:01.712384 2701137 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.crt ...
I0520 15:18:01.712418 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.crt: {Name:mk3c88772f18d37648dd02a2eab38b5dd5a2b645 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:01.713156 2701137 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.key ...
I0520 15:18:01.713182 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/client.key: {Name:mk7f6818a0aee6904964b70d7fe905a74b36ab15 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:01.713331 2701137 certs.go:363] generating signed profile cert for "minikube": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key.83362166
I0520 15:18:01.713361 2701137 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt.83362166 with IP's: [10.96.0.1 127.0.0.1 10.0.0.1 192.168.85.2]
I0520 15:18:02.131061 2701137 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt.83362166 ...
I0520 15:18:02.131103 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt.83362166: {Name:mk756ac8bb02971a142db0cb5f83ceb98e3ed3ea Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:02.131914 2701137 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key.83362166 ...
I0520 15:18:02.131938 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key.83362166: {Name:mk84fb3758c90ccaf8ae0c5ac6b6c294d651e9b4 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:02.132044 2701137 certs.go:381] copying /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt.83362166 -> /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt
I0520 15:18:02.132130 2701137 certs.go:385] copying /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key.83362166 -> /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key
I0520 15:18:02.132194 2701137 certs.go:363] generating signed profile cert for "aggregator": /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.key
I0520 15:18:02.132217 2701137 crypto.go:68] Generating cert /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.crt with IP's: []
I0520 15:18:03.155347 2701137 crypto.go:156] Writing cert to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.crt ...
I0520 15:18:03.155384 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.crt: {Name:mkb37186d2726c9d53933245ea8a41607915def9 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:03.155589 2701137 crypto.go:164] Writing key to /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.key ...
I0520 15:18:03.155606 2701137 lock.go:35] WriteFile acquiring /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.key: {Name:mkd4810335a52b78f4b6b6a1959381906de7bd51 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0520 15:18:03.156184 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908.pem (1338 bytes)
W0520 15:18:03.156234 2701137 certs.go:480] ignoring /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908_empty.pem, impossibly tiny 0 bytes
I0520 15:18:03.156249 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca-key.pem (1675 bytes)
I0520 15:18:03.156274 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/ca.pem (1082 bytes)
I0520 15:18:03.156304 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/cert.pem (1123 bytes)
I0520 15:18:03.156330 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/key.pem (1679 bytes)
I0520 15:18:03.156379 2701137 certs.go:484] found cert: /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem (1708 bytes)
I0520 15:18:03.157048 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.crt --> /var/lib/minikube/certs/ca.crt (1111 bytes)
I0520 15:18:03.186245 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.key --> /var/lib/minikube/certs/ca.key (1679 bytes)
I0520 15:18:03.214146 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.crt --> /var/lib/minikube/certs/proxy-client-ca.crt (1119 bytes)
I0520 15:18:03.239877 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/proxy-client-ca.key --> /var/lib/minikube/certs/proxy-client-ca.key (1675 bytes)
I0520 15:18:03.269307 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.crt --> /var/lib/minikube/certs/apiserver.crt (1428 bytes)
I0520 15:18:03.295346 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/apiserver.key --> /var/lib/minikube/certs/apiserver.key (1679 bytes)
I0520 15:18:03.321616 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.crt --> /var/lib/minikube/certs/proxy-client.crt (1147 bytes)
I0520 15:18:03.349013 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/profiles/embed-certs-684040/proxy-client.key --> /var/lib/minikube/certs/proxy-client.key (1675 bytes)
I0520 15:18:03.377792 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/certs/2400908.pem --> /usr/share/ca-certificates/2400908.pem (1338 bytes)
I0520 15:18:03.404719 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/files/etc/ssl/certs/24009082.pem --> /usr/share/ca-certificates/24009082.pem (1708 bytes)
I0520 15:18:03.435278 2701137 ssh_runner.go:362] scp /home/jenkins/minikube-integration/18929-2395474/.minikube/ca.crt --> /usr/share/ca-certificates/minikubeCA.pem (1111 bytes)
I0520 15:18:03.460880 2701137 ssh_runner.go:362] scp memory --> /var/lib/minikube/kubeconfig (738 bytes)
I0520 15:18:03.479770 2701137 ssh_runner.go:195] Run: openssl version
I0520 15:18:03.485325 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/minikubeCA.pem && ln -fs /usr/share/ca-certificates/minikubeCA.pem /etc/ssl/certs/minikubeCA.pem"
I0520 15:18:03.495185 2701137 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/minikubeCA.pem
I0520 15:18:03.500106 2701137 certs.go:528] hashing: -rw-r--r-- 1 root root 1111 May 20 14:18 /usr/share/ca-certificates/minikubeCA.pem
I0520 15:18:03.500180 2701137 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/minikubeCA.pem
I0520 15:18:03.507388 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/b5213941.0 || ln -fs /etc/ssl/certs/minikubeCA.pem /etc/ssl/certs/b5213941.0"
I0520 15:18:03.517205 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/2400908.pem && ln -fs /usr/share/ca-certificates/2400908.pem /etc/ssl/certs/2400908.pem"
I0520 15:18:03.526630 2701137 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/2400908.pem
I0520 15:18:03.530162 2701137 certs.go:528] hashing: -rw-r--r-- 1 root root 1338 May 20 14:24 /usr/share/ca-certificates/2400908.pem
I0520 15:18:03.530245 2701137 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/2400908.pem
I0520 15:18:03.538161 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/51391683.0 || ln -fs /etc/ssl/certs/2400908.pem /etc/ssl/certs/51391683.0"
I0520 15:18:03.547682 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -s /usr/share/ca-certificates/24009082.pem && ln -fs /usr/share/ca-certificates/24009082.pem /etc/ssl/certs/24009082.pem"
I0520 15:18:03.557542 2701137 ssh_runner.go:195] Run: ls -la /usr/share/ca-certificates/24009082.pem
I0520 15:18:03.561155 2701137 certs.go:528] hashing: -rw-r--r-- 1 root root 1708 May 20 14:24 /usr/share/ca-certificates/24009082.pem
I0520 15:18:03.561220 2701137 ssh_runner.go:195] Run: openssl x509 -hash -noout -in /usr/share/ca-certificates/24009082.pem
I0520 15:18:03.568220 2701137 ssh_runner.go:195] Run: sudo /bin/bash -c "test -L /etc/ssl/certs/3ec20f2e.0 || ln -fs /etc/ssl/certs/24009082.pem /etc/ssl/certs/3ec20f2e.0"
I0520 15:18:03.578075 2701137 ssh_runner.go:195] Run: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt
I0520 15:18:03.581362 2701137 certs.go:399] 'apiserver-kubelet-client' cert doesn't exist, likely first start: stat /var/lib/minikube/certs/apiserver-kubelet-client.crt: Process exited with status 1
stdout:
stderr:
stat: cannot statx '/var/lib/minikube/certs/apiserver-kubelet-client.crt': No such file or directory
I0520 15:18:03.581416 2701137 kubeadm.go:391] StartCluster: {Name:embed-certs-684040 KeepContext:false EmbedCerts:true MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1715707529-18887@sha256:734a2adad159415bd51b0392bc8976bd7e3e543e9165c2374d6e59bac37aed3a Memory:2200 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.1 ClusterName:embed-certs-684040 Namespace:default APIServerHAVIP: APIServerName:minikubeCA APISe
rverNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:192.168.85.2 Port:8443 KubernetesVersion:v1.30.1 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true apps_running:true default_sa:true extra:true kubelet:true node_ready:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/home/jenkins:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false Custo
mQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0520 15:18:03.581553 2701137 ssh_runner.go:195] Run: docker ps --filter status=paused --filter=name=k8s_.*_(kube-system)_ --format={{.ID}}
I0520 15:18:03.597131 2701137 ssh_runner.go:195] Run: sudo ls /var/lib/kubelet/kubeadm-flags.env /var/lib/kubelet/config.yaml /var/lib/minikube/etcd
I0520 15:18:03.606273 2701137 ssh_runner.go:195] Run: sudo cp /var/tmp/minikube/kubeadm.yaml.new /var/tmp/minikube/kubeadm.yaml
I0520 15:18:03.615560 2701137 kubeadm.go:213] ignoring SystemVerification for kubeadm because of docker driver
I0520 15:18:03.615741 2701137 ssh_runner.go:195] Run: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf
I0520 15:18:03.625596 2701137 kubeadm.go:154] config check failed, skipping stale config cleanup: sudo ls -la /etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
ls: cannot access '/etc/kubernetes/admin.conf': No such file or directory
ls: cannot access '/etc/kubernetes/kubelet.conf': No such file or directory
ls: cannot access '/etc/kubernetes/controller-manager.conf': No such file or directory
ls: cannot access '/etc/kubernetes/scheduler.conf': No such file or directory
I0520 15:18:03.625616 2701137 kubeadm.go:156] found existing configuration files:
I0520 15:18:03.625731 2701137 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf
I0520 15:18:03.634637 2701137 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/admin.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/admin.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/admin.conf: No such file or directory
I0520 15:18:03.634736 2701137 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/admin.conf
I0520 15:18:03.643507 2701137 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf
I0520 15:18:03.652465 2701137 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/kubelet.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/kubelet.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/kubelet.conf: No such file or directory
I0520 15:18:03.652557 2701137 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/kubelet.conf
I0520 15:18:03.661251 2701137 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf
I0520 15:18:03.670462 2701137 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/controller-manager.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/controller-manager.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/controller-manager.conf: No such file or directory
I0520 15:18:03.670560 2701137 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/controller-manager.conf
I0520 15:18:03.679024 2701137 ssh_runner.go:195] Run: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf
I0520 15:18:03.688235 2701137 kubeadm.go:162] "https://control-plane.minikube.internal:8443" may not be in /etc/kubernetes/scheduler.conf - will remove: sudo grep https://control-plane.minikube.internal:8443 /etc/kubernetes/scheduler.conf: Process exited with status 2
stdout:
stderr:
grep: /etc/kubernetes/scheduler.conf: No such file or directory
I0520 15:18:03.688301 2701137 ssh_runner.go:195] Run: sudo rm -f /etc/kubernetes/scheduler.conf
I0520 15:18:03.697357 2701137 ssh_runner.go:286] Start: /bin/bash -c "sudo env PATH="/var/lib/minikube/binaries/v1.30.1:$PATH" kubeadm init --config /var/tmp/minikube/kubeadm.yaml --ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests,DirAvailable--var-lib-minikube,DirAvailable--var-lib-minikube-etcd,FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml,FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml,FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml,FileAvailable--etc-kubernetes-manifests-etcd.yaml,Port-10250,Swap,NumCPU,Mem,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables"
I0520 15:18:03.744603 2701137 kubeadm.go:309] [init] Using Kubernetes version: v1.30.1
I0520 15:18:03.744665 2701137 kubeadm.go:309] [preflight] Running pre-flight checks
I0520 15:18:03.790152 2701137 kubeadm.go:309] [preflight] The system verification failed. Printing the output from the verification:
I0520 15:18:03.790286 2701137 kubeadm.go:309] [0;37mKERNEL_VERSION[0m: [0;32m5.15.0-1058-aws[0m
I0520 15:18:03.790352 2701137 kubeadm.go:309] [0;37mOS[0m: [0;32mLinux[0m
I0520 15:18:03.790425 2701137 kubeadm.go:309] [0;37mCGROUPS_CPU[0m: [0;32menabled[0m
I0520 15:18:03.790502 2701137 kubeadm.go:309] [0;37mCGROUPS_CPUACCT[0m: [0;32menabled[0m
I0520 15:18:03.790572 2701137 kubeadm.go:309] [0;37mCGROUPS_CPUSET[0m: [0;32menabled[0m
I0520 15:18:03.790651 2701137 kubeadm.go:309] [0;37mCGROUPS_DEVICES[0m: [0;32menabled[0m
I0520 15:18:03.790721 2701137 kubeadm.go:309] [0;37mCGROUPS_FREEZER[0m: [0;32menabled[0m
I0520 15:18:03.790800 2701137 kubeadm.go:309] [0;37mCGROUPS_MEMORY[0m: [0;32menabled[0m
I0520 15:18:03.790865 2701137 kubeadm.go:309] [0;37mCGROUPS_PIDS[0m: [0;32menabled[0m
I0520 15:18:03.790941 2701137 kubeadm.go:309] [0;37mCGROUPS_HUGETLB[0m: [0;32menabled[0m
I0520 15:18:03.791010 2701137 kubeadm.go:309] [0;37mCGROUPS_BLKIO[0m: [0;32menabled[0m
I0520 15:18:03.866855 2701137 kubeadm.go:309] [preflight] Pulling images required for setting up a Kubernetes cluster
I0520 15:18:03.867003 2701137 kubeadm.go:309] [preflight] This might take a minute or two, depending on the speed of your internet connection
I0520 15:18:03.867120 2701137 kubeadm.go:309] [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
I0520 15:18:04.188237 2701137 kubeadm.go:309] [certs] Using certificateDir folder "/var/lib/minikube/certs"
I0520 15:18:04.193456 2701137 out.go:204] - Generating certificates and keys ...
I0520 15:18:04.193637 2701137 kubeadm.go:309] [certs] Using existing ca certificate authority
I0520 15:18:04.193756 2701137 kubeadm.go:309] [certs] Using existing apiserver certificate and key on disk
I0520 15:18:03.757342 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:06.257208 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:04.937579 2701137 kubeadm.go:309] [certs] Generating "apiserver-kubelet-client" certificate and key
I0520 15:18:05.307709 2701137 kubeadm.go:309] [certs] Generating "front-proxy-ca" certificate and key
I0520 15:18:05.714799 2701137 kubeadm.go:309] [certs] Generating "front-proxy-client" certificate and key
I0520 15:18:06.687501 2701137 kubeadm.go:309] [certs] Generating "etcd/ca" certificate and key
I0520 15:18:07.726068 2701137 kubeadm.go:309] [certs] Generating "etcd/server" certificate and key
I0520 15:18:07.726201 2701137 kubeadm.go:309] [certs] etcd/server serving cert is signed for DNS names [embed-certs-684040 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0520 15:18:08.546231 2701137 kubeadm.go:309] [certs] Generating "etcd/peer" certificate and key
I0520 15:18:08.546729 2701137 kubeadm.go:309] [certs] etcd/peer serving cert is signed for DNS names [embed-certs-684040 localhost] and IPs [192.168.85.2 127.0.0.1 ::1]
I0520 15:18:08.861270 2701137 kubeadm.go:309] [certs] Generating "etcd/healthcheck-client" certificate and key
I0520 15:18:08.258452 2687927 pod_ready.go:102] pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace has status "Ready":"False"
I0520 15:18:10.256495 2687927 pod_ready.go:81] duration metric: took 4m0.007137082s for pod "metrics-server-9975d5f86-sks2q" in "kube-system" namespace to be "Ready" ...
E0520 15:18:10.256524 2687927 pod_ready.go:66] WaitExtra: waitPodCondition: context deadline exceeded
I0520 15:18:10.256535 2687927 pod_ready.go:38] duration metric: took 5m25.143545959s for extra waiting for all system-critical and pods with labels [k8s-app=kube-dns component=etcd component=kube-apiserver component=kube-controller-manager k8s-app=kube-proxy component=kube-scheduler] to be "Ready" ...
I0520 15:18:10.256554 2687927 api_server.go:52] waiting for apiserver process to appear ...
I0520 15:18:10.256637 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0520 15:18:10.283268 2687927 logs.go:276] 2 containers: [7b906f9c7e42 21cda61304ec]
I0520 15:18:10.283354 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0520 15:18:10.302345 2687927 logs.go:276] 2 containers: [4183e6237564 78716dd9f5e6]
I0520 15:18:10.302441 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0520 15:18:10.329708 2687927 logs.go:276] 2 containers: [57e027c8e0da 5876807206c9]
I0520 15:18:10.329805 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0520 15:18:10.356545 2687927 logs.go:276] 2 containers: [20ae03428247 798f34afc547]
I0520 15:18:10.356646 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0520 15:18:10.378250 2687927 logs.go:276] 2 containers: [75eb680f6808 36b9b4fb305d]
I0520 15:18:10.378352 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0520 15:18:10.406769 2687927 logs.go:276] 2 containers: [18ca6dabc240 7c325beb8903]
I0520 15:18:10.406878 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0520 15:18:10.424347 2687927 logs.go:276] 0 containers: []
W0520 15:18:10.424373 2687927 logs.go:278] No container was found matching "kindnet"
I0520 15:18:10.424462 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0520 15:18:10.446113 2687927 logs.go:276] 2 containers: [ee9aab3e1ca5 623598d9bb43]
I0520 15:18:10.446197 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0520 15:18:10.465324 2687927 logs.go:276] 1 containers: [e9d80f29a80f]
I0520 15:18:10.465359 2687927 logs.go:123] Gathering logs for kubelet ...
I0520 15:18:10.465373 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0520 15:18:10.547614 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076509 1198 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.547881 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076589 1198 reflector.go:138] object-"kube-system"/"kube-proxy-token-4znn8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-4znn8" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.548103 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076649 1198 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.548320 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076901 1198 reflector.go:138] object-"kube-system"/"coredns-token-rk862": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-rk862" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.549299 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.252268 1198 reflector.go:138] object-"default"/"default-token-kj425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-kj425" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.549534 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.263194 1198 reflector.go:138] object-"kube-system"/"storage-provisioner-token-rpfqn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-rpfqn" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:10.559239 2687927 logs.go:138] Found kubelet problem: May 20 15:12:46 old-k8s-version-185009 kubelet[1198]: E0520 15:12:46.633160 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.559629 2687927 logs.go:138] Found kubelet problem: May 20 15:12:47 old-k8s-version-185009 kubelet[1198]: E0520 15:12:47.647957 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.560622 2687927 logs.go:138] Found kubelet problem: May 20 15:12:48 old-k8s-version-185009 kubelet[1198]: E0520 15:12:48.926924 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.563115 2687927 logs.go:138] Found kubelet problem: May 20 15:13:00 old-k8s-version-185009 kubelet[1198]: E0520 15:13:00.334303 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.567886 2687927 logs.go:138] Found kubelet problem: May 20 15:13:05 old-k8s-version-185009 kubelet[1198]: E0520 15:13:05.302095 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.568104 2687927 logs.go:138] Found kubelet problem: May 20 15:13:06 old-k8s-version-185009 kubelet[1198]: E0520 15:13:06.152746 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.568296 2687927 logs.go:138] Found kubelet problem: May 20 15:13:11 old-k8s-version-185009 kubelet[1198]: E0520 15:13:11.129523 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.569266 2687927 logs.go:138] Found kubelet problem: May 20 15:13:18 old-k8s-version-185009 kubelet[1198]: E0520 15:13:18.383550 1198 pod_workers.go:191] Error syncing pod 55c63b6d-7e3d-4491-9a29-4bc83a4d54fa ("storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"
W0520 15:18:10.571561 2687927 logs.go:138] Found kubelet problem: May 20 15:13:19 old-k8s-version-185009 kubelet[1198]: E0520 15:13:19.635684 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.573759 2687927 logs.go:138] Found kubelet problem: May 20 15:13:23 old-k8s-version-185009 kubelet[1198]: E0520 15:13:23.158433 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.574313 2687927 logs.go:138] Found kubelet problem: May 20 15:13:32 old-k8s-version-185009 kubelet[1198]: E0520 15:13:32.119072 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.574640 2687927 logs.go:138] Found kubelet problem: May 20 15:13:35 old-k8s-version-185009 kubelet[1198]: E0520 15:13:35.124065 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577236 2687927 logs.go:138] Found kubelet problem: May 20 15:13:43 old-k8s-version-185009 kubelet[1198]: E0520 15:13:43.578964 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.577454 2687927 logs.go:138] Found kubelet problem: May 20 15:13:47 old-k8s-version-185009 kubelet[1198]: E0520 15:13:47.118959 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577738 2687927 logs.go:138] Found kubelet problem: May 20 15:13:58 old-k8s-version-185009 kubelet[1198]: E0520 15:13:58.119337 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.577940 2687927 logs.go:138] Found kubelet problem: May 20 15:14:01 old-k8s-version-185009 kubelet[1198]: E0520 15:14:01.118790 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.578455 2687927 logs.go:138] Found kubelet problem: May 20 15:14:10 old-k8s-version-185009 kubelet[1198]: E0520 15:14:10.148705 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.580731 2687927 logs.go:138] Found kubelet problem: May 20 15:14:14 old-k8s-version-185009 kubelet[1198]: E0520 15:14:14.139015 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.583477 2687927 logs.go:138] Found kubelet problem: May 20 15:14:25 old-k8s-version-185009 kubelet[1198]: E0520 15:14:25.575698 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.583771 2687927 logs.go:138] Found kubelet problem: May 20 15:14:28 old-k8s-version-185009 kubelet[1198]: E0520 15:14:28.126943 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584099 2687927 logs.go:138] Found kubelet problem: May 20 15:14:38 old-k8s-version-185009 kubelet[1198]: E0520 15:14:38.118687 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584312 2687927 logs.go:138] Found kubelet problem: May 20 15:14:42 old-k8s-version-185009 kubelet[1198]: E0520 15:14:42.119209 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584520 2687927 logs.go:138] Found kubelet problem: May 20 15:14:53 old-k8s-version-185009 kubelet[1198]: E0520 15:14:53.121794 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584720 2687927 logs.go:138] Found kubelet problem: May 20 15:14:56 old-k8s-version-185009 kubelet[1198]: E0520 15:14:56.118640 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.584936 2687927 logs.go:138] Found kubelet problem: May 20 15:15:08 old-k8s-version-185009 kubelet[1198]: E0520 15:15:08.118744 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585137 2687927 logs.go:138] Found kubelet problem: May 20 15:15:11 old-k8s-version-185009 kubelet[1198]: E0520 15:15:11.125539 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585333 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.119290 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.585557 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.129546 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.586072 2687927 logs.go:138] Found kubelet problem: May 20 15:15:34 old-k8s-version-185009 kubelet[1198]: E0520 15:15:34.118580 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.588388 2687927 logs.go:138] Found kubelet problem: May 20 15:15:35 old-k8s-version-185009 kubelet[1198]: E0520 15:15:35.142356 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:10.588591 2687927 logs.go:138] Found kubelet problem: May 20 15:15:47 old-k8s-version-185009 kubelet[1198]: E0520 15:15:47.118833 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591221 2687927 logs.go:138] Found kubelet problem: May 20 15:15:49 old-k8s-version-185009 kubelet[1198]: E0520 15:15:49.579168 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:10.591436 2687927 logs.go:138] Found kubelet problem: May 20 15:15:58 old-k8s-version-185009 kubelet[1198]: E0520 15:15:58.119002 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591673 2687927 logs.go:138] Found kubelet problem: May 20 15:16:03 old-k8s-version-185009 kubelet[1198]: E0520 15:16:03.129847 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.591871 2687927 logs.go:138] Found kubelet problem: May 20 15:16:09 old-k8s-version-185009 kubelet[1198]: E0520 15:16:09.118709 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592081 2687927 logs.go:138] Found kubelet problem: May 20 15:16:15 old-k8s-version-185009 kubelet[1198]: E0520 15:16:15.140673 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592292 2687927 logs.go:138] Found kubelet problem: May 20 15:16:22 old-k8s-version-185009 kubelet[1198]: E0520 15:16:22.118599 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592517 2687927 logs.go:138] Found kubelet problem: May 20 15:16:26 old-k8s-version-185009 kubelet[1198]: E0520 15:16:26.125485 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592724 2687927 logs.go:138] Found kubelet problem: May 20 15:16:35 old-k8s-version-185009 kubelet[1198]: E0520 15:16:35.118669 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.592937 2687927 logs.go:138] Found kubelet problem: May 20 15:16:37 old-k8s-version-185009 kubelet[1198]: E0520 15:16:37.119240 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593148 2687927 logs.go:138] Found kubelet problem: May 20 15:16:49 old-k8s-version-185009 kubelet[1198]: E0520 15:16:49.118660 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593456 2687927 logs.go:138] Found kubelet problem: May 20 15:16:51 old-k8s-version-185009 kubelet[1198]: E0520 15:16:51.118889 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.593781 2687927 logs.go:138] Found kubelet problem: May 20 15:17:01 old-k8s-version-185009 kubelet[1198]: E0520 15:17:01.131117 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594012 2687927 logs.go:138] Found kubelet problem: May 20 15:17:02 old-k8s-version-185009 kubelet[1198]: E0520 15:17:02.119112 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594271 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118969 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594481 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118971 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594687 2687927 logs.go:138] Found kubelet problem: May 20 15:17:25 old-k8s-version-185009 kubelet[1198]: E0520 15:17:25.119102 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.594902 2687927 logs.go:138] Found kubelet problem: May 20 15:17:27 old-k8s-version-185009 kubelet[1198]: E0520 15:17:27.141000 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595111 2687927 logs.go:138] Found kubelet problem: May 20 15:17:37 old-k8s-version-185009 kubelet[1198]: E0520 15:17:37.118917 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595323 2687927 logs.go:138] Found kubelet problem: May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595526 2687927 logs.go:138] Found kubelet problem: May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595751 2687927 logs.go:138] Found kubelet problem: May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.595966 2687927 logs.go:138] Found kubelet problem: May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:10.596194 2687927 logs.go:138] Found kubelet problem: May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:10.596208 2687927 logs.go:123] Gathering logs for kube-apiserver [7b906f9c7e42] ...
I0520 15:18:10.596221 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b906f9c7e42"
I0520 15:18:10.656241 2687927 logs.go:123] Gathering logs for kube-scheduler [798f34afc547] ...
I0520 15:18:10.656300 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 798f34afc547"
I0520 15:18:10.702047 2687927 logs.go:123] Gathering logs for kube-controller-manager [18ca6dabc240] ...
I0520 15:18:10.702088 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 18ca6dabc240"
I0520 15:18:10.778491 2687927 logs.go:123] Gathering logs for storage-provisioner [ee9aab3e1ca5] ...
I0520 15:18:10.778536 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 ee9aab3e1ca5"
I0520 15:18:10.810518 2687927 logs.go:123] Gathering logs for storage-provisioner [623598d9bb43] ...
I0520 15:18:10.810557 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 623598d9bb43"
I0520 15:18:10.842415 2687927 logs.go:123] Gathering logs for dmesg ...
I0520 15:18:10.842455 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0520 15:18:10.874520 2687927 logs.go:123] Gathering logs for etcd [4183e6237564] ...
I0520 15:18:10.874572 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4183e6237564"
I0520 15:18:10.918596 2687927 logs.go:123] Gathering logs for kubernetes-dashboard [e9d80f29a80f] ...
I0520 15:18:10.918666 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 e9d80f29a80f"
I0520 15:18:10.949484 2687927 logs.go:123] Gathering logs for kube-apiserver [21cda61304ec] ...
I0520 15:18:10.949565 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 21cda61304ec"
I0520 15:18:11.042227 2687927 logs.go:123] Gathering logs for coredns [57e027c8e0da] ...
I0520 15:18:11.042306 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 57e027c8e0da"
I0520 15:18:11.067073 2687927 logs.go:123] Gathering logs for coredns [5876807206c9] ...
I0520 15:18:11.067098 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 5876807206c9"
I0520 15:18:11.098711 2687927 logs.go:123] Gathering logs for container status ...
I0520 15:18:11.098814 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0520 15:18:11.167370 2687927 logs.go:123] Gathering logs for describe nodes ...
I0520 15:18:11.167517 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0520 15:18:11.349855 2687927 logs.go:123] Gathering logs for etcd [78716dd9f5e6] ...
I0520 15:18:11.349928 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 78716dd9f5e6"
I0520 15:18:11.395012 2687927 logs.go:123] Gathering logs for kube-scheduler [20ae03428247] ...
I0520 15:18:11.395091 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 20ae03428247"
I0520 15:18:11.436527 2687927 logs.go:123] Gathering logs for kube-proxy [75eb680f6808] ...
I0520 15:18:11.436579 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 75eb680f6808"
I0520 15:18:11.472843 2687927 logs.go:123] Gathering logs for kube-proxy [36b9b4fb305d] ...
I0520 15:18:11.472886 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 36b9b4fb305d"
I0520 15:18:11.500352 2687927 logs.go:123] Gathering logs for kube-controller-manager [7c325beb8903] ...
I0520 15:18:11.500394 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7c325beb8903"
I0520 15:18:11.565557 2687927 logs.go:123] Gathering logs for Docker ...
I0520 15:18:11.565612 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0520 15:18:11.597615 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:11.597661 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0520 15:18:11.597729 2687927 out.go:239] X Problems detected in kubelet:
W0520 15:18:11.597744 2687927 out.go:239] May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597760 2687927 out.go:239] May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597767 2687927 out.go:239] May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597775 2687927 out.go:239] May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:11.597929 2687927 out.go:239] May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:11.597943 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:11.597951 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:18:09.426020 2701137 kubeadm.go:309] [certs] Generating "apiserver-etcd-client" certificate and key
I0520 15:18:09.707531 2701137 kubeadm.go:309] [certs] Generating "sa" key and public key
I0520 15:18:09.708093 2701137 kubeadm.go:309] [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
I0520 15:18:10.155128 2701137 kubeadm.go:309] [kubeconfig] Writing "admin.conf" kubeconfig file
I0520 15:18:11.169947 2701137 kubeadm.go:309] [kubeconfig] Writing "super-admin.conf" kubeconfig file
I0520 15:18:11.465315 2701137 kubeadm.go:309] [kubeconfig] Writing "kubelet.conf" kubeconfig file
I0520 15:18:11.731212 2701137 kubeadm.go:309] [kubeconfig] Writing "controller-manager.conf" kubeconfig file
I0520 15:18:12.189552 2701137 kubeadm.go:309] [kubeconfig] Writing "scheduler.conf" kubeconfig file
I0520 15:18:12.190415 2701137 kubeadm.go:309] [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
I0520 15:18:12.195452 2701137 kubeadm.go:309] [control-plane] Using manifest folder "/etc/kubernetes/manifests"
I0520 15:18:12.198883 2701137 out.go:204] - Booting up control plane ...
I0520 15:18:12.198997 2701137 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-apiserver"
I0520 15:18:12.199078 2701137 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-controller-manager"
I0520 15:18:12.199150 2701137 kubeadm.go:309] [control-plane] Creating static Pod manifest for "kube-scheduler"
I0520 15:18:12.210510 2701137 kubeadm.go:309] [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
I0520 15:18:12.211432 2701137 kubeadm.go:309] [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
I0520 15:18:12.211666 2701137 kubeadm.go:309] [kubelet-start] Starting the kubelet
I0520 15:18:12.353508 2701137 kubeadm.go:309] [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
I0520 15:18:12.353602 2701137 kubeadm.go:309] [kubelet-check] Waiting for a healthy kubelet. This can take up to 4m0s
I0520 15:18:14.353733 2701137 kubeadm.go:309] [kubelet-check] The kubelet is healthy after 2.000937405s
I0520 15:18:14.353840 2701137 kubeadm.go:309] [api-check] Waiting for a healthy API server. This can take up to 4m0s
I0520 15:18:21.599316 2687927 ssh_runner.go:195] Run: sudo pgrep -xnf kube-apiserver.*minikube.*
I0520 15:18:21.614072 2687927 api_server.go:72] duration metric: took 5m51.702320208s to wait for apiserver process to appear ...
I0520 15:18:21.614097 2687927 api_server.go:88] waiting for apiserver healthz status ...
I0520 15:18:21.614176 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-apiserver --format={{.ID}}
I0520 15:18:21.640018 2687927 logs.go:276] 2 containers: [7b906f9c7e42 21cda61304ec]
I0520 15:18:21.640104 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_etcd --format={{.ID}}
I0520 15:18:21.656535 2687927 logs.go:276] 2 containers: [4183e6237564 78716dd9f5e6]
I0520 15:18:21.656612 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_coredns --format={{.ID}}
I0520 15:18:21.857303 2701137 kubeadm.go:309] [api-check] The API server is healthy after 7.501427563s
I0520 15:18:21.913568 2701137 kubeadm.go:309] [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0520 15:18:21.933148 2701137 kubeadm.go:309] [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
I0520 15:18:22.006020 2701137 kubeadm.go:309] [upload-certs] Skipping phase. Please see --upload-certs
I0520 15:18:22.006215 2701137 kubeadm.go:309] [mark-control-plane] Marking the node embed-certs-684040 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
I0520 15:18:22.038348 2701137 kubeadm.go:309] [bootstrap-token] Using token: 1og1xx.vf5upa72r09vtfj1
I0520 15:18:22.041079 2701137 out.go:204] - Configuring RBAC rules ...
I0520 15:18:22.041208 2701137 kubeadm.go:309] [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
I0520 15:18:22.060382 2701137 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
I0520 15:18:22.072215 2701137 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0520 15:18:22.078301 2701137 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
I0520 15:18:22.084620 2701137 kubeadm.go:309] [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
I0520 15:18:22.093323 2701137 kubeadm.go:309] [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
I0520 15:18:22.263592 2701137 kubeadm.go:309] [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
I0520 15:18:22.718783 2701137 kubeadm.go:309] [addons] Applied essential addon: CoreDNS
I0520 15:18:23.262619 2701137 kubeadm.go:309] [addons] Applied essential addon: kube-proxy
I0520 15:18:23.263705 2701137 kubeadm.go:309]
I0520 15:18:23.263785 2701137 kubeadm.go:309] Your Kubernetes control-plane has initialized successfully!
I0520 15:18:23.263791 2701137 kubeadm.go:309]
I0520 15:18:23.263865 2701137 kubeadm.go:309] To start using your cluster, you need to run the following as a regular user:
I0520 15:18:23.263870 2701137 kubeadm.go:309]
I0520 15:18:23.263895 2701137 kubeadm.go:309] mkdir -p $HOME/.kube
I0520 15:18:23.263951 2701137 kubeadm.go:309] sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
I0520 15:18:23.264000 2701137 kubeadm.go:309] sudo chown $(id -u):$(id -g) $HOME/.kube/config
I0520 15:18:23.264005 2701137 kubeadm.go:309]
I0520 15:18:23.264057 2701137 kubeadm.go:309] Alternatively, if you are the root user, you can run:
I0520 15:18:23.264061 2701137 kubeadm.go:309]
I0520 15:18:23.264106 2701137 kubeadm.go:309] export KUBECONFIG=/etc/kubernetes/admin.conf
I0520 15:18:23.264111 2701137 kubeadm.go:309]
I0520 15:18:23.264161 2701137 kubeadm.go:309] You should now deploy a pod network to the cluster.
I0520 15:18:23.264232 2701137 kubeadm.go:309] Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
I0520 15:18:23.264298 2701137 kubeadm.go:309] https://kubernetes.io/docs/concepts/cluster-administration/addons/
I0520 15:18:23.264302 2701137 kubeadm.go:309]
I0520 15:18:23.264383 2701137 kubeadm.go:309] You can now join any number of control-plane nodes by copying certificate authorities
I0520 15:18:23.264456 2701137 kubeadm.go:309] and service account keys on each node and then running the following as root:
I0520 15:18:23.264460 2701137 kubeadm.go:309]
I0520 15:18:23.264541 2701137 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token 1og1xx.vf5upa72r09vtfj1 \
I0520 15:18:23.264646 2701137 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:02a40e6af316713a71980e77822669323fa1d64d9bd0513bcb005e042aa1234c \
I0520 15:18:23.264666 2701137 kubeadm.go:309] --control-plane
I0520 15:18:23.264670 2701137 kubeadm.go:309]
I0520 15:18:23.264752 2701137 kubeadm.go:309] Then you can join any number of worker nodes by running the following on each as root:
I0520 15:18:23.264757 2701137 kubeadm.go:309]
I0520 15:18:23.265023 2701137 kubeadm.go:309] kubeadm join control-plane.minikube.internal:8443 --token 1og1xx.vf5upa72r09vtfj1 \
I0520 15:18:23.265142 2701137 kubeadm.go:309] --discovery-token-ca-cert-hash sha256:02a40e6af316713a71980e77822669323fa1d64d9bd0513bcb005e042aa1234c
I0520 15:18:23.269425 2701137 kubeadm.go:309] [WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/5.15.0-1058-aws\n", err: exit status 1
I0520 15:18:23.269542 2701137 kubeadm.go:309] [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
I0520 15:18:23.269566 2701137 cni.go:84] Creating CNI manager for ""
I0520 15:18:23.269583 2701137 cni.go:158] "docker" driver + "docker" container runtime found on kubernetes v1.24+, recommending bridge
I0520 15:18:23.272769 2701137 out.go:177] * Configuring bridge CNI (Container Networking Interface) ...
I0520 15:18:23.276071 2701137 ssh_runner.go:195] Run: sudo mkdir -p /etc/cni/net.d
I0520 15:18:23.285189 2701137 ssh_runner.go:362] scp memory --> /etc/cni/net.d/1-k8s.conflist (496 bytes)
I0520 15:18:23.305377 2701137 ssh_runner.go:195] Run: /bin/bash -c "cat /proc/$(pgrep kube-apiserver)/oom_adj"
I0520 15:18:23.305528 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl create clusterrolebinding minikube-rbac --clusterrole=cluster-admin --serviceaccount=kube-system:default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:23.305636 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl --kubeconfig=/var/lib/minikube/kubeconfig label --overwrite nodes embed-certs-684040 minikube.k8s.io/updated_at=2024_05_20T15_18_23_0700 minikube.k8s.io/version=v1.33.1 minikube.k8s.io/commit=76465265bbb001d7b835613165bf9ac6b2521d45 minikube.k8s.io/name=embed-certs-684040 minikube.k8s.io/primary=true
I0520 15:18:23.423473 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:23.423596 2701137 ops.go:34] apiserver oom_adj: -16
I0520 15:18:23.924384 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:21.675372 2687927 logs.go:276] 2 containers: [57e027c8e0da 5876807206c9]
I0520 15:18:21.675454 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-scheduler --format={{.ID}}
I0520 15:18:21.694106 2687927 logs.go:276] 2 containers: [20ae03428247 798f34afc547]
I0520 15:18:21.694191 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-proxy --format={{.ID}}
I0520 15:18:21.713309 2687927 logs.go:276] 2 containers: [75eb680f6808 36b9b4fb305d]
I0520 15:18:21.713391 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kube-controller-manager --format={{.ID}}
I0520 15:18:21.736072 2687927 logs.go:276] 2 containers: [18ca6dabc240 7c325beb8903]
I0520 15:18:21.736157 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kindnet --format={{.ID}}
I0520 15:18:21.751873 2687927 logs.go:276] 0 containers: []
W0520 15:18:21.751894 2687927 logs.go:278] No container was found matching "kindnet"
I0520 15:18:21.751953 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_kubernetes-dashboard --format={{.ID}}
I0520 15:18:21.769052 2687927 logs.go:276] 1 containers: [e9d80f29a80f]
I0520 15:18:21.769133 2687927 ssh_runner.go:195] Run: docker ps -a --filter=name=k8s_storage-provisioner --format={{.ID}}
I0520 15:18:21.786563 2687927 logs.go:276] 2 containers: [ee9aab3e1ca5 623598d9bb43]
I0520 15:18:21.786597 2687927 logs.go:123] Gathering logs for kube-proxy [36b9b4fb305d] ...
I0520 15:18:21.786609 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 36b9b4fb305d"
I0520 15:18:21.809092 2687927 logs.go:123] Gathering logs for kube-controller-manager [7c325beb8903] ...
I0520 15:18:21.809118 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7c325beb8903"
I0520 15:18:21.860586 2687927 logs.go:123] Gathering logs for storage-provisioner [ee9aab3e1ca5] ...
I0520 15:18:21.860642 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 ee9aab3e1ca5"
I0520 15:18:21.896739 2687927 logs.go:123] Gathering logs for kubelet ...
I0520 15:18:21.896810 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u kubelet -n 400"
W0520 15:18:21.977326 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076509 1198 reflector.go:138] object-"kube-system"/"kube-proxy": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.977623 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076589 1198 reflector.go:138] object-"kube-system"/"kube-proxy-token-4znn8": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "kube-proxy-token-4znn8" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.977905 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076649 1198 reflector.go:138] object-"kube-system"/"coredns": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "coredns" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.978156 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.076901 1198 reflector.go:138] object-"kube-system"/"coredns-token-rk862": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "coredns-token-rk862" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.979172 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.252268 1198 reflector.go:138] object-"default"/"default-token-kj425": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "default-token-kj425" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "default": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.979475 2687927 logs.go:138] Found kubelet problem: May 20 15:12:45 old-k8s-version-185009 kubelet[1198]: E0520 15:12:45.263194 1198 reflector.go:138] object-"kube-system"/"storage-provisioner-token-rpfqn": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets "storage-provisioner-token-rpfqn" is forbidden: User "system:node:old-k8s-version-185009" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node 'old-k8s-version-185009' and this object
W0520 15:18:21.988228 2687927 logs.go:138] Found kubelet problem: May 20 15:12:46 old-k8s-version-185009 kubelet[1198]: E0520 15:12:46.633160 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:21.988671 2687927 logs.go:138] Found kubelet problem: May 20 15:12:47 old-k8s-version-185009 kubelet[1198]: E0520 15:12:47.647957 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.989730 2687927 logs.go:138] Found kubelet problem: May 20 15:12:48 old-k8s-version-185009 kubelet[1198]: E0520 15:12:48.926924 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.992237 2687927 logs.go:138] Found kubelet problem: May 20 15:13:00 old-k8s-version-185009 kubelet[1198]: E0520 15:13:00.334303 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:21.997546 2687927 logs.go:138] Found kubelet problem: May 20 15:13:05 old-k8s-version-185009 kubelet[1198]: E0520 15:13:05.302095 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:21.997800 2687927 logs.go:138] Found kubelet problem: May 20 15:13:06 old-k8s-version-185009 kubelet[1198]: E0520 15:13:06.152746 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.998019 2687927 logs.go:138] Found kubelet problem: May 20 15:13:11 old-k8s-version-185009 kubelet[1198]: E0520 15:13:11.129523 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:21.999028 2687927 logs.go:138] Found kubelet problem: May 20 15:13:18 old-k8s-version-185009 kubelet[1198]: E0520 15:13:18.383550 1198 pod_workers.go:191] Error syncing pod 55c63b6d-7e3d-4491-9a29-4bc83a4d54fa ("storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"), skipping: failed to "StartContainer" for "storage-provisioner" with CrashLoopBackOff: "back-off 10s restarting failed container=storage-provisioner pod=storage-provisioner_kube-system(55c63b6d-7e3d-4491-9a29-4bc83a4d54fa)"
W0520 15:18:22.001353 2687927 logs.go:138] Found kubelet problem: May 20 15:13:19 old-k8s-version-185009 kubelet[1198]: E0520 15:13:19.635684 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.003942 2687927 logs.go:138] Found kubelet problem: May 20 15:13:23 old-k8s-version-185009 kubelet[1198]: E0520 15:13:23.158433 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.006875 2687927 logs.go:138] Found kubelet problem: May 20 15:13:32 old-k8s-version-185009 kubelet[1198]: E0520 15:13:32.119072 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.007244 2687927 logs.go:138] Found kubelet problem: May 20 15:13:35 old-k8s-version-185009 kubelet[1198]: E0520 15:13:35.124065 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.009571 2687927 logs.go:138] Found kubelet problem: May 20 15:13:43 old-k8s-version-185009 kubelet[1198]: E0520 15:13:43.578964 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.009800 2687927 logs.go:138] Found kubelet problem: May 20 15:13:47 old-k8s-version-185009 kubelet[1198]: E0520 15:13:47.118959 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010034 2687927 logs.go:138] Found kubelet problem: May 20 15:13:58 old-k8s-version-185009 kubelet[1198]: E0520 15:13:58.119337 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010345 2687927 logs.go:138] Found kubelet problem: May 20 15:14:01 old-k8s-version-185009 kubelet[1198]: E0520 15:14:01.118790 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.010635 2687927 logs.go:138] Found kubelet problem: May 20 15:14:10 old-k8s-version-185009 kubelet[1198]: E0520 15:14:10.148705 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.012895 2687927 logs.go:138] Found kubelet problem: May 20 15:14:14 old-k8s-version-185009 kubelet[1198]: E0520 15:14:14.139015 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.015404 2687927 logs.go:138] Found kubelet problem: May 20 15:14:25 old-k8s-version-185009 kubelet[1198]: E0520 15:14:25.575698 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.015647 2687927 logs.go:138] Found kubelet problem: May 20 15:14:28 old-k8s-version-185009 kubelet[1198]: E0520 15:14:28.126943 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.015879 2687927 logs.go:138] Found kubelet problem: May 20 15:14:38 old-k8s-version-185009 kubelet[1198]: E0520 15:14:38.118687 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016099 2687927 logs.go:138] Found kubelet problem: May 20 15:14:42 old-k8s-version-185009 kubelet[1198]: E0520 15:14:42.119209 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016340 2687927 logs.go:138] Found kubelet problem: May 20 15:14:53 old-k8s-version-185009 kubelet[1198]: E0520 15:14:53.121794 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016567 2687927 logs.go:138] Found kubelet problem: May 20 15:14:56 old-k8s-version-185009 kubelet[1198]: E0520 15:14:56.118640 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.016803 2687927 logs.go:138] Found kubelet problem: May 20 15:15:08 old-k8s-version-185009 kubelet[1198]: E0520 15:15:08.118744 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017026 2687927 logs.go:138] Found kubelet problem: May 20 15:15:11 old-k8s-version-185009 kubelet[1198]: E0520 15:15:11.125539 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017249 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.119290 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.017486 2687927 logs.go:138] Found kubelet problem: May 20 15:15:23 old-k8s-version-185009 kubelet[1198]: E0520 15:15:23.129546 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.019229 2687927 logs.go:138] Found kubelet problem: May 20 15:15:34 old-k8s-version-185009 kubelet[1198]: E0520 15:15:34.118580 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.021424 2687927 logs.go:138] Found kubelet problem: May 20 15:15:35 old-k8s-version-185009 kubelet[1198]: E0520 15:15:35.142356 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
W0520 15:18:22.021691 2687927 logs.go:138] Found kubelet problem: May 20 15:15:47 old-k8s-version-185009 kubelet[1198]: E0520 15:15:47.118833 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024033 2687927 logs.go:138] Found kubelet problem: May 20 15:15:49 old-k8s-version-185009 kubelet[1198]: E0520 15:15:49.579168 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ErrImagePull: "rpc error: code = Unknown desc = [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of registry.k8s.io/echoserver:1.4 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/"
W0520 15:18:22.024251 2687927 logs.go:138] Found kubelet problem: May 20 15:15:58 old-k8s-version-185009 kubelet[1198]: E0520 15:15:58.119002 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024484 2687927 logs.go:138] Found kubelet problem: May 20 15:16:03 old-k8s-version-185009 kubelet[1198]: E0520 15:16:03.129847 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024703 2687927 logs.go:138] Found kubelet problem: May 20 15:16:09 old-k8s-version-185009 kubelet[1198]: E0520 15:16:09.118709 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.024938 2687927 logs.go:138] Found kubelet problem: May 20 15:16:15 old-k8s-version-185009 kubelet[1198]: E0520 15:16:15.140673 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025166 2687927 logs.go:138] Found kubelet problem: May 20 15:16:22 old-k8s-version-185009 kubelet[1198]: E0520 15:16:22.118599 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025535 2687927 logs.go:138] Found kubelet problem: May 20 15:16:26 old-k8s-version-185009 kubelet[1198]: E0520 15:16:26.125485 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.025809 2687927 logs.go:138] Found kubelet problem: May 20 15:16:35 old-k8s-version-185009 kubelet[1198]: E0520 15:16:35.118669 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026048 2687927 logs.go:138] Found kubelet problem: May 20 15:16:37 old-k8s-version-185009 kubelet[1198]: E0520 15:16:37.119240 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026311 2687927 logs.go:138] Found kubelet problem: May 20 15:16:49 old-k8s-version-185009 kubelet[1198]: E0520 15:16:49.118660 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026556 2687927 logs.go:138] Found kubelet problem: May 20 15:16:51 old-k8s-version-185009 kubelet[1198]: E0520 15:16:51.118889 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.026776 2687927 logs.go:138] Found kubelet problem: May 20 15:17:01 old-k8s-version-185009 kubelet[1198]: E0520 15:17:01.131117 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027012 2687927 logs.go:138] Found kubelet problem: May 20 15:17:02 old-k8s-version-185009 kubelet[1198]: E0520 15:17:02.119112 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027240 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118969 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027488 2687927 logs.go:138] Found kubelet problem: May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118971 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027714 2687927 logs.go:138] Found kubelet problem: May 20 15:17:25 old-k8s-version-185009 kubelet[1198]: E0520 15:17:25.119102 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.027944 2687927 logs.go:138] Found kubelet problem: May 20 15:17:27 old-k8s-version-185009 kubelet[1198]: E0520 15:17:27.141000 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028163 2687927 logs.go:138] Found kubelet problem: May 20 15:17:37 old-k8s-version-185009 kubelet[1198]: E0520 15:17:37.118917 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028395 2687927 logs.go:138] Found kubelet problem: May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028633 2687927 logs.go:138] Found kubelet problem: May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.028865 2687927 logs.go:138] Found kubelet problem: May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029088 2687927 logs.go:138] Found kubelet problem: May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029320 2687927 logs.go:138] Found kubelet problem: May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029540 2687927 logs.go:138] Found kubelet problem: May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.029784 2687927 logs.go:138] Found kubelet problem: May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:22.029813 2687927 logs.go:123] Gathering logs for dmesg ...
I0520 15:18:22.029846 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo dmesg -PH -L=never --level warn,err,crit,alert,emerg | tail -n 400"
I0520 15:18:22.053942 2687927 logs.go:123] Gathering logs for describe nodes ...
I0520 15:18:22.054014 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.20.0/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig"
I0520 15:18:22.227606 2687927 logs.go:123] Gathering logs for coredns [57e027c8e0da] ...
I0520 15:18:22.227642 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 57e027c8e0da"
I0520 15:18:22.251970 2687927 logs.go:123] Gathering logs for coredns [5876807206c9] ...
I0520 15:18:22.251999 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 5876807206c9"
I0520 15:18:22.282757 2687927 logs.go:123] Gathering logs for storage-provisioner [623598d9bb43] ...
I0520 15:18:22.282789 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 623598d9bb43"
I0520 15:18:22.318436 2687927 logs.go:123] Gathering logs for Docker ...
I0520 15:18:22.318468 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo journalctl -u docker -u cri-docker -n 400"
I0520 15:18:22.362088 2687927 logs.go:123] Gathering logs for container status ...
I0520 15:18:22.362124 2687927 ssh_runner.go:195] Run: /bin/bash -c "sudo `which crictl || echo crictl` ps -a || sudo docker ps -a"
I0520 15:18:22.428320 2687927 logs.go:123] Gathering logs for kube-apiserver [21cda61304ec] ...
I0520 15:18:22.428347 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 21cda61304ec"
I0520 15:18:22.512882 2687927 logs.go:123] Gathering logs for kubernetes-dashboard [e9d80f29a80f] ...
I0520 15:18:22.512921 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 e9d80f29a80f"
I0520 15:18:22.547849 2687927 logs.go:123] Gathering logs for kube-apiserver [7b906f9c7e42] ...
I0520 15:18:22.547961 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 7b906f9c7e42"
I0520 15:18:22.611472 2687927 logs.go:123] Gathering logs for etcd [4183e6237564] ...
I0520 15:18:22.611514 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 4183e6237564"
I0520 15:18:22.640116 2687927 logs.go:123] Gathering logs for kube-scheduler [20ae03428247] ...
I0520 15:18:22.640149 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 20ae03428247"
I0520 15:18:22.669795 2687927 logs.go:123] Gathering logs for kube-proxy [75eb680f6808] ...
I0520 15:18:22.669823 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 75eb680f6808"
I0520 15:18:22.700074 2687927 logs.go:123] Gathering logs for etcd [78716dd9f5e6] ...
I0520 15:18:22.700103 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 78716dd9f5e6"
I0520 15:18:22.744444 2687927 logs.go:123] Gathering logs for kube-scheduler [798f34afc547] ...
I0520 15:18:22.744475 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 798f34afc547"
I0520 15:18:22.785309 2687927 logs.go:123] Gathering logs for kube-controller-manager [18ca6dabc240] ...
I0520 15:18:22.785340 2687927 ssh_runner.go:195] Run: /bin/bash -c "docker logs --tail 400 18ca6dabc240"
I0520 15:18:22.849036 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:22.849070 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
W0520 15:18:22.849150 2687927 out.go:239] X Problems detected in kubelet:
W0520 15:18:22.849163 2687927 out.go:239] May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849202 2687927 out.go:239] May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849215 2687927 out.go:239] May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849222 2687927 out.go:239] May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
W0520 15:18:22.849241 2687927 out.go:239] May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
I0520 15:18:22.849252 2687927 out.go:304] Setting ErrFile to fd 2...
I0520 15:18:22.849258 2687927 out.go:338] TERM=,COLORTERM=, which probably does not support color
I0520 15:18:24.423864 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:24.924331 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:25.423647 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:25.923948 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:26.424421 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:26.924082 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:27.424200 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:27.924199 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:28.423642 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:28.923646 2701137 ssh_runner.go:195] Run: sudo /var/lib/minikube/binaries/v1.30.1/kubectl get sa default --kubeconfig=/var/lib/minikube/kubeconfig
I0520 15:18:32.850044 2687927 api_server.go:253] Checking apiserver healthz at https://192.168.76.2:8443/healthz ...
I0520 15:18:32.868528 2687927 api_server.go:279] https://192.168.76.2:8443/healthz returned 200:
ok
I0520 15:18:32.871090 2687927 out.go:177]
W0520 15:18:32.873480 2687927 out.go:239] X Exiting due to K8S_UNHEALTHY_CONTROL_PLANE: wait 6m0s for node: wait for healthy API server: controlPlane never updated to v1.20.0
W0520 15:18:32.873524 2687927 out.go:239] * Suggestion: Control Plane could not update, try minikube delete --all --purge
W0520 15:18:32.873552 2687927 out.go:239] * Related issue: https://github.com/kubernetes/minikube/issues/11417
W0520 15:18:32.873563 2687927 out.go:239] *
W0520 15:18:32.874427 2687927 out.go:239] ╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ * If the above advice does not help, please let us know: │
│ https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ * Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯
I0520 15:18:32.877815 2687927 out.go:177]
==> Docker <==
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:11 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:11 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:21 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:21 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:21 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:21 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:21 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:21 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:22 old-k8s-version-185009 dockerd[971]: 2024/05/20 15:18:22 http: superfluous response.WriteHeader call from go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*respWriterWrapper).WriteHeader (wrap.go:98)
May 20 15:18:25 old-k8s-version-185009 dockerd[971]: time="2024-05-20T15:18:25.144614879Z" level=warning msg="Error getting v2 registry: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=edac6e53d50bf88e traceID=3bd25debab99289b187e20d49040c04a
May 20 15:18:25 old-k8s-version-185009 dockerd[971]: time="2024-05-20T15:18:25.144681668Z" level=info msg="Attempting next endpoint for pull after error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=edac6e53d50bf88e traceID=3bd25debab99289b187e20d49040c04a
May 20 15:18:25 old-k8s-version-185009 dockerd[971]: time="2024-05-20T15:18:25.147602930Z" level=error msg="Handler for POST /v1.40/images/create returned error: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host" spanID=edac6e53d50bf88e traceID=3bd25debab99289b187e20d49040c04a
==> container status <==
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
ee9aab3e1ca5d ba04bb24b9575 5 minutes ago Running storage-provisioner 3 bb33854bd5575 storage-provisioner
e9d80f29a80fe kubernetesui/dashboard@sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93 5 minutes ago Running kubernetes-dashboard 0 2894a00a88fe7 kubernetes-dashboard-cd95d586-sm45l
8954a825d2441 1611cd07b61d5 5 minutes ago Running busybox 1 681ffaa9c688d busybox
57e027c8e0da5 db91994f4ee8f 5 minutes ago Running coredns 1 a4c0d78f72e14 coredns-74ff55c5b-frwjt
623598d9bb43e ba04bb24b9575 5 minutes ago Exited storage-provisioner 2 bb33854bd5575 storage-provisioner
75eb680f68084 25a5233254979 5 minutes ago Running kube-proxy 1 90ef0c46a1c0a kube-proxy-p9rc9
20ae034282470 e7605f88f17d6 5 minutes ago Running kube-scheduler 1 838e5a86ddade kube-scheduler-old-k8s-version-185009
4183e62375648 05b738aa1bc63 6 minutes ago Running etcd 1 047b43737abbf etcd-old-k8s-version-185009
7b906f9c7e424 2c08bbbc02d3a 6 minutes ago Running kube-apiserver 1 85480e749845a kube-apiserver-old-k8s-version-185009
18ca6dabc240c 1df8a2b116bd1 6 minutes ago Running kube-controller-manager 1 42dfe894d746a kube-controller-manager-old-k8s-version-185009
f5399203d319f gcr.io/k8s-minikube/busybox@sha256:2d03e6ceeb99250061dd110530b0ece7998cd84121f952adef120ea7c5a6f00e 6 minutes ago Exited busybox 0 238e68b7ca85f busybox
36b9b4fb305df 25a5233254979 8 minutes ago Exited kube-proxy 0 41866660a14bd kube-proxy-p9rc9
5876807206c98 db91994f4ee8f 8 minutes ago Exited coredns 0 94183831a4502 coredns-74ff55c5b-frwjt
798f34afc5472 e7605f88f17d6 8 minutes ago Exited kube-scheduler 0 c96c48f5407b1 kube-scheduler-old-k8s-version-185009
7c325beb89037 1df8a2b116bd1 8 minutes ago Exited kube-controller-manager 0 0d6532bd0c2fd kube-controller-manager-old-k8s-version-185009
21cda61304ec4 2c08bbbc02d3a 8 minutes ago Exited kube-apiserver 0 0b9c6b9f1e428 kube-apiserver-old-k8s-version-185009
78716dd9f5e6b 05b738aa1bc63 8 minutes ago Exited etcd 0 28ef02173f451 etcd-old-k8s-version-185009
==> coredns [57e027c8e0da] <==
.:53
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] 127.0.0.1:40913 - 41832 "HINFO IN 332871789167854277.7996132527519855949. udp 56 false 512" NXDOMAIN qr,rd,ra 56 0.043024295s
==> coredns [5876807206c9] <==
I0520 15:10:44.807032 1 trace.go:116] Trace[2019727887]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-20 15:10:14.806393265 +0000 UTC m=+0.054097011) (total time: 30.000538089s):
Trace[2019727887]: [30.000538089s] [30.000538089s] END
E0520 15:10:44.807082 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0520 15:10:44.808962 1 trace.go:116] Trace[939984059]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-20 15:10:14.8075352 +0000 UTC m=+0.055238938) (total time: 30.001404599s):
Trace[939984059]: [30.001404599s] [30.001404599s] END
E0520 15:10:44.808979 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Service: Get "https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
I0520 15:10:44.809724 1 trace.go:116] Trace[1474941318]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125 (started: 2024-05-20 15:10:14.809090214 +0000 UTC m=+0.056793960) (total time: 30.000615395s):
Trace[1474941318]: [30.000615395s] [30.000615395s] END
E0520 15:10:44.809742 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": dial tcp 10.96.0.1:443: i/o timeout
E0520 15:12:10.439294 1 reflector.go:382] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to watch *v1.Namespace: Get "https://10.96.0.1:443/api/v1/namespaces?allowWatchBookmarks=true&resourceVersion=200&timeout=6m5s&timeoutSeconds=365&watch=true": dial tcp 10.96.0.1:443: connect: connection refused
E0520 15:12:10.439336 1 reflector.go:382] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to watch *v1.Service: Get "https://10.96.0.1:443/api/v1/services?allowWatchBookmarks=true&resourceVersion=599&timeout=8m23s&timeoutSeconds=503&watch=true": dial tcp 10.96.0.1:443: connect: connection refused
E0520 15:12:10.439389 1 reflector.go:382] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to watch *v1.Endpoints: Get "https://10.96.0.1:443/api/v1/endpoints?allowWatchBookmarks=true&resourceVersion=604&timeout=6m27s&timeoutSeconds=387&watch=true": dial tcp 10.96.0.1:443: connect: connection refused
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.7.0
linux/arm64, go1.14.4, f59c03d
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] Reloading
[INFO] plugin/health: Going into lameduck mode for 5s
[INFO] plugin/reload: Running configuration MD5 = b494d968e357ba1b925cee838fbd78ed
[INFO] Reloading complete
[INFO] 127.0.0.1:50869 - 15078 "HINFO IN 2381389028460171299.4678444895080844093. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.01153117s
[INFO] SIGTERM: Shutting down servers then terminating
[INFO] plugin/health: Going into lameduck mode for 5s
==> describe nodes <==
Name: old-k8s-version-185009
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=old-k8s-version-185009
kubernetes.io/os=linux
minikube.k8s.io/commit=76465265bbb001d7b835613165bf9ac6b2521d45
minikube.k8s.io/name=old-k8s-version-185009
minikube.k8s.io/primary=true
minikube.k8s.io/updated_at=2024_05_20T15_09_58_0700
minikube.k8s.io/version=v1.33.1
node-role.kubernetes.io/control-plane=
node-role.kubernetes.io/master=
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 20 May 2024 15:09:55 +0000
Taints: <none>
Unschedulable: false
Lease:
HolderIdentity: old-k8s-version-185009
AcquireTime: <unset>
RenewTime: Mon, 20 May 2024 15:18:27 +0000
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
MemoryPressure False Mon, 20 May 2024 15:13:35 +0000 Mon, 20 May 2024 15:09:48 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Mon, 20 May 2024 15:13:35 +0000 Mon, 20 May 2024 15:09:48 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Mon, 20 May 2024 15:13:35 +0000 Mon, 20 May 2024 15:09:48 +0000 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Mon, 20 May 2024 15:13:35 +0000 Mon, 20 May 2024 15:10:12 +0000 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.76.2
Hostname: old-k8s-version-185009
Capacity:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022496Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 203034800Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
hugepages-32Mi: 0
hugepages-64Ki: 0
memory: 8022496Ki
pods: 110
System Info:
Machine ID: f546c781dab943478f45b3d0d7ba2f3b
System UUID: 881a12a7-0b67-4f04-a497-540e7b3ebfb8
Boot ID: df9684e8-d429-41b3-8a9f-ef96b9c9133b
Kernel Version: 5.15.0-1058-aws
OS Image: Ubuntu 22.04.4 LTS
Operating System: linux
Architecture: arm64
Container Runtime Version: docker://26.1.2
Kubelet Version: v1.20.0
Kube-Proxy Version: v1.20.0
PodCIDR: 10.244.0.0/24
PodCIDRs: 10.244.0.0/24
Non-terminated Pods: (11 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
default busybox 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 6m38s
kube-system coredns-74ff55c5b-frwjt 100m (5%!)(MISSING) 0 (0%!)(MISSING) 70Mi (0%!)(MISSING) 170Mi (2%!)(MISSING) 8m21s
kube-system etcd-old-k8s-version-185009 100m (5%!)(MISSING) 0 (0%!)(MISSING) 100Mi (1%!)(MISSING) 0 (0%!)(MISSING) 8m32s
kube-system kube-apiserver-old-k8s-version-185009 250m (12%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m32s
kube-system kube-controller-manager-old-k8s-version-185009 200m (10%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m32s
kube-system kube-proxy-p9rc9 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m21s
kube-system kube-scheduler-old-k8s-version-185009 100m (5%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m32s
kube-system metrics-server-9975d5f86-sks2q 100m (5%!)(MISSING) 0 (0%!)(MISSING) 200Mi (2%!)(MISSING) 0 (0%!)(MISSING) 6m24s
kube-system storage-provisioner 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 8m19s
kubernetes-dashboard dashboard-metrics-scraper-8d5bb5db8-5757j 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m31s
kubernetes-dashboard kubernetes-dashboard-cd95d586-sm45l 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 0 (0%!)(MISSING) 5m31s
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 850m (42%!)(MISSING) 0 (0%!)(MISSING)
memory 370Mi (4%!)(MISSING) 170Mi (2%!)(MISSING)
ephemeral-storage 100Mi (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-1Gi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-2Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-32Mi 0 (0%!)(MISSING) 0 (0%!)(MISSING)
hugepages-64Ki 0 (0%!)(MISSING) 0 (0%!)(MISSING)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal NodeHasSufficientMemory 8m47s (x5 over 8m47s) kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m47s (x5 over 8m47s) kubelet Node old-k8s-version-185009 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m47s (x5 over 8m47s) kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 8m47s kubelet Updated Node Allocatable limit across pods
Normal Starting 8m33s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 8m32s kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 8m32s kubelet Node old-k8s-version-185009 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 8m32s kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientPID
Normal NodeNotReady 8m32s kubelet Node old-k8s-version-185009 status is now: NodeNotReady
Normal NodeAllocatableEnforced 8m32s kubelet Updated Node Allocatable limit across pods
Normal NodeReady 8m22s kubelet Node old-k8s-version-185009 status is now: NodeReady
Normal Starting 8m19s kube-proxy Starting kube-proxy.
Normal Starting 6m2s kubelet Starting kubelet.
Normal NodeHasSufficientMemory 6m1s (x8 over 6m1s) kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientMemory
Normal NodeHasNoDiskPressure 6m1s (x8 over 6m1s) kubelet Node old-k8s-version-185009 status is now: NodeHasNoDiskPressure
Normal NodeHasSufficientPID 6m1s (x7 over 6m1s) kubelet Node old-k8s-version-185009 status is now: NodeHasSufficientPID
Normal NodeAllocatableEnforced 6m1s kubelet Updated Node Allocatable limit across pods
Normal Starting 5m47s kube-proxy Starting kube-proxy.
==> dmesg <==
[ +0.001252] FS-Cache: O-key=[8] '7b8a3b0000000000'
[ +0.000813] FS-Cache: N-cookie c=000001bc [p=000001b3 fl=2 nc=0 na=1]
[ +0.000995] FS-Cache: N-cookie d=00000000865130fc{9p.inode} n=00000000c223b28b
[ +0.001110] FS-Cache: N-key=[8] '7b8a3b0000000000'
[ +0.003192] FS-Cache: Duplicate cookie detected
[ +0.000812] FS-Cache: O-cookie c=000001b6 [p=000001b3 fl=226 nc=0 na=1]
[ +0.001167] FS-Cache: O-cookie d=00000000865130fc{9p.inode} n=00000000a8f7c898
[ +0.001110] FS-Cache: O-key=[8] '7b8a3b0000000000'
[ +0.000787] FS-Cache: N-cookie c=000001bd [p=000001b3 fl=2 nc=0 na=1]
[ +0.000989] FS-Cache: N-cookie d=00000000865130fc{9p.inode} n=00000000e950dce3
[ +0.001172] FS-Cache: N-key=[8] '7b8a3b0000000000'
[ +3.075935] FS-Cache: Duplicate cookie detected
[ +0.000794] FS-Cache: O-cookie c=000001b5 [p=000001b3 fl=226 nc=0 na=1]
[ +0.000971] FS-Cache: O-cookie d=00000000865130fc{9p.inode} n=0000000046618fe3
[ +0.001437] FS-Cache: O-key=[8] '7a8a3b0000000000'
[ +0.000848] FS-Cache: N-cookie c=000001bf [p=000001b3 fl=2 nc=0 na=1]
[ +0.001217] FS-Cache: N-cookie d=00000000865130fc{9p.inode} n=000000002d6a31f9
[ +0.001179] FS-Cache: N-key=[8] '7a8a3b0000000000'
[ +0.322477] FS-Cache: Duplicate cookie detected
[ +0.000760] FS-Cache: O-cookie c=000001b9 [p=000001b3 fl=226 nc=0 na=1]
[ +0.001102] FS-Cache: O-cookie d=00000000865130fc{9p.inode} n=0000000017488ba3
[ +0.001248] FS-Cache: O-key=[8] '808a3b0000000000'
[ +0.000786] FS-Cache: N-cookie c=000001c0 [p=000001b3 fl=2 nc=0 na=1]
[ +0.000973] FS-Cache: N-cookie d=00000000865130fc{9p.inode} n=000000006a946acd
[ +0.001094] FS-Cache: N-key=[8] '808a3b0000000000'
==> etcd [4183e6237564] <==
2024-05-20 15:14:28.571460 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:14:38.571490 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:14:48.571450 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:14:58.571723 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:08.571453 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:18.571678 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:28.571471 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:38.571481 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:48.571495 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:15:58.571493 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:08.571665 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:18.571429 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:28.571430 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:38.571550 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:48.571447 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:16:58.571606 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:08.571515 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:18.571554 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:28.571608 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:38.571587 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:48.571639 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:17:58.571441 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:18:08.571604 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:18:18.571608 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:18:28.571525 I | etcdserver/api/etcdhttp: /health OK (status code 200)
==> etcd [78716dd9f5e6] <==
2024-05-20 15:09:49.230213 I | etcdserver: published {Name:old-k8s-version-185009 ClientURLs:[https://192.168.76.2:2379]} to cluster 6f20f2c4b2fb5f8a
2024-05-20 15:09:49.231132 I | embed: ready to serve client requests
2024-05-20 15:09:49.231268 I | etcdserver: setting up the initial cluster version to 3.4
2024-05-20 15:09:49.232649 I | embed: serving client requests on 192.168.76.2:2379
2024-05-20 15:09:49.232727 I | embed: ready to serve client requests
2024-05-20 15:09:49.237247 I | embed: serving client requests on 127.0.0.1:2379
2024-05-20 15:09:49.243803 N | etcdserver/membership: set the initial cluster version to 3.4
2024-05-20 15:09:49.255820 I | etcdserver/api: enabled capabilities for version 3.4
2024-05-20 15:10:09.248698 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:09.937718 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:19.934581 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:29.934508 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:39.934507 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:49.934473 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:10:59.934750 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:09.934909 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:19.934711 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:29.934734 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:39.934750 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:49.934553 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:11:59.935100 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:12:09.938345 I | etcdserver/api/etcdhttp: /health OK (status code 200)
2024-05-20 15:12:10.767915 N | pkg/osutil: received terminated signal, shutting down...
WARNING: 2024/05/20 15:12:10 grpc: addrConn.createTransport failed to connect to {127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
2024-05-20 15:12:10.821492 I | etcdserver: skipped leadership transfer for single voting member cluster
==> kernel <==
15:18:34 up 1 day, 23:01, 0 users, load average: 1.25, 2.09, 2.88
Linux old-k8s-version-185009 5.15.0-1058-aws #64~20.04.1-Ubuntu SMP Tue Apr 9 11:11:55 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.4 LTS"
==> kube-apiserver [21cda61304ec] <==
W0520 15:12:10.794221 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
I0520 15:12:10.794475 1 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
I0520 15:12:10.794558 1 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
I0520 15:12:10.794729 1 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
W0520 15:12:10.794783 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794819 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794852 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794886 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794922 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794956 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.794992 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.795027 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.795064 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.795100 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.795137 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
I0520 15:12:10.808473 1 clientconn.go:897] blockingPicker: the picked transport is not ready, loop back to repick
W0520 15:12:10.809312 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.810748 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.811001 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.811221 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.811419 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.811627 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.811830 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.812026 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
W0520 15:12:10.812224 1 clientconn.go:1223] grpc: addrConn.createTransport failed to connect to {https://127.0.0.1:2379 <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused". Reconnecting...
==> kube-apiserver [7b906f9c7e42] <==
I0520 15:15:08.276619 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:15:08.276628 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0520 15:15:38.795946 1 client.go:360] parsed scheme: "passthrough"
I0520 15:15:38.796007 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:15:38.796016 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0520 15:15:47.889083 1 handler_proxy.go:102] no RequestInfo found in the context
E0520 15:15:47.889164 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0520 15:15:47.889177 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0520 15:16:14.555679 1 client.go:360] parsed scheme: "passthrough"
I0520 15:16:14.555721 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:16:14.555729 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0520 15:16:52.617532 1 client.go:360] parsed scheme: "passthrough"
I0520 15:16:52.617581 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:16:52.617590 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
I0520 15:17:31.918515 1 client.go:360] parsed scheme: "passthrough"
I0520 15:17:31.918567 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:17:31.918576 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
W0520 15:17:46.060504 1 handler_proxy.go:102] no RequestInfo found in the context
E0520 15:17:46.060685 1 controller.go:116] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0520 15:17:46.060714 1 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0520 15:18:10.130155 1 client.go:360] parsed scheme: "passthrough"
I0520 15:18:10.130222 1 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{https://127.0.0.1:2379 <nil> 0 <nil>}] <nil> <nil>}
I0520 15:18:10.130233 1 clientconn.go:948] ClientConn switching balancer to "pick_first"
==> kube-controller-manager [18ca6dabc240] <==
W0520 15:14:09.392185 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:14:35.477289 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:14:41.042626 1 request.go:655] Throttling request took 1.048401022s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0520 15:14:41.894123 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:15:05.979014 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:15:13.544505 1 request.go:655] Throttling request took 1.048473782s, request: GET:https://192.168.76.2:8443/apis/apiextensions.k8s.io/v1beta1?timeout=32s
W0520 15:15:14.396432 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:15:36.481000 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:15:46.046932 1 request.go:655] Throttling request took 1.048447069s, request: GET:https://192.168.76.2:8443/apis/networking.k8s.io/v1beta1?timeout=32s
W0520 15:15:46.902091 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:16:06.983095 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:16:18.552765 1 request.go:655] Throttling request took 1.048678009s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0520 15:16:19.403971 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:16:37.485568 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:16:51.054521 1 request.go:655] Throttling request took 1.044709151s, request: GET:https://192.168.76.2:8443/apis/admissionregistration.k8s.io/v1beta1?timeout=32s
W0520 15:16:51.905954 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:17:07.990740 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:17:23.556337 1 request.go:655] Throttling request took 1.048312976s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0520 15:17:24.407778 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:17:38.493474 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:17:56.058346 1 request.go:655] Throttling request took 1.046500124s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0520 15:17:56.909812 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
E0520 15:18:08.995430 1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
I0520 15:18:28.560294 1 request.go:655] Throttling request took 1.048244151s, request: GET:https://192.168.76.2:8443/apis/extensions/v1beta1?timeout=32s
W0520 15:18:29.411777 1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
==> kube-controller-manager [7c325beb8903] <==
I0520 15:10:13.582077 1 disruption.go:339] Sending events to api server.
I0520 15:10:13.584306 1 shared_informer.go:247] Caches are synced for resource quota
I0520 15:10:13.605572 1 shared_informer.go:247] Caches are synced for resource quota
E0520 15:10:13.648836 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
E0520 15:10:13.678570 1 clusterroleaggregation_controller.go:181] admin failed with : Operation cannot be fulfilled on clusterroles.rbac.authorization.k8s.io "admin": the object has been modified; please apply your changes to the latest version and try again
I0520 15:10:13.804025 1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I0520 15:10:14.004288 1 shared_informer.go:247] Caches are synced for garbage collector
I0520 15:10:14.040564 1 shared_informer.go:247] Caches are synced for garbage collector
I0520 15:10:14.040584 1 garbagecollector.go:151] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
I0520 15:10:14.985433 1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set coredns-74ff55c5b to 1"
I0520 15:10:15.000380 1 event.go:291] "Event occurred" object="kube-system/coredns-74ff55c5b" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: coredns-74ff55c5b-rmbdz"
I0520 15:12:09.385233 1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-9975d5f86 to 1"
I0520 15:12:10.441826 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": http2: server sent GOAWAY and closed the connection; LastStreamID=1011, ErrCode=NO_ERROR, debug=\"\""
E0520 15:12:10.441924 1 event.go:273] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"metrics-server-9975d5f86.17d13b1d65177f3e", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"metrics-server-9975d5f86", UID:"fd91584b-a334-435d-82e5-ab14821bd937", APIVersion:"apps/v1", ResourceVersion:"582", FieldPath:""}, Reason:"FailedCreate", Message:"Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": http2: server sent GOAWAY
and closed the connection; LastStreamID=1011, ErrCode=NO_ERROR, debug=\"\"", Source:v1.EventSource{Component:"replicaset-controller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc18af7f29a491b3e, ext:142090168731, loc:(*time.Location)(0x632eb80)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc18af7f29a491b3e, ext:142090168731, loc:(*time.Location)(0x632eb80)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Post "https://192.168.76.2:8443/api/v1/namespaces/kube-system/events": dial tcp 192.168.76.2:8443: connect: connection refused'(may retry after sleeping)
E0520 15:12:10.447227 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
I0520 15:12:10.453179 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": dial tcp 192.168.76.2:8443: connect: connection refused"
E0520 15:12:10.453360 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
I0520 15:12:10.466872 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": dial tcp 192.168.76.2:8443: connect: connection refused"
E0520 15:12:10.467377 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
I0520 15:12:10.490553 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": dial tcp 192.168.76.2:8443: connect: connection refused"
E0520 15:12:10.490790 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
I0520 15:12:10.534643 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": dial tcp 192.168.76.2:8443: connect: connection refused"
E0520 15:12:10.534907 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
I0520 15:12:10.616881 1 event.go:291] "Event occurred" object="kube-system/metrics-server-9975d5f86" kind="ReplicaSet" apiVersion="apps/v1" type="Warning" reason="FailedCreate" message="Error creating: Post \"https://192.168.76.2:8443/api/v1/namespaces/kube-system/pods\": dial tcp 192.168.76.2:8443: connect: connection refused"
E0520 15:12:10.617332 1 replica_set.go:532] sync "kube-system/metrics-server-9975d5f86" failed with Get "https://192.168.76.2:8443/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-9975d5f86": dial tcp 192.168.76.2:8443: connect: connection refused
==> kube-proxy [36b9b4fb305d] <==
I0520 15:10:15.895560 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0520 15:10:15.895672 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0520 15:10:15.929982 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0520 15:10:15.930765 1 server_others.go:185] Using iptables Proxier.
I0520 15:10:15.937077 1 server.go:650] Version: v1.20.0
I0520 15:10:15.939462 1 config.go:315] Starting service config controller
I0520 15:10:15.940513 1 shared_informer.go:240] Waiting for caches to sync for service config
I0520 15:10:15.939677 1 config.go:224] Starting endpoint slice config controller
I0520 15:10:15.941368 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0520 15:10:16.040890 1 shared_informer.go:247] Caches are synced for service config
I0520 15:10:16.041589 1 shared_informer.go:247] Caches are synced for endpoint slice config
==> kube-proxy [75eb680f6808] <==
I0520 15:12:47.652748 1 node.go:172] Successfully retrieved node IP: 192.168.76.2
I0520 15:12:47.653075 1 server_others.go:142] kube-proxy node IP is an IPv4 address (192.168.76.2), assume IPv4 operation
W0520 15:12:47.686772 1 server_others.go:578] Unknown proxy mode "", assuming iptables proxy
I0520 15:12:47.687101 1 server_others.go:185] Using iptables Proxier.
I0520 15:12:47.687484 1 server.go:650] Version: v1.20.0
I0520 15:12:47.688418 1 config.go:315] Starting service config controller
I0520 15:12:47.688582 1 shared_informer.go:240] Waiting for caches to sync for service config
I0520 15:12:47.688699 1 config.go:224] Starting endpoint slice config controller
I0520 15:12:47.688796 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0520 15:12:47.792180 1 shared_informer.go:247] Caches are synced for endpoint slice config
I0520 15:12:47.792242 1 shared_informer.go:247] Caches are synced for service config
==> kube-scheduler [20ae03428247] <==
I0520 15:12:40.276296 1 serving.go:331] Generated self-signed cert in-memory
W0520 15:12:44.841735 1 requestheader_controller.go:193] Unable to get configmap/extension-apiserver-authentication in kube-system. Usually fixed by 'kubectl create rolebinding -n kube-system ROLEBINDING_NAME --role=extension-apiserver-authentication-reader --serviceaccount=YOUR_NS:YOUR_SA'
W0520 15:12:44.841872 1 authentication.go:332] Error looking up in-cluster authentication configuration: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
W0520 15:12:44.841904 1 authentication.go:333] Continuing without authentication configuration. This may treat all requests as anonymous.
W0520 15:12:44.841962 1 authentication.go:334] To require authentication configuration lookup to succeed, set --authentication-tolerate-lookup-failure=false
I0520 15:12:45.242731 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0520 15:12:45.242878 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0520 15:12:45.242886 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0520 15:12:45.242901 1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0520 15:12:45.347311 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kube-scheduler [798f34afc547] <==
I0520 15:09:55.003706 1 secure_serving.go:197] Serving securely on 127.0.0.1:10259
I0520 15:09:55.009130 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0520 15:09:55.009189 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0520 15:09:55.009261 1 tlsconfig.go:240] Starting DynamicServingCertificateController
E0520 15:09:55.057555 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0520 15:09:55.057823 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0520 15:09:55.058020 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0520 15:09:55.058183 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StorageClass: failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0520 15:09:55.058301 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
E0520 15:09:55.058318 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0520 15:09:55.058494 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSINode: failed to list *v1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0520 15:09:55.058724 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0520 15:09:55.058730 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0520 15:09:55.059842 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0520 15:09:55.058837 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0520 15:09:55.058905 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0520 15:09:55.955119 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0520 15:09:55.989119 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolumeClaim: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0520 15:09:56.049536 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0520 15:09:56.097796 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0520 15:09:56.107155 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicationController: failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0520 15:09:56.122994 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.PersistentVolume: failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0520 15:09:56.137761 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.ReplicaSet: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0520 15:09:56.181070 1 reflector.go:138] k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206: Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps "extension-apiserver-authentication" is forbidden: User "system:kube-scheduler" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
I0520 15:09:58.309331 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
==> kubelet <==
May 20 15:16:22 old-k8s-version-185009 kubelet[1198]: E0520 15:16:22.118599 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:16:26 old-k8s-version-185009 kubelet[1198]: E0520 15:16:26.125485 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:16:35 old-k8s-version-185009 kubelet[1198]: E0520 15:16:35.118669 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:16:37 old-k8s-version-185009 kubelet[1198]: E0520 15:16:37.119240 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:16:49 old-k8s-version-185009 kubelet[1198]: E0520 15:16:49.118660 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:16:51 old-k8s-version-185009 kubelet[1198]: E0520 15:16:51.118889 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:01 old-k8s-version-185009 kubelet[1198]: E0520 15:17:01.131117 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:02 old-k8s-version-185009 kubelet[1198]: E0520 15:17:02.119112 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118969 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:14 old-k8s-version-185009 kubelet[1198]: E0520 15:17:14.118971 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:25 old-k8s-version-185009 kubelet[1198]: E0520 15:17:25.119102 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:27 old-k8s-version-185009 kubelet[1198]: E0520 15:17:27.141000 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:37 old-k8s-version-185009 kubelet[1198]: E0520 15:17:37.118917 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:42 old-k8s-version-185009 kubelet[1198]: E0520 15:17:42.119257 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:48 old-k8s-version-185009 kubelet[1198]: E0520 15:17:48.118714 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:17:54 old-k8s-version-185009 kubelet[1198]: E0520 15:17:54.118780 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:17:59 old-k8s-version-185009 kubelet[1198]: E0520 15:17:59.122757 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:18:07 old-k8s-version-185009 kubelet[1198]: E0520 15:18:07.136708 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:18:13 old-k8s-version-185009 kubelet[1198]: E0520 15:18:13.119243 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ImagePullBackOff: "Back-off pulling image \"fake.domain/registry.k8s.io/echoserver:1.4\""
May 20 15:18:18 old-k8s-version-185009 kubelet[1198]: E0520 15:18:18.118661 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
May 20 15:18:25 old-k8s-version-185009 kubelet[1198]: E0520 15:18:25.148188 1198 remote_image.go:113] PullImage "fake.domain/registry.k8s.io/echoserver:1.4" from image service failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
May 20 15:18:25 old-k8s-version-185009 kubelet[1198]: E0520 15:18:25.148233 1198 kuberuntime_image.go:51] Pull image "fake.domain/registry.k8s.io/echoserver:1.4" failed: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
May 20 15:18:25 old-k8s-version-185009 kubelet[1198]: E0520 15:18:25.148387 1198 kuberuntime_manager.go:829] container &Container{Name:metrics-server,Image:fake.domain/registry.k8s.io/echoserver:1.4,Command:[],Args:[--cert-dir=/tmp --secure-port=4443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-status-port --metric-resolution=60s --kubelet-insecure-tls],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:4443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{cpu: {{100 -3} {<nil>} 100m DecimalSI},memory: {{209715200 0} {<nil>} BinarySI},},},VolumeMounts:[]VolumeMount{VolumeMount{Name:tmp-dir,ReadOnly:false,MountPath:/tmp,SubPath:,MountPropagation:nil,SubPathExpr:,},VolumeMount{Name:metrics-server-token-lnh45,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,},},LivenessProbe:&Probe{Handler:Handler{Exe
c:nil,HTTPGet:&HTTPGetAction{Path:/livez,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,} start failed in pod metrics-server-9975d5f86-sks2q_kube-system(8a4fa9
23-b0b9-460e-9c5c-4dd8330c5815): ErrImagePull: rpc error: code = Unknown desc = Error response from daemon: Get "https://fake.domain/v2/": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host
May 20 15:18:25 old-k8s-version-185009 kubelet[1198]: E0520 15:18:25.148423 1198 pod_workers.go:191] Error syncing pod 8a4fa923-b0b9-460e-9c5c-4dd8330c5815 ("metrics-server-9975d5f86-sks2q_kube-system(8a4fa923-b0b9-460e-9c5c-4dd8330c5815)"), skipping: failed to "StartContainer" for "metrics-server" with ErrImagePull: "rpc error: code = Unknown desc = Error response from daemon: Get \"https://fake.domain/v2/\": dial tcp: lookup fake.domain on 192.168.76.1:53: no such host"
May 20 15:18:29 old-k8s-version-185009 kubelet[1198]: E0520 15:18:29.129576 1198 pod_workers.go:191] Error syncing pod f03f4535-8bca-4657-baaf-e36f3e155b1e ("dashboard-metrics-scraper-8d5bb5db8-5757j_kubernetes-dashboard(f03f4535-8bca-4657-baaf-e36f3e155b1e)"), skipping: failed to "StartContainer" for "dashboard-metrics-scraper" with ImagePullBackOff: "Back-off pulling image \"registry.k8s.io/echoserver:1.4\""
==> kubernetes-dashboard [e9d80f29a80f] <==
2024/05/20 15:13:11 Using namespace: kubernetes-dashboard
2024/05/20 15:13:11 Using in-cluster config to connect to apiserver
2024/05/20 15:13:11 Using secret token for csrf signing
2024/05/20 15:13:11 Initializing csrf token from kubernetes-dashboard-csrf secret
2024/05/20 15:13:11 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2024/05/20 15:13:11 Successful initial request to the apiserver, version: v1.20.0
2024/05/20 15:13:11 Generating JWE encryption key
2024/05/20 15:13:11 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2024/05/20 15:13:11 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2024/05/20 15:13:11 Initializing JWE encryption key from synchronized object
2024/05/20 15:13:11 Creating in-cluster Sidecar client
2024/05/20 15:13:11 Serving insecurely on HTTP port: 9090
2024/05/20 15:13:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:13:41 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:14:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:14:41 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:15:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:15:41 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:16:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:16:41 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:17:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:17:41 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:18:11 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2024/05/20 15:13:11 Starting overwatch
==> storage-provisioner [623598d9bb43] <==
I0520 15:12:47.579092 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
F0520 15:13:17.584084 1 main.go:39] error getting server version: Get "https://10.96.0.1:443/version?timeout=32s": dial tcp 10.96.0.1:443: i/o timeout
==> storage-provisioner [ee9aab3e1ca5] <==
I0520 15:13:34.479441 1 storage_provisioner.go:116] Initializing the minikube storage provisioner...
I0520 15:13:34.512149 1 storage_provisioner.go:141] Storage provisioner initialized, now starting service!
I0520 15:13:34.513217 1 leaderelection.go:243] attempting to acquire leader lease kube-system/k8s.io-minikube-hostpath...
I0520 15:13:52.028155 1 leaderelection.go:253] successfully acquired lease kube-system/k8s.io-minikube-hostpath
I0520 15:13:52.028616 1 controller.go:835] Starting provisioner controller k8s.io/minikube-hostpath_old-k8s-version-185009_664bc4cf-aa3d-4bd0-8552-83d0ca5b0c48!
I0520 15:13:52.033255 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"k8s.io-minikube-hostpath", UID:"436e75b2-65bf-45eb-953b-cca248352553", APIVersion:"v1", ResourceVersion:"832", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' old-k8s-version-185009_664bc4cf-aa3d-4bd0-8552-83d0ca5b0c48 became leader
I0520 15:13:52.130844 1 controller.go:884] Started provisioner controller k8s.io/minikube-hostpath_old-k8s-version-185009_664bc4cf-aa3d-4bd0-8552-83d0ca5b0c48!
-- /stdout --
helpers_test.go:254: (dbg) Run: out/minikube-linux-arm64 status --format={{.APIServer}} -p old-k8s-version-185009 -n old-k8s-version-185009
helpers_test.go:261: (dbg) Run: kubectl --context old-k8s-version-185009 get po -o=jsonpath={.items[*].metadata.name} -A --field-selector=status.phase!=Running
helpers_test.go:272: non-running pods: metrics-server-9975d5f86-sks2q dashboard-metrics-scraper-8d5bb5db8-5757j
helpers_test.go:274: ======> post-mortem[TestStartStop/group/old-k8s-version/serial/SecondStart]: describe non-running pods <======
helpers_test.go:277: (dbg) Run: kubectl --context old-k8s-version-185009 describe pod metrics-server-9975d5f86-sks2q dashboard-metrics-scraper-8d5bb5db8-5757j
helpers_test.go:277: (dbg) Non-zero exit: kubectl --context old-k8s-version-185009 describe pod metrics-server-9975d5f86-sks2q dashboard-metrics-scraper-8d5bb5db8-5757j: exit status 1 (116.711017ms)
** stderr **
Error from server (NotFound): pods "metrics-server-9975d5f86-sks2q" not found
Error from server (NotFound): pods "dashboard-metrics-scraper-8d5bb5db8-5757j" not found
** /stderr **
helpers_test.go:279: kubectl --context old-k8s-version-185009 describe pod metrics-server-9975d5f86-sks2q dashboard-metrics-scraper-8d5bb5db8-5757j: exit status 1
--- FAIL: TestStartStop/group/old-k8s-version/serial/SecondStart (374.40s)